We’re Still in the First Generation of Search Engines. Here’s What Could Be Next.
Run a search for anything right now, using Google or your preferred search engine. Pay attention to the features of the search engine results pages (SERPs) you encounter. How fast did they appear? What do they look like? What kinds of information are you presented with?
If you ran a similar search in, say 2002, how different would the layout look? How different would the experience be?
As someone who was conducting lots of searches in 2002, I can tell you we’ve come a long way in terms of design, usability, and overall performance. And yet, I’d still say we’re in the “first generation” of search engines.
How can I make this claim? And if this is the first generation, what would the second generation look like?
A Short History of Search Engines
Let’s start with a short history of search engines. How did we get to this point and how much innovation have we truly seen?
Search engines began to emerge in the 1990s but exploded in popularity with the launch of Google in 1997. Within a few years, Google’s seamless and minimalistic experience led it to be the search engine of choice for most of the population online. For the record, it remains the most popular search engine in the world by far (despite a handful of interesting new competitors, like DuckDuckGo).
Originally, Google’s search algorithm functioned with an algorithm called PageRank. It would estimate the perceived authority, or trustworthiness, of various sites based on the links pointing to them – including the quality and quantity of such links. Sites with lots of inbound links from other high-authority sources could be considered trustworthy and were ranked higher. Additionally, Google would consider the context of each webpage, using user queries as a guide to find on-topic content.
This led to heavy abuse from webmasters around the world. Stuffing keywords and spamming links were common ways to manipulate the system and ultimately rank higher than the competition.
In response, Google began releasing a series of regular updates, all designed to improve the average user’s search experience and provide them with better results. Many of these updates improved the quality standards used by Google to calculate relevance and authority of websites; for example, clear link spam became disregarded (with the link spammers penalized) and good content became rewarded more than bad content.
Other updates sought to make the search experience more robust. Over time, Google has expanded the search engine’s functionality and purpose with local search, business entries, reviews, images, embedded videos, news stories, and sometimes, direct information on the topic you’re interested in. There are even AI-based elements of Google search that continually self-improve to better serve users. These all functionally expand the capacity of search without interfering with the core experience.
Because of changes like these, Google is a totally different beast than it was in 1997. But in some ways, it hasn’t changed much at all.
The Core Search Experience
This is why I believe we’re still well within the bounds of the “first generation” of online search. We can do a lot more with search now than we have before – but the core mechanics of the search experience are still instantly recognizable, and in many ways, function just as they did 20+ years ago.
For example, when you want to search for something online, you still (typically) pull up a search engine, enter your query, and browse through a list of entries to find what you’re looking for. There are growing exceptions to this, which I’ll cover in the next section, but this is the primary search experience and the way we typically think about searching.
When Google (and other search engines) fetch results, they still consider the same broad criteria: relevance and authority. The parameters for what constitutes relevance and authority may have evolved, but the basics are the same as they’ve ever been. A site with lots of inbound, authoritative links and contextually relevant material will easily climb to the top of the SERPs.
Similarly, as a user, you can generally expect the same kind of experience when perusing results. You can take a look (or listen) at Google’s top suggestion for your query, or leaf through the myriad results that also turn up for your search.
We’ve come to expect that this is the only real way to search for things online. But could a second generation of search fundamentally change how this works?
Hints at the Second Generation
Right now, there are some promising candidates that could fundamentally change how we search; they’re peppered into our existing, core search experience.
- The Knowledge Graph and rich answers. Over the past few years, Google has stepped up its efforts to provide users with direct answers, rather than having them comb through websites. The Knowledge Graph sometimes gives you immediate information on your topic of choice. Other times, you’re presented with a “rich answer” pulled directly from a website that Google feels is most authoritative on the subject. In the future, we may be doing less browsing, and instead accepting Google’s best choices – at least with some topics.
- Smart speakers and voice search. Smart speakers with personal digital assistants like Alexa are another potential avenue for development. Voice search evolved from becoming a gimmicky, annoying, and incompetent function to being seamlessly integrated into our conversations when we want it. Voice-based search, if it advances, could introduce us to new ways of browsing the internet and engaging with online content. It could even open the door to other physical modes of search, such as searching with gestures, eye movements, or body language.
- The internet of things and distributed search functionality. The internet of things is also seeing so much momentum that the term itself is falling out of the common lexicon. Most households in the United States have dozens of digital, internet-connected devices, all of which can search in their own way. A distributed search feature, or one that can be easily used from device to device, could be in our future.
- Data and personalization. We’ve also seen big increases in the reliance of search engines on personal data. Search results are formulated in part based on your personal history and interests. In the future, this may replace PageRank-style systems as the primary mode of consideration when formulating results, sparking the growth of fundamentally new, second-generation engines.
- Complete ground-up innovation. What’s most exciting is the promise of a complete, ground-up innovation, forcing us to rethink search entirely. Such a move would require a massive leap forward in terms of technology and a lot of luck – since many people would be reluctant to transition to a new system. This may prove difficult with the current state of the internet, but if websites and pages are fundamentally upgraded or redrawn, the way we search will surely follow.
However, there are some caveats to keep in mind when considering the next generation of search. Here’s a big one: most companies aren’t interested in starting completely from scratch. Google has nearly 25 years of data, experience, and investment in a very particular mode of search. While they could carry over some of it to draw up plans for a new type of search
A Vision of the Future of Search
So what does the future of search look like? It’s hard to say. There’s a good chance we’ll remain in this first generation, with iterative tweaks and improvements, for another decade or two. But sooner or later, some genius entrepreneur will come along to disrupt the way we do things – or maybe one of the existing tech giants will step up to teach the world how to search in a new way, from the ground up. And when that time comes, we’ll look back at our current search capabilities with the same humor and nostalgia that we look back on film projectors and VHS tapes.