Question: What Comes After Google?
Yahoo just released a new Search Assistant feature this week (TechCrunch) (Read/WriteWeb). Ask has been trying a new interface lift for a while (TechCrunch) (Read/WriteWeb). While these are all very nice incremental improvements to search, are they enough to supplant Google? Do they tackle the fundamental problem of information retrieval in a paradigm shifting way? The answer is probably not.
Now imagine several years into the future. Will you find information in the same way in the future as you do today? Again, probably not.
This may sound like an obvious “duh”-ism, but its ramification certainly is not. As unfathomable as it may seem, Google, as we know it today, will probably not be how we find most of our information in a few years. Since Youlicit is an information retrieval company, we had to ask ourselves, “If not Google, then what?”
What is the logic that dictates the evolution of information retrieval paradigms?
Evolution of Information Retrieval Paradigms
To answer this question, we first plotted the different paradigms of information retrieval on a timeline. If we can figure out what the axis of this graph represents, then we should be able to predict which new solutions will succeed and which will not by simply identifying the solutions that maximize the metric along this axis.
If you’re a start-up, this understanding can guide you in building a successful innovative product. If you’re a venture capitalist or technology evaluator, this insight gives you a criterion for determining which technologies to invest in and which ones will fade away as fads of the day.
After plotting them on a timeline, we then explored the three major paradigms of information retrieval:
- Manual Organization
- Algorithmic Search
- User-Generated Recommendations
Information retrieval, during its infancy, started off as a very rigid and structured process. Those who remember Gopher or Jerry and David’s Guide to the World Wide Web (later known as Yahoo) know how attempts were made to organize sites into a pre-determined hierarchy. However, as the number of web sites exploded exponentially, manually organizing sites into structured directories became practically impossible:
“A universal ontology is difficult and expensive to construct and maintain when there involve hundreds of millions of users with diverse background. When used to organize Web objects, ontology faces two hard problems: unlike physical objects, digital content is seldom semantically pure to fit in a specific category; and it is difficult to predict the paths, through which a user would explore to discover a digital object.”
— Clay Shirkey
Too many sites to categorize? No problem. Algorithmic search to the rescue. Web search engines, such as Altavista and Google, arrived and allowed the web to grow in its chaotic unstructured way while still providing a level of organization in the form of keywords. Now instead of having to know the correct directory hierarchy, users only needed to know the keyword combination (and page number) for sites they were looking for.
Counter to intuition, search engines actually decrease the relevance of individual results as compared to those in a manually organized directory. A hand-picked set of results are always better than an algorithmically generated set of results. However, since search engines have a much greater coverage of the Web, the average relevance of search results from a given set of topics is usually better than the average relevance of directory results on the same set of topics.
The other improvement made by search was the replacement of directory hierarchies by keywords as the primary recall mechanism. While still not perfect, guessing and checking keywords took a lot less effort than guessing and checking hierarchies. Seach engines effectively decreased the recall effort.
Recently, we’ve witnessed the niche adoption of tagging, voting, stumbling and other “user-generated relevance” as a means of finding information. Why? It’s because they improved something along either the average relevance dimension or the recall effort dimension.
Take Del.icio.us and Digg for instance. In the scope of technology related content, the average relevance of results from these folksonomy sites is better than from search engines because these folksonomy sites have been able to increase coverage by effectively crowdsourcing an easier manual organization process.
StumbleUpon went the other route. Instead of improving average relevance, it decided to reduce the recall effort from guessing and checking keywords to a one-click no-thought “stumble.” In doing so, it did something ingenious: StumbleUpon removed the world’s most scarce resource from the information retrieval process… human thought.
Answer: The Solutions that Maximizes the “Search Metric”
As well as they’ve done, both kinds of user-generated recommendation services are plateauing well before crossing the chasm into the mainstream market. Why? We think it’s because they’ve only focused on a singular improvement, either average relevance or recall effort, but not both.
In order for an information retrieval solution to penetrate into the mass market, the solution has to take a dual approach. It has to concurrently maximize average relevance and minimize recall effort. It’s simply a matter of optimizing the Average Relevance / Recall Effort ratio, or as we like to call it, the “Search Metric.” The solution that does this best will probably gain the most mindshare and supplant algorithmic search as the primary mode of information retrieval. And that is what comes after Google.
Does this imply that algorithmic search will become extinct anytime soon? Absolutely not. It just means that more and more people will find larger percentages of their daily information through means other than search. Our bets are on online “word of mouth” or user-generated recommendations.
We’ve builit Youlicit with this assumption at the core. Youlicit is a “word of mouth” or recommendation engine (as opposed to a search engine). We’re trying to maximize the Search Metric by combining user-generated relevance with one-click no-thought recall. We want to improve the information retrieval landscape by enabling the average user to harness the wisdom of the crowds with very little effort. If you’re as obsessed with this problem as we are, we’d love nothing more than to hear from you!
— The Youlicit Team