Facebook flyers experiment – Pt. 2

October 12, 2007

(See Facebook Flyers Experiment Pt 1 for background info)

After running the facebook flyer campaigns for a week, the results are out. To recap, the five campaigns were:

  1. All high school students (ages 13-19) with $0.05 per click.
  2. All high school students (ages 13-19) with $0.03 per click
  3. All people between the ages of 17-40 who live in the top technology cities as identified by Wired and some others (Seattle, San Francisco, LA, Austin, Orlando, New York, Boston, Philadelphia, Washington DC, Pittsburgh & Chicago)
  4. All college students attending some of the top technical schools (MIT, Stanford, Harvard, Princeton, UPenn, Columbia, Cornell, Carnegie Mellon, Illinois, U. Texas, U. Maryland, Georgia Tech, Cal Tech, Berkeley, UCLA & Penn State).
  5. All college students majoring in Engineering and otherwise technical majors (engineering disciplines, computer science, web design, etc)

As expected, most impressions occurred in campaigns 1 and 2, high school students, being these were the least specific, hence most probably the largest, user group targeted. What was also interesting to note was that there was a negligible difference in impressions between $0.03 and $0.05 campaigns (6.35% more impressions on the 0.5 cents per click) indicating that the demand for targeting these groups is less than the supply (i.e. number of impressions being demanded are far less than the number of potential impressions). In an ideal world, the number of impressions for the $0.05 per click campaign would be 40% more.

The second most impressions came from college students in technology majors, followed by adults in “the top technology cities” and then college students in the top technology schools. Again these are directly related to the specificity of the groups targeted.

As for the click through rates? They were low, atrociously low. The most clicks for any campaign were a whopping 1 click! The table below breaks down the CTRs for each campaign:

Campaign Impressions CTR (%)
1 5,904 0
2 5,551 0.018
3 894 0
4 852 0.117
5 1001 0.0999

Clickthrough rates of 0% to 0.1% were about as low as predicted. No conclusive statements can be made about which campaigns were better targeted due to the incredibly low number of clicks.

Being avid facebook users ourselves, we rarely find ourselves clicking on such flyers. In analyzing our own actions, we hypothesized some possible reasons to explain these poor click through rates:

  1. Facebook users are rarely in search for “external” information (information not available within Facebook) as opposed to say when one is searching on Google.
  2. Users find Facebook content far too engaging to click on a link that will direct them away from the site.
  3. The placement of the Facebook flyers is not at an optimum place on the pages.
  4. The flyer we created failed to capture the interest of the audience.

To perform a more rigorous study, one would need to run these campaigns for a much longer time than just a week. However, it is hard to imagine the CTR’s being significantly higher if the campaigns were to be prolonged (see Mashable post). It would be interesting to examine how using more captivating flyer designs (specifically targeted to each user group) would affect the click through rates, if at all. Needless to say, we are exploring other possible ways in which we can tap into the Facebook userbase as a means to generate traffic. The Youlicit team would love to hear from you if you have fared better than us, in terms of CTRs, with Facebook Flyers or are interested in sharing ideas to better target Facebook users.

On a side note, for all campaigns, the highest impressions occurred on a Friday, Saturday and Sunday – on average 86.8% of the total impressions – highly indicative of Facebook usage habits.


Facebook flyers experiment – Pt. 1

October 4, 2007

Facebook recently launched an updated version of their Flyers product called Flyers Pro. The main difference between this and the now Flyers Basic product is that advertisers can now pay per click instead of paying per impression.

In theory, advertising on Facebook has immense appeal. Given its incredible user base (now over 30 million active users) and extremely high visitor-return frequency (ranking second in visits/visitor) it is an advertiser’s haven. With Facebook’s improved targeting options – you can now filter on keywords (“Beatles”, “Lord of the Rings”, “iPhone”), countries, cities, age, workplace (“Google”, “Goldman Sachs”) and colleges – advertisers can now (theoretically) target their ads much better and reach the exact demographics they are looking for. Social networking sites, however, are infamous for their abysmal click-through rates (ranging from 0.01% to less than 1%). Unheeding these statistics, however, we decided to try out this new platform.

To test out the new Flyers platform, we created several campaigns to see:

  1. what demographics get us the most impressions for a given cost per click (CPC), and
  2. what demographics have the highest click through rates (CTRs).

The first question is to gauge the current demand of advertising to certain demographics on Facebook (independent of product and creative) and the second question is for us to see which Facebook demographics Youlicit appeals to the most (highly dependent on the ad, the placement of the ad, the product being advertised and the audience). The different demographics we targeted were as follows:

  • All high school students (ages 13-19)
  • All college students majoring in Engineering and otherwise technical majors (engineering disciplines, computer science, web design, etc)
  • All college students attending some of the top technical schools (MIT, Stanford, Harvard, Princeton, UPenn, Columbia, Cornell, Carnegie Mellon, Illinois, U. Texas, U. Maryland, Georgia Tech, Cal Tech, Berkeley, UCLA & Penn State). Nothing personal if your school isn’t listed, this was just a random sampling.
  • All people between the ages of 17-40 who live in the top technology cities as identified by Wired and some others (Seattle, San Francisco, LA, Austin, Orlando, New York, Boston, Philadelphia, Washington DC, Pittsburgh & Chicago)

We created this flyer to be used on all the campaigns:

Facebook Flyer

Given our current download rates per visit on Youlicit, we set our maximum costs per click to $0.03 in order to keep our user acquisition costs to under $1/user (however we did set 2 campaigns for the high school demographics to compare number of impressions for CPC’s of $0.05 and $0.03). We launched these campaigns this week for a period of 7 days and will see how each one fares after that. Stay tuned for the results…

(see the results here)


What Comes After Google?

October 4, 2007

Question: What Comes After Google?

Yahoo just released a new Search Assistant feature this week (TechCrunch) (Read/WriteWeb). Ask has been trying a new interface lift for a while (TechCrunch) (Read/WriteWeb). While these are all very nice incremental improvements to search, are they enough to supplant Google? Do they tackle the fundamental problem of information retrieval in a paradigm shifting way? The answer is probably not.

Now imagine several years into the future. Will you find information in the same way in the future as you do today? Again, probably not.

This may sound like an obvious “duh”-ism, but its ramification certainly is not. As unfathomable as it may seem, Google, as we know it today, will probably not be how we find most of our information in a few years. Since Youlicit is an information retrieval company, we had to ask ourselves, “If not Google, then what?”

What is the logic that dictates the evolution of information retrieval paradigms?

Evolution of Information Retrieval Paradigms

To answer this question, we first plotted the different paradigms of information retrieval on a timeline. If we can figure out what the axis of this graph represents, then we should be able to predict which new solutions will succeed and which will not by simply identifying the solutions that maximize the metric along this axis.

If you’re a start-up, this understanding can guide you in building a successful innovative product. If you’re a venture capitalist or technology evaluator, this insight gives you a criterion for determining which technologies to invest in and which ones will fade away as fads of the day.

Evolution of Information Retrieval Paradigms

After plotting them on a timeline, we then explored the three major paradigms of information retrieval:

  1. Manual Organization
  2. Algorithmic Search
  3. User-Generated Recommendations

Manual Organization

Information retrieval, during its infancy, started off as a very rigid and structured process. Those who remember Gopher or Jerry and David’s Guide to the World Wide Web (later known as Yahoo) know how attempts were made to organize sites into a pre-determined hierarchy. However, as the number of web sites exploded exponentially, manually organizing sites into structured directories became practically impossible:

“A universal ontology is difficult and expensive to construct and maintain when there involve hundreds of millions of users with diverse background. When used to organize Web objects, ontology faces two hard problems: unlike physical objects, digital content is seldom semantically pure to fit in a specific category; and it is difficult to predict the paths, through which a user would explore to discover a digital object.”
Clay Shirkey

Algorithmic Search

Too many sites to categorize? No problem. Algorithmic search to the rescue. Web search engines, such as Altavista and Google, arrived and allowed the web to grow in its chaotic unstructured way while still providing a level of organization in the form of keywords. Now instead of having to know the correct directory hierarchy, users only needed to know the keyword combination (and page number) for sites they were looking for.

Counter to intuition, search engines actually decrease the relevance of individual results as compared to those in a manually organized directory. A hand-picked set of results are always better than an algorithmically generated set of results. However, since search engines have a much greater coverage of the Web, the average relevance of search results from a given set of topics is usually better than the average relevance of directory results on the same set of topics.

The other improvement made by search was the replacement of directory hierarchies by keywords as the primary recall mechanism. While still not perfect, guessing and checking keywords took a lot less effort than guessing and checking hierarchies. Seach engines effectively decreased the recall effort.

User-Generated Recommendations

Recently, we’ve witnessed the niche adoption of tagging, voting, stumbling and other “user-generated relevance” as a means of finding information. Why? It’s because they improved something along either the average relevance dimension or the recall effort dimension.

Take Del.icio.us and Digg for instance. In the scope of technology related content, the average relevance of results from these folksonomy sites is better than from search engines because these folksonomy sites have been able to increase coverage by effectively crowdsourcing an easier manual organization process.

StumbleUpon went the other route. Instead of improving average relevance, it decided to reduce the recall effort from guessing and checking keywords to a one-click no-thought “stumble.” In doing so, it did something ingenious: StumbleUpon removed the world’s most scarce resource from the information retrieval process… human thought.

Answer: The Solutions that Maximizes the “Search Metric”

As well as they’ve done, both kinds of user-generated recommendation services are plateauing well before crossing the chasm into the mainstream market. Why? We think it’s because they’ve only focused on a singular improvement, either average relevance or recall effort, but not both.

In order for an information retrieval solution to penetrate into the mass market, the solution has to take a dual approach. It has to concurrently maximize average relevance and minimize recall effort. It’s simply a matter of optimizing the Average Relevance / Recall Effort ratio, or as we like to call it, the “Search Metric.” The solution that does this best will probably gain the most mindshare and supplant algorithmic search as the primary mode of information retrieval. And that is what comes after Google.

Afterthought

Does this imply that algorithmic search will become extinct anytime soon? Absolutely not. It just means that more and more people will find larger percentages of their daily information through means other than search. Our bets are on online “word of mouth” or user-generated recommendations.

Evolution of Information Retrieval Paradigms

We’ve builit Youlicit with this assumption at the core. Youlicit is a “word of mouth” or recommendation engine (as opposed to a search engine). We’re trying to maximize the Search Metric by combining user-generated relevance with one-click no-thought recall. We want to improve the information retrieval landscape by enabling the average user to harness the wisdom of the crowds with very little effort. If you’re as obsessed with this problem as we are, we’d love nothing more than to hear from you!

— The Youlicit Team