Comscore’s Google Numbers Are Too Low

A New York Times article compares the market position of some Google products to its competitors [via Techcrunch] and notes that they’re often trailing the competition by quite a bit. Check out this graph and judge for yourself. The numbers are based on Comscore Media Metrix.

While some bloggers are defending Google furiously, others are joining the bashing. As usual the truth is probably somewhere in between.

Two things are important that I haven’t really seen in any of the comments:

Trends matter

Trends matter more than current numbers. Several products are fairly new. What really matters is if they’re still growing and how fast. Being number 10 today is ok if you’re number 1 tomorrow. Comparing Comscore numbers over a period of time should give some indication here.

Comscore’s methodology is biased

Google has a clear disadvantage when it comes to how Comscore calculates its numbers (except for search). Comscore’s numbers are based on a panel. A piece of software is installed on the participants’ computer. It monitors every URL visited and sends a report back to Comscore. The total numbers are achieved by extrapolating the actual numbers.

I’m convinced that advanced Internet users are underrepresented in this panel. Or how do you like the idea of having a HTTP proxy that monitors all your traffic. Now this is exactly the crowd that uses Google’s services more than the average user. Therefore I expect these numbers to be biased in favor of the more main stream competitors. By how much is hard to tell though.

Related reading: Google Backlash – Round 2

Update 12/28/06: Fred Wilson, a board member of comScore, explains the bias:

Comscore is a “consumer panel”. It measures mainstream web users. It is not a “leading edge” panel and it will almost certainly undercount “geek” services like Delicious and Digg.