Category Archives: metrics

How to improve PG&E’s usage and charges graph

I was pleasantly surprised that PG&E includes graphs for usage and billing history on their website. You can view electricity usage and charges in a pretty neat (but simple) graph.

I’ve been living in my current apartment since March 2007 which is the first data point on the x-axis. I wonder if there is a limit to how many months they show?

Making it easier to compare

The electricity usage is obviously seasonal and you can see the ups and downs in the graph. It would make more sense to have a 1 year wide graph and overlay last years data over the current data in a line or area chart. This would make the comparison easier and more meaningful. Alternatively a 12-months trend line would provide additional information but with a significant delay.

I created the above graph using Google Docs. I like the look and it makes comparing the charges from one year to the next a lot easier. Even adding another year shouldn’t impact readability. Now I have to find out why I’m paying 15-20% more this year.

It would also be interesting to compare usage and charges in the same graph. Am I paying more because I’m using more or because electricity is more expensive?

What are other companies providing?

Not much. I would love to have similar graphs for my bank account, credit card and phone bill. In fact I initially started this post rambling about AT&T and how they make it really hard to look at usage history. I’ll save that for another post. Maybe I’ll whip out Greasemonkey and create a little script…

Update 7/8/08: I’ll have to check out Skydeck for my AT&T billing needs. Still in closed beta.

Visitors or Users? Myspace vs. Facebook Traffic

ReadWriteWeb picked up the latest Hitwise numbers that indicate that Myspace is still way bigger than Facebook in terms of traffic. Depending on which metric you’re looking at you end up with a very different picture though.

I’ve only recently started using Myspace and I use it exclusively to listen to music. I don’t even have a Myspace account. I consider myself more of a visitor to the site than a user of the ‘social network’ Myspace. However I’m a user of their ‘DJ/Band pages’ and a consumer of the online music.

I’m a pretty engaged Facebook user on the other hand. I have an account with over 300 friends and log in about 4 days a week if not more. I change my status, send messages and post on walls.

My engagement and therefore user value is clearly higher on Facebook.

I think Hitwise could measure active accounts by looking at the subset of URLs that is only accessible when logged in and report that number separately from overall traffic to the entire site.

Have You Missed This?

I haven’t posted here in a while. Busy and stuff… You know how the story goes. And I was without Internet access at home for a while. Ah, good times!

Exciting personal news: I finally moved back to San Francisco – after over 2.5 years in Mountain View. Yeah! Trying to get my life back. Commute has been working out so far. And spring in SF is nice. A shame the snowboard season in Tahoe was so crappy this year. :(

A few thoughts that I meant to post in the last few weeks:

  • Visits instead of page views: What do you do when your page view metric becomes meaningless (due to Ajax) and you’re restricted by the data provided by your installed client app? Settle for less: comScore introduces visits. This metric is fairer but doesn’t provide enough information around engagement.
  • Inaccurate metrics: Valleywag’s summary of comScore’s problematic approach and how the numbers will never be accurate.
  • Attention: Compete tries Attention as an engagement metric (basically glorified time spent). Seth’s take on it: meaningless.
  • Firefox computer: The other day I was rambling again how the value of “MY” computer has gone down dramatically. I just need “A” computer. As long as it has a decent Internet connection I have access to all my important data. Most of what I do is somewhere online today (or moving there). Pretty much what Toni describes as The Firefox Computer.
  • Internet access at work: A friend of mine doesn’t have Internet access at work. It blows my mind how this is still possible in 2007! A computer without Internet connection has become useless to me (see bullet above). Then again I work for an Internet company. I guess trust in your employees is still not the norm.
  • Disabled comments: I just disabled comments on this blog (which doesn’t make it a blog anymore according to some). I got a ton of spam comments and basically no real comments. True, I could have installed Akismet and contributed to the 1 billion blocked spam comments. But I’m just tired of dealing with it. If you want to comment on one of my posts, do it on your blog!

I’m back!

Conflicting Metrics Data – Again

I’ve written about the metrics companies quite a bit recently (see So ComScore Understands the Page View Problem? and Page Views Are Already Dead!. And comScore is getting a beating again. Today Techcrunch reports on new conflicting data between the two metrics companies Hitwise and comScore:

[Hitwise's] data is saying that this week, for the first time, Google Blogsearch surpassed Technorati in total visits.

(…) Comscore tells a much different story, and one that makes little sense given the facts.

As mentioned before, comScore skews towards “regular” consumers (rather than more advanced users/geeks). So Technorati has less geeky users? Hard to believe…

And Hitwise, what’s up with that 0.0023% vs. 0.0025% of total internet traffic? There must be a better way to represent these numbers! You know, one that looks less ridiculous.

Performancing Selling Out?

Performancing has managed to position itself as a valuable resource for bloggers offering advice on their blog, a blogger Firefox extension, and even a blog advertisement network. Now Techcrunch reports that they’re selling out to PayPerPost, a service that has received plenty of negative press and comments from bloggers recently.

I stopped reading the Performancing blog a while ago as it was too focussed on pro-bloggers. Some advice seemed to be more concerned about making money then adding value for the reader. So is the sell out really that much of a surprise?

And one comment for TechCrunch: We all know that you get exclusives. But why do you have to rub it in with “They will be announcing … tomorrow”? Or am I overly sensitive?

Good luck to Performancing! You have a good reputation to lose.

Update 1/5/07: Looks like the deal is off: Metrics goes Open Source – PayPerPost Deal is Off

So ComScore Understands the Page View Problem?

None of the metrics companies have made a public statement about the controversy surrounding the page view (PV) metric yet. However Fred Wilson’s comment indicates for the first time that they understand the problem and are working on a solution. Fred’s blog A VC is well read and I had no clue that he’s on the board of comScore.

He also stresses the fact that PVs should not be confused with number of ads seen:

Ajax and other more modern web technologies allow for new ads to be displayed without a page reload. Ad views can grow even as page views decline.

And he brings up another point that I haven’t really considered yet. More and more pages are integrating widgets and third party content. How should those be measured? Who should get credit for the page view? In fact comScore is probably tracking this as a full PV of the target server today if the request is triggered client-side and the response has content-type text/html. See Comscore’s Google Numbers Are Too Low for some technical details.

He wraps up with a promise:

The bottom line is a page view isn’t a page view anymore. It’s a lot more and a lot less. And we are going to come up with new measurement terms in 2007 that recognize this fact.

Google’s Adam Cutts was also joining the discussion a few days ago with his post Page view metrics? Bah, humbug!. Thanks for defending Yahoo! btw:

I want to come to Yahoo’s defense about something. A recent spate of reports says that Yahoo has been surpassed by various companies in terms of page views. Why is that relatively bogus? Because of Yahoo’s switch to AJAX for its mail.

More: Page Views Are Already Dead!

Update 12/28/06: Two more links related to Ajax and PVs: AJAX Affects Page Views (on O’Reilly Radar) and MySpace Passes Yahoo In Page Views But Not Audience (on search engine land)

Update 2 12/28/06: Fred Wilson defended comScore back in late October 06 noting that he’s an investor and board member (I must have missed that). He also points to the bias due to the panel approach that I also mentioned in Comscore’s Google Numbers Are Too Low.

Update 3 12/28/06: If comScore understands that the PV metric is obsolete and has to be replaced with something more meaningful, why is it still making the headline and first paragraph in their press release? To their credit, they’re including a comment about Ajax (but only further down):

Further, Yahoo’s increased integration of AJAX technology may have had a dampening effect on page views, as the technology enables real-time site updates without the need to refresh a page.

Page Views Are Already Dead!

The Ajaxian is asking: Death of Page View Metrics? based on Steve Rubel’s prediction that the metric has four years to live.

The page view metric (PV) is already dead! Not everyone got the memo though…

Page Views (PVs) have never been a great metric (better than hits though). It is easy to architect a site to generate more PVs. Mike Davidson explains how MySpace is abusing this. And it’s also easy to game the system for example with a hidden iframe.

In fact, a higher number of PVs indicates low usability. If more PVs was better than there would be no Ajax! Remember how online maps used to work? High PV count. Compare this to how they work now with only one PV.

Steve Rubel mentions several Google products but I believe Yahoo! Mail* could actually spearhead the change. Take the largest email provider with its huge number of PVs and see the numbers drop as more and more users migrate to the beta. I believe this should be visible in the October/November numbers – see ComScore. We will see a declining number of PVs while the number of users is growing.

This is not the first time this topic comes up. evhead, Zawondy and others wrote about it before. I think the geeks understand the problem. However, it takes more time for advertisers, press and Wall Street to understand that the game has changed. I believe the transition will happen in two stages:

  1. Page view metric is declared dead after losing significance over time
  2. A new metric is introduced as unique users and time spent are not sufficient to measure engagement

It will be a slow transition to a better metric. I sure hope it’s not going to take four years though!

I’m surprised by the reaction (or lack thereof) of the metrics companies, mainly ComScore and NetRatings. Their entire business model is based on the page view metric (unique users and time spent are closely related). They’re only slowly starting to understand and acknowledge the problem.

The only usable metric today is unique users/visitors. But even that metric has to be taken with a grain of salt. (Don’t get me started on the accuracy of their current metrics…)

* Disclaimer: I work for Yahoo! Mail. The thoughts above are obviously mine.

Comscore’s Google Numbers Are Too Low

A New York Times article compares the market position of some Google products to its competitors [via Techcrunch] and notes that they’re often trailing the competition by quite a bit. Check out this graph and judge for yourself. The numbers are based on Comscore Media Metrix.

While some bloggers are defending Google furiously, others are joining the bashing. As usual the truth is probably somewhere in between.

Two things are important that I haven’t really seen in any of the comments:

Trends matter

Trends matter more than current numbers. Several products are fairly new. What really matters is if they’re still growing and how fast. Being number 10 today is ok if you’re number 1 tomorrow. Comparing Comscore numbers over a period of time should give some indication here.

Comscore’s methodology is biased

Google has a clear disadvantage when it comes to how Comscore calculates its numbers (except for search). Comscore’s numbers are based on a panel. A piece of software is installed on the participants’ computer. It monitors every URL visited and sends a report back to Comscore. The total numbers are achieved by extrapolating the actual numbers.

I’m convinced that advanced Internet users are underrepresented in this panel. Or how do you like the idea of having a HTTP proxy that monitors all your traffic. Now this is exactly the crowd that uses Google’s services more than the average user. Therefore I expect these numbers to be biased in favor of the more main stream competitors. By how much is hard to tell though.

Related reading: Google Backlash – Round 2

Update 12/28/06: Fred Wilson, a board member of comScore, explains the bias:

Comscore is a “consumer panel”. It measures mainstream web users. It is not a “leading edge” panel and it will almost certainly undercount “geek” services like Delicious and Digg.