How to Measure Web Site Popularity

For years, it has been known that website popularity follows a zipf distribution. What this means is that if you number all the websites in the world by how popular they are, then their traffic is inversely proportional to their rank. That is, website #10 gets one tenth the traffic of website #1. Similarly, #100 gets one tenth the traffic of #10.

Given this fact, and given the fact that I run a website, it's only natural for me to wonder how popular my website is, compared to all the others.

There are several ways to measure the popularity of a website. You could measure page-views per day, or unique visitors per day, or number of inbound links. Unfortunately, on all of those scales, my website is almost too small to measure (350 hits/day, 35 sites/day, and no inbound links except the ones I made myself).

If the web followed a linear distribution, instead of a zipf distribution, then the above statistics would mean that my website must be near the bottom of the heap, probably in the bottom 10%. If this were the case, I would become completely depressed, and probably give it up entirely.

However, the web follows a zipf distribution, which means that the vast, vast, vast majority of websites have hardly any traffic, while a select few have tons of traffic. It also means that if your website has any traffic at all, then it's actually pretty far up in the rankings. I decided to take heart from this fact, and set out to find out what my actual rank is.

I decided to measure popularity as according to Google. The advantage of this metric is that a large proportion of all website visitors are referred from Google. So if Google thinks your website is popular, then it is popular, almost by definition.

So how do you find out how popular Google thinks you are? Simply go to Google and type in some search terms that are 1) relatively random, 2) somewhere on Google's cached version of your page, and 3) produce few enough results for your site to show up in the first 700 results.

I started by searching for "burchett". My site showed up as number 118 out of 47,900. Then I searched for "kimberley", but I hit Google's limit of 700 results before I found my website. Then I searched for "About Kim" which turned out to be pretty representative, even though it's the title of my blog. Here's the full list:

Search Term	Placement	Percentage
burchett	118/47,900	99.75%
kimberley	>700/570,000	<99.88%
"about kim"	32/16,600	99.81%
java abandoned code	49/28,300	99.83%
movable type iraq	279/48,100	99.42%
website blog code	426/117,000	99.64%

As you can see, the results are pretty consistent; my site is in the top 0.5% to 0.2% of all web sites, according to Google. Google currently indexes 3 billion pages, which means my site is somewhere between number 15 million and number 6 million.

Let's double check these numbers. Suppose my site is number 10 million. Since I currently get about 350 hits a day, this would imply that the #1 website gets about 3.5 billion hits per day. Google apparently gets about 150 million hits a day, which means that this estimate is off by no more than a factor of 20 -- and if anything, my site is actually more popular than I estimated.

Here's another interesting way to look at these numbers. In April, I averaged 150 hits per day. Since I'm averaging twice that this month, that means my website is now twice as high up in the ranking as it was last month. In other words, I moved from number 20 million to number 10 million, in only a month (assuming that the web as a whole stayed constant, which it didn't -- but it certainly didn't grow by 100% per month, so it hardly matters).

---

Update May 30, 7:30pm: When I first wrote this, I forgot that Google scores pages higher if the search term is contained in the page's title. I also forgot to account for the fact that Google removes "similar" pages from the results. Google is obviously applying some heuristics, because I'm now 74/47,800 when searching for "burchett" (higher than before). On the other hand, when I search for "compiler abandoned" (words which aren't in the title of any of my pages), then my site is nowhere in the first 568 out of only 22,600.

burchett	74/47,800	99.90%
compiler abandoned	>568/22,600	<97.49%

Posted on May 28, 2003 06:34 PM
More blog articles