Mac Musings

Unknown OS Redux

Daniel Knight - 2002.01.15 -

Watching site traffic over the weekend, I've been stunned to see "OS unknown" account for one-third of all site traffic. I speculated on Saturday (Is OS X the Unknown OS?) that perhaps Mac OS X accounted for all this traffic.

A closer look at the logs, though, showed that I should have been looking elsewhere. Analog reports requests, gigabytes served, and pages served by operating system. Requests will be very high, about 5 times higher than pages served, because of included graphics. That's consistent between Macs, Windows, Unix, BeOS, OS/2, and Amiga.

It's not consistent for "OS unknown." By only looking at pages served, I'd missed a crucial piece of information. Instead of handling roughly 4.6 requests per page, the average so far this month is 1.36 requests per page. In short, these unknown computers are mostly ignoring the graphics and just looking at the text. In other words, the bulk of them are probably search engine spiders examing the site.

So much for the OS X theory.

What's really happened is that our logs haven't been as well purged of spiders over the past couple months. Here's what our site stats look like based on requests, not pages served:

 Month     Mac      Win     *nix     ???   ratio
2000.09   51.20%   42.72%   3.19%   2.57%   2.93
2000.10   52.90%   41.59%   2.48%   2.77%   2.59
2000.11   51.55%   42.06%   3.56%   2.57%   3.29
2000.12   52.19%   42.67%   2.18%   2.68%   3.83
2001.01   51.95%   43.30%   2.04%   2.48%   3.38
2001.02   53.78%   41.30%   2.13%   2.59%   3.92
2001.03   52.15%   43.19%   1.91%   2.55%   4.77
2001.04   50.67%   44.14%   1.93%   3.83%   3.91
2001.05   49.37%   45.56%   1.99%   2.88%   4.05
2001.06   49.01%   45.91%   2.09%   2.81%   2.92
2001.07   51.01%   43.77%   2.15%   2.88%   4.21
2001.08   49.23%   45.35%   2.19%   3.06%   2.56
2001.09   49.15%   45.59%   2.07%   3.05%   2.98
2001.10   46.60%   47.63%   2.31%   3.35%   2.25
2001.11   47.51%   47.41%   2.02%   2.96%   2.36
2001.12   46.43%   45.37%   3.72%   4.37%   1.61
2002.01   47.55%   44.17%   2.14%   5.98%   1.36

The final column is the ratio of requests to pages served for OS unknown. The higher that is, the higher the percentage of visitors vs. spiders.

Over the past four months, the overall average for Mac, Windows, and Unix visitors to the site is 4.6 requests per page. To estimate what percentage of "OS unknown" is real users instead of spiders, we begin by subtracting the number of pages served from requests, then divide the result by 3.6. That number is not impressive.

In the end, only about 0.5% (+/- 0.1%) of the visitors to our site are using an unknown OS. The remainder of the OS unknown category is spiders.

So much for my theory of the missing Mac OS X users.