Analysis of the Statistical Differences between StatCounter and Google Analytics

Abstract

This article examines the quantitative differences between StatCounter and Google Analytics across two statistics, page views and visits, for one website over a two month period. While no significant difference was found in the number of reported page views, the difference in reported visits was highly significant at the .0001 level of probability with a very large effect r = .60. While it appears that Google does underreport visits and visitors, it is nevertheless far more statistically consistent than StatCounter. Nevertheless, the real value of these numbers seems limited to trend analysis over time.

Analysis of the Statistical Differences between StatCounter and Google Analytics

It is a generally well-known fact among SEO experts and webmasters that there are considerable differences in the numbers being reported between the two most popular website statistical reporting services in use today: Google Analytics and StatCounter.

The general consensus seems to be that Google Analytics consistently underreports actual activity while StatCounter consistently over reports. These differences have been attributed to the fact that Google does not record JavaScript disabled visitors of any kind while StatCounter records image-enabled bots, but not all (see, for example, StatCounter Forum and Martinez [2006] for detailed discussions).

In addition, StatCounter does not distinguish between total visits and absolute unique visitors and the difference between a first-time and returning visitor is determined by the length of time between visits as set by the site’s webmaster up to a 24-hour time delay between visits. Google Analytics, on the other hand, uses one cookie to track unique visitors (_utma, that never changes unless the cookie is deleted) and two other cookies in unison to track visits, _utmb and _utmc. Unlike StatCounter, the counter timing is controlled by Google, not the webmaster, and a new visit is defined as either a visit from a unique visitor or a returning visitor after more than a 30-minute time delay between returning visits. That is, if a user visits a site for one minute, closes the browser and returns within 29 minutes, the visit counter is not incremented. If the same user returns 31 minutes later, the total number of visits is then incremented by one visit, at least in theory as reported by Google.

Google calculates the total number of visits as the sum of the number of visits from absolute unique visitors and those contributed by returning visitors after a 30-minute delay between visits for any given period of time. StatCounter defines a unique visitor as one who has returned after a time period determined by the webmaster up to a 24-hour period. Thus, in theory, the total number of visits being reported by Google for any given time period should be equivalent to the total number of “unique visitors” in StatCounter assuming the “maximum visit length” property has been set to 30 minutes by the webmaster. Restated, as first-time or returning visitors who return multiple times within a 30-minute period should only be counted once, the total number of “unique visitors” (all visitors who return after a 30-minute delay between visits) in StatCounter should be equivalent to the total number of “unique visits” in Google Analytics.

If it is true that Google Analytics is underreporting and StatCounter is consistently over reporting, the question remains: What is the quantitative difference between the two reporting services and is it a statistically significant one? This article examines and discusses the quantitative differences between Google Analytics and StatCounter in regard to reported page views and visits.

Method

Data consisting of daily page views and the number of visits/unique visitors for the months of August and September 2009 were collected from a brochure website of a retail store that has been online for over five years and has relatively consistent statistics from month to month.

In preparation for the study, the cookie time limit in StatCounter was set to 30 minutes so that the number of total unique visitors would be theoretically equivalent to how Google Analytics defines total visits.

SPSS version 16.02 was used to enter and analyze the daily data for three variables: reporting site, page views, and number of visits, defined as number of total visits in Google and number of total visitors in StatCounter. Box plots were generated to illustrate differences in variance and independent t-tests were conducted to explore the presence of any significant differences between the two reporting services.

Results

Page Views

StatCounter reported daily page views that ranged from a minimum of 578 to a maximum of 1545 with a mean of 998.07, SD = 214.19, in comparison to Google Analytics which reported daily page views that ranged from 705 to 1391 with a mean of 1038.03, SD = 186.82. Figure 1 illustrates the comparative range of reported page views and relative means.

An independent t-test revealed that, on the average, the reported differences in page views were not statistically significant t(120) = -.77, p = .698 and the effect was minuscule r = .001.

Figure 1: Reported Page Views By Service

Visits

StatCounter reported daily visits (visitors) that ranged from a minimum of 225 to a maximum of 495 with a mean of 322.27, SD = 63.86, in comparison to Google which reported daily visits that ranged from 182 to 314 with a mean of 248.60 visits per day, SD = 31.06. Figure 2 illustrates the comparative range of reported visits and relative means.

An independent t-test revealed that, on the average, the reported differences in page views were highly significant t(120) = 5.68, p = .0001 with a very large effect r = .60.

Figure 2: Reported Visits By Service

Discussion

Page Views

On the average, the number of page views being reported by both statistical reporting services is fairly consistent, although the difference in the range of page views between StatCounter and Google Analytics is approximately 30 percent (967 as opposed to 686, respectively). This is a particularly curious finding as it suggests that StatCounter is not only over reporting but, on occasion, is underreporting as well. A possible explanation for this finding is that Google is more consistent in how it distinguishes between page views from human beings as opposed to search engine bots and spiders. That is, StatCounter may be occasionally and inconsistently ignoring page views from human beings when it should be counting them. Nevertheless, the between-group differences in mean variances are not significantly different than their within-group variances and, therefore, both services are in relative agreement about how many human page views a site is experiencing, on the average.

Visits

The difference in the number of reported visits is so large that the likelihood that these two sets of statistics represent the same website is 1 in 10,000.

If Google Analytics is incrementing the visit counter by one visit after a 30-minute delay in browser activity from the same computer and StatCounter is incrementing the visitor counter by one visitor after a 30-minute delay in browser activity, then the number of visits and visitors being reported by GA and StatCounter, respectively, should be similar.

As the difference in the average number of daily visits (visitors) is so huge, it appears that both reporting services are committing reporting errors, but in opposite directions. It is highly unlikely that Google Analytics is incrementing the visit counter by one visit after a 30-minute delay between visits from the same user: The actual time delay is probably much greater than 30 minutes such that the same user who returns after a one- to two-hour delay (perhaps even longer) is being ignored as a new visit. Conversely, it appears that StatCounter is not only including visits from spiders and bots but is very likely ignoring the cookie time limit set by the webmaster used for determining how much time should elapse before a visitor is counted as unique. It appears that all return visits are incrementing the unique visitor counter by one, irrespective of how short the time delay between visits actually is.

Given the enormous discrepancy between the number of visits being reported by Google Analytics and StatCounter, it appears that both sets of statistics are, at best, only rough approximations of what is really occurring on one’s website.

Despite this fact, Google Analytics is statistically far more internally consistent from day to day in how it distinguishes between human beings and machines than its counterpart StatCounter. As Google Analytics is clearly underreporting the total number of visits and visitors on a consistent basis, the real advantage to these statistics does not lie in the raw scores but in examining a trend analysis over time. That is, a significant increase (or decrease) in the total number of visits and visitors over a period of several months probably has more meaning in Google Analytics than it does in StatCounter.

References

Martinez, Michael (2006). How Reliable is Google Analytics? SEOMoz.org. Retrieved October 10, 2009 from http://www.seomoz.org/blog/how-reliable-is-google-analytics.

StatCounter Forum. Differences Between Google Analytics and StatCounter. Retrieved October 10, 2009 from http://forum.statcounter.com/vb/showthread.php?t=27572.