Measures That Mean Squat

If something is in the digital realm, chances are it can be measured - but that’s beside the point. The true value lies in being able to answer  the question “What should we measure?”. There are multiple articles, books, and by now, possibly, PhD dissertations on KPI discovery methods, so I won’t go into any of that.

Instead, I wanted to share 5 metrics that I think are for the most part useless and make my antennas go up whenever I see them:

Page Views

This metric is a direct descendent of the even more meaningless metric of “Hits”. While it certainly has its proper uses, page views are often quoted as one of the key indicator of a site’s success -- and therein lies the problem. “Our site had a million page views last month,” you might hear people say. Well, guess what, unless you are a site whose sole purpose is to serve content - this metric means practically nothing. The thing with page views is that they can be easily manipulated. Pop-ups, pop-unders, auto page refreshing, spidering, chopping up content into smaller chunks - these are just some of the techniques that have been used to inflate this number.

Unique Page Views

This metric became part of some analytics packages not long ago. In previous years I had heard it used mostly by folks who were mixing up the terms "page views" and "unique visitors” (without really understanding either). The intent of this metric seems to be to measure the number of visits/sessions where a specific page was viewed at least once. My gripe with this metric is that it essentially tries to take over the definition of the visits/sessions metric but uses it in a much narrower context. 

Visits from Africa/Asia/Australia

I should start by saying that I have absolutely nothing against these continents. If you are running paid campaigns and your website’s target audience is primarily in the United States or Europe, I’d seriously question any activity that is coming from IPs in Africa/Asia/Australia. In my experience such activity is in most cases click fraud or some form of non-human traffic, or other unsubstantiated activity. You might as well be getting visits from Mars, they will not translate into any meaningful activity on your site.

Registered Users

The usual suspect statements are along the lines of “Our company has acquired a million registered users”. Great, how many of them are actually active? The problem with cumulative metrics such as registered users is that they might sound impressive, as they count the number of users since the service was established, but they do not reflect the present state of the site/product. Total app downloads is another similar metric - an app may have been downloaded many times, but that does not necessarily translate into the app being used. Supportive metrics such as app launches, app uninstalls, number of app installs per user should be sought in order to perform a more complete analysis. 

Percentage increases without the actual numbers

“Last month our user base grew by 100 percent.”  That sounds impressive, but it could be just masking the fact that the user base grew from 10 to 20 users. Whenever I see something like this, I am reminded of one my professors who used to say “You can get 1000 points for completing this task”. His private joke was that he never supplied the scale of what 1000 points were worth, so the students never knew what 1000 points really meant. Whenever you see percentages, but no absolute numbers, you should ask for the underlying data as well.

The above selection is just a short list of metrics that are commonly misused. The reality is that any metric can be misrepresented or used in the wrong context, and result in driving the wrong conclusions. My advice is to question the metrics, make sure you understand their definition and meaning before including them in your analysis.

Stay tuned for our next week's post on how to fully understand and validate any metric, dimension, or segment in Adobe Analytics/Omniture/SiteCatalyst.