Digital companies who crow about the specifics they have on their audience numbers and the insights they have on them, may be overstating this fact. Twitter has admitted it miscounted its monthly user numbers for three years, and Facebook has come under fire for inconsistencies in its stated user numbers versus official census data in America. These are not the only prominent examples, which begs the question: Just how trustworthy is online audience measurement?
IAB seeks service provider
In South Africa, a tender is currently out for an online audience measurement vendor for the Interactive Advertising Bureau (IAB). At present, Effective Measure (EM) handles this aspect of the IAB’s business. The company uses a site-based measurement methodology with JavaScript tags embedded in sites of publishers who are IAB members (currently standing at 58). Greg Mason, regional lead for Sub-Saharan Africa at Effective Measure, explains how tags work. “This tag is placed onto a website and is triggered for each page impression a visitor makes to the site. When the tag is triggered, cookies are created within the visitor’s browser, which allows the measurement tool to accurately identify device type and country of access (via IP address).” A software development kit (SDK), which caters for application measurement, is also part of the Effective Measure system.
Ryan Smit, managing director at BMI-TechKnowledge, says all publishers who are IAB members report on their EM stats only in order to ensure that everyone is comparing apples with apples.
The other most commonly used tool
Beside EM, the other most commonly used online measurement tool in SA is Google Analytics (GA). While this methodology generally measures the same audience metrics (including unique visitors, page impressions, visits, visit duration, frequency, % mobile vs. desktop etc.) as the EM system, different numbers may appear when comparing the two.
Mason clarifies this difference, “There is no generalised measurement methodology across different measurement tools. As a result, stats do vary across different measurement tools… The methodology is quite different for these two tools. The main difference is the GA tag is inserted in the head of the website code, which means it is one of the first things to load on the webpage. EM code is placed right at the bottom of the body of the website code, which means that it is the last thing to load on a website. This means unique browsers and page views are recorded only once all other content on the site is loaded,” he explains.
Just because it’s used, doesn’t make it right
Even though the EM and GA systems are used by most publishers, it doesn’t mean that they are the best possible methods, or even accurate for that matter. Arthur Goldstuck, managing director of World Wide Worx, describes GA as “hit and miss”. “Sometimes it undercounts badly and in other ways it over counts, and it is possible to game the system as well by creating what appears to be far more traffic than there actually is,” he says. Similarly, he describes the EM system as, “as reliable as they get”, but he wouldn’t put it past people to game that system either.
Publishers manipulating their numbers
While publishers may get their online audience measurement data from a well-known source like EM or GA, the other problem that arises is when they manipulate or straight up lie, inflating these numbers to attract more advertising. Mason believes this does not happen often, but stresses, “The few instances like this do harm the reputation of the industry and do cast doubt around the validity and integrity of the industry”.
Goldstuck has quite the opposite view, believing this practice is extremely common in South Africa for a number of reasons. “It’s a combination of corporate ego, wanting to show you’re the biggest or by how far you’re the biggest or that you’re at least competitive,” he says. “People being given targets, often unrealistic, by their own bosses so they manipulate numbers to show that those targets have been reached, or purely a commercial gain so presenting higher numbers to obtain higher advertising revenue.”
But Riaan Wolmarans, managing editor of digital at Tiso Blackstar Group, says this practice is a self-defeating exercise. “Using inflated traffic figures to land an advertising deal would quickly become problematic when the campaign performance reports don’t match up to the expectations,” he says.
Did Facebook lie?
Independent commentator Justin McCarthy is of the opinion that Facebook deliberately disguised their errors and claimed innocence. His main problem is with the social media giant’s video count. “They’re not supposed to count the pre-roll, the first five seconds as a view, those are their own rules, and then they got caught by some analytics company … It is daylight f***ing robbery … It’s nothing short of malfeasance,” he says passionately.
The problem, he adds, is that Facebook sets its own rules about its measurement standards, has only recently allowed third party auditing of its measurement system, and there are no global measurement standards.
So are the numbers trustworthy?
In order to ascertain whether audience numbers are valid, knowing the source of the data and understanding when and how those stats were collected is vital. “As Effective Measure is the official measurement partner of the IAB SA and is the locally accepted measurement currency, any stats presented by a media company that are Effective Measure stats can be trusted as verified and authentic stats,” Mason says.
Wolmarans stresses that EM numbers are available publicly via that platform to anyone else in the media and advertising industries, hence people can and should compare the numbers for themselves.
Goldstuck agrees, adding that historical data trend mapping can also uncover deception, particularly if there is a sudden spike in audience numbers for no discernable reason.
Industry accepted metrics
Even if Effective Measure is replaced as the online measurement vendor for the IAB, standard industry measurement metrics (unique visitors, page impressions, visits, visit duration, frequency, and % mobile vs. desktop) will still be measured. Other new metrics may also be added, but none of the industry accepted metrics will be removed. The platform, or the way the data is presented to IAB members, may also change.
Digital metrics are bullshit
Mark Ritson, an outspoken critic of current online audience measurement methods, summed up the argument of many in an article he wrote for Marketing Week. In it he stated, “Digital metrics are bullshit because they are not reliable enough. They are bullshit because there are too many of them. They are bullshit because they obfuscate comparison rather than optimise it. They are bullshit because, when it all comes down to it, two giant companies (Facebook and Google) will not share their source data with each other or anyone else.”
So having read all of this, which side of the fence are you on? Do you trust online audience numbers? Or do you believe that the system is not standardised, uniform or simple enough to make sense of what is really going on?
Follow Michael Bratt on Twitter @MichaelBratt8
