DStv has come in for a fair amount of flak over the recent upheaval in the South African Audience Research Foundation’s Television Audience Measurement Survey TAMS). But DStv says “poor quality data” has impacted on the the entire television industry in SA.
The ongoing issue centres around the decision by Saarf late in 2012 to use DStv’s audited figures for weighting the pay channel’s penetration, PVR penetration and the bouquet percentages within the DStv universe on the TAMS until after an audit into its methodology was completed. That has been done, and the findings threw up more issues that needed “rectification”.
The audit discovered over-sampling at the top end of the television viewing market, and what Saarf calls “instability” in the LSM 1-4 sector. It also raised the question whether DStv benefitted from the weighting at the top end, and how free-to-air channels the SABC and e.tv were compromised.
“e.tv is still evaluating the impact of the serious shortcomings of the TAMS Panel as uncovered by the audit report. We are also waiting for the outcome of several remedial proposals made to Saarf,” says the free-to-air channel’s COO, Bronwyn Keene-Young. She says the broadcaster will give a “more comprehensive response” as to how they intend proceeding shortly.
DStv Media Sales, however, questions whether spending would have been different if TAMs was operating at 100% efficiency or whether there has been a dramatic shift in reported audience share between the broadcasters as a result of TAMS deficiencies. But it says poor quality data has negatively impacted on the entire industry, an on both buyers and sellers.
DStv Media Sales CEO Chris Hitchings says DStv has always participated in and supported efforts by the TV broadcast industry to improve the accuracy of audience measurement. “Accurate audience measurement is critical for strategic planning for the advertising industry and for individual broadcasters. Media agencies use the data to plan campaigns and assess their efficacy, and broadcasters use it to set rates, analyse programmes and channels,” he says.
The National Association of Broadcaster (NAB) says although the nature and extent of prejudice suffered by broadcasters may differ, “it must be noted that all TV broadcasters have been prejudiced by the integrity of the TAMS data. Individual television operators may therefore determine the most appropriate course of action depending on their individual assessment of the impact on their respective operations”.
The Saarf audit, conducted by CESP in France, revealed communication and reporting of panel deficiencies over the years; highlighted the ageing technology used in the panel; questioned the panel’s weighting, balance, and management; and cast doubt on its technical oversight and controls.
“The NAB Television Broadcasting members are unanimous in the view that the integrity of the TAMS Panel has been severely compromised and that the CESP audit findings must be addressed immediately,” NAB says. “The NAB has therefore engaged both the Saarf Board, as well as Nielsen Media Research (the research supplier since 1987) to discuss the potential financial and reputational damage to television broadcasters and to develop a way forward.”
Media planning expert and author, Gordon Muller, says TAMS data has “progressively been unable to keep pace with a multi-screen/multi-channel environment. Nothing to debate there. That’s correct. The point is that shifts in TAMS sampling and panels are an attempt to address this problem. It’s better to have a hiatus en route to better data than reliable reporting of incorrect data,” he says. “On the other hand, if we are still having this debate next year, then we have a serious problem.”
DStv says the size of its reported audience was derived from the AMPS survey for the period 2009 to 2012. It says that during this period, AMPS recorded increasingly higher DStv subscriber numbers than reality but that by the time they were incorporated into TAMS, they were lower than reality. “In other words, the rapid growth in DStv subscribers meant that, from 2009 to 2012, actual DStv subscriber numbers were consistently higher than those reported by TAMS,” it says.
“DStv consequently had a lower reported share of the TV market from 2009 to 2012,” it explained in a statement. “In July 2012, this changed, and for the first time the AMPS ‘over-read’ derived from AMPS 11b was now ahead of the active DStv subscriber base (when these subscriber figures were adopted into the TAMs universes).”
DStv Media Sales points out that they were in fact responsible for reporting the discrepancies. “DStv did not attract an undue share of ad spend on the basis of the sizing in TAMs until actual DStv subscriber numbers were used by TAMs, DStv consistently advised clients of the discrepancy, and provided them with accurate information from our own system, DStv-I,” it says.
Muller says he doesn’t believe TAMS in isolation “would significantly create macro (inter-media) shifts in media investment, any more than RAMS creates macro (inter-media) shifts in radio investment or AMPS AIR readership figures create macro (inter-media) shifts in print media investment”.
He says there are a “whole bunch” of other factors at play, “not least of all the competence of the media owner and the ability of the planner to intelligently interpret the data. Leo Bogart observed, ‘The Great Idea in advertising is far more than the sum of the recognition scores, the ratings and all the other superficial indicators of its success; it is in the realm of myth, to which measurements cannot apply’.
“Despite the desperate attempts of procurement departments to reduce media planning to a commodity based trading exercise, the best media plans are still an exercise in ‘paint by numbers’. It’s not the individual dots that make up the big picture but the manner in which you connect them,” he says.
In a perfect world, says DStv, a panel would represent the population it is measuring exactly, and therefore the panel efficiency would be 100%. But, it says, the efficiency of the TAMS panel has tracked well below the CESP recommended norm of 70%. In fact by the end of 2012 it had dropped below 40%.
“Efficiency does not make the TAMS data ‘wrong’, however it increases the volatility of the daily results and the range of variation in audience delivery,” it says.
Muller doesn’t believe any research is 100% efficient, saying this is an “unrealistic expectation”. “Inappropriate use of research data is, and always has been, a much bigger problem than the data itself,” he says. “Bear in mind that few if any TV planners/ buyers use the data without fist adjusting the parameters to fit the circumstances of the specific campaign. That’s a real skill set and some planners are better at working the data than others.”
In 2012 the panel efficiency dropped from 48% to 37%. Initially CESP identified two contributing factors to this sudden drop; the HD boosted homes and the under-sample of the lower end of market LSM 1-4.
DStv took it upon themselves to undertake further analysis using international weighting experts Robert Ruud and Toby Syfret. “We showed the TAMS audit committee that the drop can directly be attributed to the unbundling of the LSM RIMS implemented by Nielsen in August 2012. The new weighting regime revealed the extent to which LSM 1-4 was under represented, which had not been as evident previously when LSM 1-4 was grouped with LSM 5 in the weighting process. Both CESP and additional testing by Nielsen confirm that the greatest contributor to lowering the Weighting Efficiency on the TAMS panel is the under-sample of LSM 1 – 4,” DStv says.
Muller questions whether shifts in audience share can be attributed solely to shifts in TAMS audiences. “Maybe the improved TAMS data is just more accurately reflecting the reality of the market place. I can certainly think of a few factors that would cause significant shifts in TV viewing. How many times can you the same programme on repeat without losing audience?” he asks.
“I repeat though, inappropriate use of data by insufficiently trained people is a bigger problem than the data itself. Particularly over segmentation, which renders samples useless. We need to standardise on TV buying audiences. Male:Female/Rich: Poor/ Married: Single: Married with Children/ Young: Old. 24 market segments at most. We can’t have infinite market segmentation on the TAMS sample base. End of story.”
