CEO of the Broadcast Research Council, Clare O’Neil, tells us why transparency of reporting, a balanced panel and creating data stability is essential.
In 2014 the broadcasters took over the management of TAMS. The 2013 audit had revealed that under SAARF’s stewardship, the system was completely broken. By the latter part of 2015, after two years of “fixing”, the BRC TAMS has been labelled “a high quality, well-functioning TAMS panel” by independent international TAMS auditor and statistician, Robert Ruud.
According to Ruud, who came out from Norway in August 2015, the South African TAMS panel (all of whose data and IP attached is owned by the Broadcast Research Council of South Africa), has been transformed over the last couple of years due to daily oversight through a battery of KPIs from the BRC to TAMS service provider, Nielsen.
This was not always the case. It took two years of grit, grind, sheer determination and will by the South African Television Broadcasters, through their joint industry committee, the Broadcast Research Council. It was almost against all odds that the BRC set out to fix the extremely dysfunctional TAMS service that the industry had become accustomed to.
The TAMS service that the TV broadcasters started investigating, through intense statistical analysis, back in 2013, was in a disgraceful state. The audit of that year (the first of its kind in twenty years) revealed a weighting efficiency of the panel at an all-time low of 38%. The weighting efficiency of the panel reveals how closely matched the Television Audience Measurement panel is to the total Television Universe, as described in the Establishment Survey.
The industry had an ageing, unbalanced, unhealthy panel, delivering data that did not reflect the TV Universe. In order to obtain usable data, the BRC needed to get the panel to an efficiency level of at least 70% minimum.
This transformation happened in a number of ways as complex variables and changes had to be implemented by the Service Provider simultaneously in order to achieve results for the South African TAMS service.
The BRC took control of the situation in 2014 by placing proper controls, oversight, management and transparent reporting with regular audits in place. These were all embedded into the new TAMS contract with Nielsen through a series of stringent KPIs (with penalties), which the BRC scrutinised each week.
Since the BRC took over full ownership of the TAMS data and all IP, these transparent KPIs now form the gold standard (aligning to world class TAMS services) of reporting across every element of the TAMS service.
They are: balance of the panel; weighting process; recruitment procedures; panelist compliance; panel maintenance; people meter technology; data cleaning and data processing.
In tandem to setting the gold standard of transparent KPI reporting, the BRC understood that the only way the weighting efficiencies could move from the 38% to a minimum of 70% would be to ensure that the panel was balanced. The weighting efficiency is a statistical measure that shows how closely the panel sample matches the population it represents. The higher the efficiency percentage, the more representative the sample.
The TAMS Universe is updated at regular intervals (with two updates this year against AMPS 2015A update in April 2016 and AMPS 2015B in September 2016) with the latest household and population figures from the current Establishment Survey (AMPS). These bi-annual updates will continue with the new Establishment Survey, currently under construction.
Balancing the panel included a focus on the “difficult to recruit” groups at the extreme ends of the spectrum: the LSM 1-4 households and DStv PVR households – both categories reflected severe imbalances. These have been improved to accepted tolerance levels.
However, keeping the panel balanced and constantly corrected for these two categories remains an incredibly difficult task as there is constant movement in those segments. The latest annual TAMS audit of August 2015 reflected an individual weighting efficiency of 79,5%.
Many other improvements were implemented over the last two years which contributed towards the stability of the data. They are: reporting of households increased from 78% to 90% (high polling); the maximum daily weight for individuals dropped from 99,000 to 22,000 so enhancing data stability; panel tenure in excess of 8 years has been reduced by 60% and by the end of February 2016 no households will exceed 8 years tenure. Further panel expansions occurred last year, yielding bigger samples, which ultimately contributes to further stability of the data outputs.
Also, the installation of new meter technology brought about accuracy of measurement; the ability to measure the changes and new developments in the TV landscape as well as DTT migration readiness and battery back-up resulting in more data collected from more meters on a daily basis, translating into a larger daily sample and a more representative panel.
Since the core TAMS panel is now managed through transparent KPI reporting, the panel is balanced through constant checks to the total TV Universe and the data has found more stability through all these measures. The result is a world class television audience measurement system.
The focus now is to extend to further panel expansions, ensuring alignment of the panel to the pending DTT roll out and planning for non-linear viewing. This will ideally integrate into the current linear-broadcast audience measurement system, eventually creating a hybrid measurement system.
