The use of emotionally appealing information in marketing is nothing new. Advertising campaigns have long been littered with stimuli to provoke emotional responses, bordering ubiquity.
Yet, marketers have, for the most part, very little return information on the emotional responses of consumers to their marketing campaigns. Largely, such feedback is indirectly measured information, such as impressions, reach and engagement.
The bandwidth of emotional communication remains biased towards the outgoing message of the marketer, with the feedback loop in return being low fidelity: sometimes as binary as a click, or a purchase, as an indication of success.
This does not mean that marketers have not attempted to quantify marketing effectiveness, but rather the measurement has traditionally been indirect and sparse.
Newer technologies, however, are starting to change this imbalance by providing marketers with the ability to receive sensory feedback, which could be used to interpret emotional responses to marketing stimuli. Such systems broadly fall into private and public systems, both of which influence their legal interpretation.
Private systems
Private systems are those which may be deployed in the home. These could also include webcam feedback of facial cues, in-car cameras in some smart and autonomous vehicles capable of driver body language monitoring, and smart adult toys with emotion sensing capabilities.
This could include active-when-used tracking (such as facial recognition systems where mobile cameras are used); and always-on tracking, such as biometric-based collecting data from smart wearables.
Public systems
Advertising in public spaces is not new, with out-of-home or outdoor media (OOH) advertising being an entire industry unto itself. OOH advertising is typically found in public common spaces, such as train stations, retail areas, billboards and other places where the attention of consumers ‘on the go’ can be commoditised.
This industry is also undergoing a resurgence, with some arguing that this is a result of the growing privacy compliance demands of digital online advertising. Reportedly, since the coming into force of the General Data Protection Regulations (GDPR) in Europe in May 2019, European demands for programmatic advertisements fell between 24% and 40%.
Digital out-of-home advertising (DOOH) is also now employed at key locations where technical resources permit. Such systems typically already use interactive capabilities, such as high-resolution touch sensitive monitors, audio capabilities and connectivity for access to cloud-based content and resources.
The inclusion of cameras and microphones in such DOOH systems is a very simple next step and, together with connectivity to a backend of cloud based facial recognition and image processing, the ability to use such systems to interpret consumer responses is easily achievable. Some facial recognition billboards are capable of recognising gender, ethnicity and age, reportedly with 85 – 90% accuracy.
Regardless of space, such ’empathic media’, as it has called, can take a variety of forms. The basic premise is an advertising system, capable of sensing aural, visual, physiological, or other feedback of a consumer in response to marketing stimuli, which is processed and acted upon in real time in the augmentation of the advertisement.
Further, the level of augmentation is also important, which lies on a spectrum of very simple (such as a change of colour or text) to highly complex (such as an on-screen digitally animated sales agent interacting with the customer).
Gladvertising
Although still uncommon, and an eerie reminder of the advertising systems in Minority Report, the technology is already being well studied, particularly in the opportunities and challenges associated with mood recognition (what the Centre for Future Studies termed ‘Gladvertising’).
Some have argued that the way such systems may interact with people does not require the use of personally identifiable personal information. This is based on the argument that such technologies do not require identifiable or even anonymised demographic data when sensing the cues used to make inferences about a person’s emotional state or mood.
Basic information such as facial muscle movements, assessed using facial coding technologies, could be enough to make inferences. If this argument is accepted, this may exclude the application of various personal information protection regimes. There is some scepticism, however, that personal information can really be excluded for a system to have sufficient information to make useful inferences, but this has not yet been tested by courts.
Personal data
In Europe, the GDPR exists as the central privacy regulation. A key criterion in material scope of the GDPR is the processing of personal data. If the relevant empathic media system processes information which is so anonymised, then there is arguably an issue of application of the GDPR.
It is argued that in the case of private systems, the likelihood of connection of the personal information (such as account profile data) would be easily made with processed information, and it would be difficult to imagine a scenario where full anonymisation is possible. In public systems, however, there is a far greater potential for anonymised scenarios.
The industry nonetheless treats the tracking of emotional reactions as personal information, using an explicit opt-in system. In private systems, this approach is already adopted by tech companies. In public spaces, systems that use empathic media clearly disclaim themselves as doing so, and only begin tracking required cues once a user has acknowledged their consent to being tracked.
Where decisions can be made on the basis of such empathic advertising, customers are often given a right to withdraw on the basis of a cooling-off period (as is the case in some jurisdictions’ consumer protection laws in relation to unsolicited direct marketing). This is particularly relevant where the empathic media is of a more sophisticated variety (such as a digitally animated customer assistant) and contain point-of-sale capabilities.
Clearly, boundaries of exploitation are needed in the use of empathic media. It is in the supplier’s best interests to self-establish ethical policies and guidelines on consent for intimate empathic media, which will prevent the need to wait for the inevitable regulatory intervention that will soon follow.
With new uses and applications constantly emerging, and this technology receiving attention from the media, governments, regulators and the general public around the world, regulations are imminent.
Ashlin Perumall is senior associate in the corporate/M&A practice at Baker McKenzie Johannesburg.
