If a chess grandmaster plays against an amateur chess player and loses, the result is considered a fluke. Or, as in the case of Niemann v Carlsen, grounds for cheating allegations, a defamation lawsuit and a Netflix documentary.
But it doesn’t affect the grandmaster’s ranking as one of the best chess players in the world.
That’s because in 1970, the World Chess Federation adopted Elo rating – a system that calculates ranking over time, as opposed to basing it on individual moments. Now, that same concept is finding a new application as the solution to an emerging conundrum in marketing circles – measuring AI search performance.
Why traditional SEO metrics fail
Traditional SEO was conveniently methodical, and success was easy to measure. Follow the formula, and your page would rank higher in search results. If you had an SEO whizz at the wheel, you’d rank on page one.
But AI has done AI things and changed everything. We’re not talking more complex formulas for ranking (although, yes, that too). The whole concept of ranking as we know it has been upended. In the world of AI search, being cited or recommended as a credible, linked source is the new page one ranking.
But the real conundrum is how to measure where your brand shows up in this new wild world of search. Between subjective natural language queries, AI’s complex approach to data processing and keyword variation, results can fluctuate enormously. Even in a single day, you could see your ranking change more frequently than the weather in Cape Town.
It means, unfortunately, that a single ranking result in AI search tells you almost nothing. It’s what we’ve taken to calling “point in time ‘noise’” and it will drive you to drink sooner than giving you any kind of helpful insight.
Elo rating for AI search
Hungarian-American physicist Arpad Elo theorised that individual results provided an inaccurate reflection of true competitive strength. His solution was to measure sustained performance across hundreds of head-to-head comparisons. At the time, he was talking about chess. But his theory holds up well for marketers in 2026.
Elo rating, when applied to chess, starts every competitor on equal footing and also counts how surprising the results were. As the matches rack up, the noise cancels out and the true ranking emerges.
When you apply the principle to AI search, a “match” is a search, and keywords become additional variables. The pattern starts to emerge over hundreds of searches and multiple keywords.
Over time, the trends cancel out the noise of the misleading point-in-time data, giving brands a more accurate sense of how they’re showing up in search results and how they measure up against competitors.
What matters in AI search
When our team applied Elo rating to 21.6 million searches that mirrored how customers use LLMs like Perplexity, Claude, ChatGPT, Gemini and others, an intriguing trend emerged: data quality was the single most important differentiating factor that separated the businesses that performed from those that didn’t.
The research also revealed that LLMs valued
Quality over proximity
While proximity is still a factor (as in Google’s famous ‘near me’ searches), data quality trumps proximity in AI search.
Structure over ambiguity
For businesses competing for the same keyword in the same location (e.g. a mall), those with well-structured data ranked noticeably higher in AI searches.
Data over size
Good news for South Africa’s burgeoning SMME sector (and less exciting for bigger brands) is that AI values quality of data over brand size. That means a smaller business with quality, up-to-date, structured data can outperform a much larger competitor in AI search.
Your website over other sources
Finally, some good news all-round – over millions of searches, LLMs (particularly Gemini) preferred to source information about a company from its own website. Next popular as a source were listings, and other internet sources trailed behind.
As frustrating, and even disheartening, as it may feel to have to reinvent your digital marketing strategy for yet another disruptive technology, our data revealed that there are patterns, they make sense, and they’re doable, even for smaller brands. Plus, this is one wave that hasn’t peaked yet.
If you act now (by making sure your content is clean, structured and consistent across all your channels), your brand can still catch it. Happy surfing!
Bryony Rose is director of enterprise International business at Yext, a VML service partner.
