Study Finds That AI Search Engines Are Wrong an Astounding Proportion of the Time

This may come as a shock, but it turns out that an astounding proportion of AI search results are flat-out incorrect, according to a new study published by the Columbia Journalism Review. We hope you were sitting down.

Conducted by researchers at the Tow Center for Digital Journalism, the analysis probed eight AI models including OpenAI's ChatGPT search and Google's Gemini, finding that overall, they gave an incorrect answer to more than 60 percent of queries. 

It should tell you something that the most accurate model to emerge from these tests, Perplexity from Perplexity AI, still answered 37 percent of its questions incorrectly. The village idiot award, meanwhile, goes to Elon Musk's chatbot Grok 3, which was wrong a staggering 94 percent of the time. Impressively bad.

"While traditional search engines typically operate as an intermediary, guiding users to news websites and other quality content, generative search tools parse and repackage information themselves, cutting off traffic flow to original sources," the authors warned. "These chatbots' conversational outputs often obfuscate serious underlying issues with information quality."

By now, of course, the proclivity of large language models to lie or wrongly report information is well documented. But that hasn't stopped tech companies from trying to supplant the traditional web search, with some releasing versions of their existing chatbots tailor-made to do just that, like ChatGPT search. Google has even debuted an "AI Mode" that only shows Gemini summaries instead of web links.

This latest study quantifies why this might be a bad idea. It was conducted by choosing ten random articles each from a pool of twenty publications, ranging from The Wall Street Journal to TechCrunch. In what should've been a softball, the chatbots were asked to identify an article's headline, its publisher, its publication date, and its URL. To make things even easier, the researchers made sure to only choose article excerpts that returned the original source within the first three results of an old-fashioned Google search.

In addition to showing the AI models were wrong over half the time, these tests exposed other idiot tendencies. A classic one? Passing off their dubious wisdom "with alarming confidence," by either not qualifying their responses or failing to decline questions they didn't know the answer to. 

This lines up with other research documenting how AI models would rather hallucinate — or make up — answers instead of admitting they're out of their depth. Maybe that's because a policy of honesty would betray just how useless the AI models can be; Microsoft's Copilot, for example, declined more questions than it answered, the researchers said.

The AI search tools were also terrible at citing their sources. ChatGPT Search linked to the wrong source article nearly 40 percent of the time, and straight up didn't bother to provide one in another 21 percent of cases. That's bad from a fact-checking point of view, and just as grim for publishers, who will be denied even the chance of getting traffic from an AI model that's scraped their content. Bodes well for the survival of our online media economy, doesn't it?

More on AI: The Entire Internet Is Being Polluted by AI Slop

The post Study Finds That AI Search Engines Are Wrong an Astounding Proportion of the Time appeared first on Futurism.