Deceptive know-it-alls

28 May 2025

Across the Net, in places ranging from Microsoft’s search engine Bing to Whatsapp, Photoshop and software from Adobe, AI assistants are popping up in more and more corners of our everyday digital lives. And the AI language models do indeed deliver amazingly convincing results – but their accuracy can’t always be guaranteed. Read our checklist to discover when, how and why artificial intelligence hallucinates and what you should do about it.

If you type “AI language models” into the search field on Google, the first hit will be an AI-generated answer. The US search engine giant first introduced this so-called “AI overview” in its home market, followed in March 2025 by markets including Germany and other European countries. If you scroll down through the expert-sounding and well-structured answers, which have so far only been shown to registered users, you will find the following disclaimer: “All responses may include mistakes.” In other words: “Large language models tend to generate new but incorrect information with great self-confidence,” says Thora Markert, director of AI Research and Governance at TÜVIT.

And, as the AI integrates its invented facts into fluent and coherent texts, these hallucinations often seem plausible. In consequence, users believe the misinformation and may well disseminate it more widely. Such hallucinations are a genuine issue, not least for the growing number of companies that use AI to communicate with their customers. After all, if its own AI chatbot constantly provides people with false information, this can quickly dent a company’s reputation.

Of course, AI models don’t lie on purpose. Unlike us, they simply have no awareness of the texts they generate. The technical reasons for their hallucinations can be many and varied. “One possible factor is that the AI might have been trained using outdated, bad or contradictory data,” Ms. Markert explains. Inadequate training and generation methods can also create fertile soil for hallucinations. Sometimes, language models simply lack sufficient information on a particular topic. They then try to fill in the gaps based on learned patterns – and come up with apparently suitable facts or quotes.

“But unclear or contradictory inputs from users can also lead to hallucinations,” Ms. Markert says. Especially when we communicate colloquially with AI, by using sarcasm, for instance, which tends to lead it by the nose.” However, what might have led to a particular hallucination in a specific case is usually not easy to determine: “AI models are usually a black box,” says the expert. How the models operate and generate their results is often not visible even to their developers.

Start by not blindly trusting AI! Small discrepancies, for example, get noticed even by non-specialists: Sentences may contradict previous sentences or parts of the generated response. Or the AI’s answers may have little or nothing to do with the question originally asked by its human user. “Regular checks of the accuracy and reliability of AI-supported systems are essential to minimise the risk of misinformation,” Ms. Markert recommends. So in the end, there’s no getting around the need to check the facts for yourself. Google, for example, also states the sources from which its AI answers were generated. If you follow these links, you can get an impression for yourself as to whether the AI has summarised and reproduced the information correctly or whether it has taken figures and statements out of context.

Even if the language model in question is built on reliable and consistent data sets, digital illusions can’t be completely ruled out. “This is why reducing hallucinations is one of the fundamental challenges facing AI developers and operators,” says Vasilios Danos, director of AI Security at TÜVIT. It’s his job and that of his team to help developers overcome these challenges. In extensive tests, the experts identify vulnerabilities with the aim of minimising risks. “In the best case scenario, we get called upon before the application is launched on the market to reduce the likelihood of hallucinations from the outset,” says the expert. This helps increase confidence in AI technologies, says Mr. Danos. “And it will ensure that their benefits can be used responsibly.”