Can Fluent AI Be Believed Even When It Lies? A Study 80% of People Failed
機械翻訳 / Machine-translated

機械翻訳 / Machine-translated
@aifriends
AI Friends(https://aifriends.jp)のクロスポスト公式アカウント。AIツールの紹介・使い方・できることを、中学生でもわかるやさしい日本語で届けます。
When an AI's response is polite and natural, don't you tend to just believe it? A study that attracted attention in November 2026 put that very psychology into numbers. This article offers a straightforward explanation of what the research found and what we can do about it.
A research team at Radboud University in the Netherlands examined the relationship between a chatbot's language style and how readily users trust it. The researchers were Mariana de Sá Siqueira, Professor Tibor Bosse, and Associate Professor Barbara Müller.
The paper was published in the academic journal Computers in Human Behavior: Artificial Humans. The university's announcement was dated October 29, 2026. In Japan, GIGAZINE covered it on November 14.
The conclusion is simple: An AI that speaks in flowing, fluent sentences was rated as just as "intelligent and pleasant" as an AI that was correct but curt — even when the fluent AI's content was wrong.
Even when hallucinations occur (the phenomenon where AI fabricates plausible-sounding falsehoods), if the AI speaks well, people stop noticing.
Participants in the experiment were asked to read conversations with a chatbot about The Lord of the Rings. Each conversation was one of four types:
Only participants who already had basic knowledge of the source material were selected — people who should have been capable of spotting errors.
After reading the conversations, participants rated the chatbot on intelligence and pleasantness. The number of participants was not stated in either the university's announcement or the GIGAZINE article.
The highest-rated chatbot was the one that was both smooth and correct. That much was expected.
What was surprising came next. The chatbot that spoke fluently but gave incorrect information was rated just as "intelligent and pleasant" as the one that gave correct information in a matter-of-fact style.
Furthermore, according to reports from Digital Information World and others, approximately 80% of participants failed to notice the errors.
Associate Professor Müller noted that "when sentences sound elegant and natural, people automatically sense intelligence in them."
Imagine this: in a meeting, you have one colleague who speaks calmly and fluently, and another who mumbles. Before you even check the details, doesn't the first person seem more "capable"? The same thing is happening with AI.
Professor Bosse points out that chatbots are designed to prioritize speaking "in a human-like, persuasive way." Whether the content is factually accurate becomes a secondary concern.
When humans speak, uncertainty shows — words come haltingly, or they add qualifiers like "probably." But LLMs (AI that can write like humans) can respond fluently and in the same steady tone even about things they don't know.
The "hesitation" cues that listeners rely on simply don't exist in AI. This makes it harder to notice when something is wrong.
Consider an office worker putting together a report. They ask an AI about a market size, and back comes a confident answer complete with specific figures and a year.
Because the writing looks clean, they paste it straight into their slides. Days later, their manager asks, "Where does this number come from?" — and that's the first moment they suspect something is off. Situations like this are far from rare.
The same pitfall exists when students research homework, or when anyone looks up health or medication information through AI.
The countermeasure proposed by the research team is having AI express uncertainty with each answer. Professor Bosse said, "If the chatbot itself signals doubt, it can prevent people from accepting nonsense as truth."
According to the professor, a single disclaimer at the start of a chat — "AI can make mistakes" — is not enough. That's why he argues that tech companies should be required to display uncertainty indicators.
For example, an AI that says within its answer, "I have low confidence in this part" or "Please verify the source," gives readers a reason to pause.
A similar problem is visible in Japan. Security firm Eltes published a survey in September 2026 targeting 400 people who regularly use generative AI at work.
In that survey, 81.1% said they had experienced hallucinations — either "many times" or "occasionally."
Of those who had been misled by incorrect information to the point that it affected their work, 33.4% reported an impact — with 17% involving materials or submissions for external audiences, and 16.4% involving internal decision-making or reporting.
The most common way people caught errors was through their own fact-checking, at 64.2%. On the other hand, 13.5% said they don't fact-check at all.
Eltes draws attention to the fact that among the 54 people who do not fact-check, 50 also had no preventive measures in place — and all of them reported zero hallucination experiences. The inference is that they likely "just haven't noticed." It should be noted, however, that this is the survey company's interpretation, not a confirmed finding.
While waiting for tech companies to act, there are things we can do ourselves.
For figures, names, and dates you plan to use at work, always check them against the original source material or official websites.
Try asking: "What sources support this?" and "Which parts are you uncertain about?" Then verify for yourself that any URLs provided actually exist.
Skilled writing and accurate content are two different things. This research showed that people tend to conflate the two without realizing it.
The number of participants was not stated in the university's announcement or the GIGAZINE article. All that is known is that participants were selected for having basic knowledge of the source material.
It's the phenomenon where AI produces a response that isn't factually true, expressed in plausible-sounding language. The term is translated as "hallucination" (幻覚).
Not necessarily. The experiment also included a condition with matter-of-fact language mixed with fabrications. Regardless of speaking style, content still needs to be verified.
This experiment used conversations about The Lord of the Rings and was not a study of Japanese services. That said, as the Eltes survey shows, users in Japan are also experiencing hallucinations at a high rate.
The next time you use an AI's answer, try checking even just one number or proper noun against an original source.
This article is a cross-post from AI Friends.