The Dark Side of AI: Why ChatGPT Invents Facts and the 3 Most Dangerous AI Hallucinations That Shocked the World

From the Famous ChatGPT Lawyer Scandal to the Air Canada Lawsuit: The Real Reason AI Hallucinates, Why People Trust It Blindly, and the Hidden Dangers for Your Health, Career, and Money

The picture was made by the author with the help of an AI program

The biggest problem with artificial intelligence today is no longer its inability to perform tasks. Ironically, it is the exact opposite: its ability to sound convincing even when it is completely wrong.

Almost everyone has experienced a situation where an AI system provided an answer that made no sense, was completely incorrect, or was entirely fabricated. Most of us simply laugh at the mistake, perhaps share it with friends or on social media if it is particularly amusing, then ask the question again, slightly adjust the prompt, and continue as if nothing serious happened.

But what if your question to AI concerned something critical to your job, your health, or your financial future, and the answer you received was a complete hallucination? What if you never bothered to verify it?

Believing that your AI assistant was always right and incapable of making mistakes, you placed your full trust in it, only to be misled to the point where you risked losing your job, damaging your health, or losing your hard-earned money in a matter of seconds.

In this article, we will examine the dangers that arise when we place too much trust in AI and allow it to make decisions about our lives without verification, analysis, or oversight. We will also explore three of the largest and most famous AI hallucinations that threatened the people who relied on them.

Finally, we will examine why people are so willing to place their fate in the hands of artificial intelligence and the psychological mechanisms that help explain this behavior.

What Is an AI Hallucination and Why Is It Called That?

An AI hallucination is a situation in which artificial intelligence produces an answer that sounds convincing, logical, and confident but is actually incorrect, fabricated, or unsupported by real facts. In simple terms, the AI invents information and presents it as if it were true.

Although the term "AI hallucination" is now widely used, its exact origin is difficult to trace because no single individual can be definitively credited with creating it. The word "hallucination" was borrowed from psychology and medicine, where it describes the perception of something that does not actually exist.

In computer science, the term began appearing as early as the 1980s, though it did not carry the same negative meaning it does today. It was often used in computer vision research to describe situations in which a computer added details to an image that were not present in the source material. One of the earliest documented examples appears in the doctoral dissertation of researcher Eric Mjolsness in 1986.

The modern meaning of the term evolved during the 2000s and became especially prominent after Google researchers used the word "hallucination" in 2017 to describe situations where AI systems generated content that was unsupported by the original data.

The term reached mainstream popularity following the release of ChatGPT, when journalists, researchers, and technology companies began using it extensively to describe instances in which AI confidently generated false or completely fabricated information. The concept became so widely recognized that in 2023, Cambridge Dictionary expanded the official definition of the word "hallucinate" to include a meaning related to artificial intelligence.

Why Does AI Invent Answers?

The simplest explanation is that artificial intelligence does not think like a human being and does not verify facts the way a journalist or scientist would.

Large language models predict which words are most likely to come next based on patterns learned from enormous amounts of data. When there is not enough information available, or when a question is vague, the model often attempts to fill in the gaps.

This is where hallucinations emerge: invented quotations, non-existent books, fabricated court cases, incorrect medical advice, or inaccurate historical information.

Why Do People Sometimes Blindly Trust AI?

Why do people place so much trust in artificial intelligence?

The answer lies in several well-known psychological mechanisms.

• The first is known as automation bias. Human beings have a natural tendency to trust computer systems more than their own judgment. If a GPS device tells someone to turn left, many people will do so even if the road ahead appears questionable. The same principle increasingly applies to artificial intelligence.

• The second factor is authority. ChatGPT and similar systems write in a grammatically correct, organized, and confident manner. People often mistake confidence for accuracy. When a response sounds as though it was written by a professor from Harvard, the human brain instinctively assumes the information must be reliable.

• The third factor is mental laziness. Verifying information takes time and effort. AI provides instant answers. Human beings naturally prefer the easier path. If an answer appears convincing enough, most users will not open additional sources or conduct independent research.

Three Famous Cases Where AI Got It Wrong

The ChatGPT Lawyer Case

Perhaps the most famous AI hallucination in history occurred in 2023 during the case of Mata v. Avianca.Two American lawyers used ChatGPT to locate legal precedents for a lawsuit against Avianca Airlines. ChatGPT provided several court decisions that appeared entirely authentic. They contained case numbers, judges' names, legal reasoning, and even quotations from the supposed rulings.

There was only one problem: none of the cases actually existed.

ChatGPT had fabricated them.

The lawyers failed to verify the sources and submitted the material to the court. When the judge attempted to locate the cited cases, it became clear they were entirely fictional. The court ultimately sanctioned the attorneys, the story became global news, and the phrase "ChatGPT lawyer" became synonymous with blind trust in artificial intelligence.

Even more remarkably, when the lawyers later asked ChatGPT whether the cases were real, the system insisted that they were and claimed they could be found in legal databases. It had not only invented the information but also defended its own invention.

The James Webb Space Telescope Error

The second major incident involved Google's Bard chatbot, the predecessor of today's Gemini.

During a public demonstration, Bard was asked a question about the James Webb Space Telescope. The AI claimed that the telescope had captured the first-ever images of planets outside our solar system.

The statement sounded impressive, but it was incorrect.

Astronomers quickly pointed out that exoplanets had been photographed long before the James Webb Space Telescope was launched. The mistake triggered significant media attention, and shares of Google's parent company lost tens of billions of dollars in market value in a single day.

The Air Canada Chatbot Case

The third case is particularly interesting because it demonstrates how AI errors can have direct financial consequences for ordinary people.

An Air Canada customer sought information regarding a bereavement fare policy after the death of a family member. The airline's chatbot informed him that he could purchase a ticket and later apply for a partial refund.

Following the chatbot's instructions, the passenger purchased the ticket and subsequently requested reimbursement. The airline denied the request, arguing that no such policy existed. The dispute eventually reached court, which ruled that the company was responsible for the information provided by its chatbot. The passenger won the case.

This and similar incidents have raised an entirely new set of questions. If artificial intelligence makes a mistake, who is responsible? The user? The company? The developer? Or the AI itself?

The law still does not have a definitive answer.

Particularly concerning is the fact that such incidents are no longer limited to individual users. During 2025 and 2026, several reports produced by consulting and research organizations were found to contain fabricated citations, invented studies, and inaccurate information generated with AI assistance. One widely discussed phenomenon became known as "vibe citing", the practice of citing sources that sound legitimate and convincing but do not actually exist.

The AI Problems Waiting for Us in the Future

Researchers increasingly warn that the combination of hallucinations and excessive trust may represent one of the greatest risks associated with generative artificial intelligence.

Medicine is a particularly concerning example.

When AI makes a mistake in a cake recipe, the result is often little more than an amusing social media post. When it makes a mistake regarding the symptoms of a heart attack or recommends inappropriate treatment, the consequences can be catastrophic or even fatal.

Recent research suggests that even highly advanced AI models can generate medical recommendations that sound professional while containing significant factual errors. An additional challenge is that these systems often appear most convincing precisely when they are wrong.

The same concerns apply to finance.

Imagine a user asking AI where to invest their life savings. If the model fabricates business information or misinterprets market trends, the resulting damage could amount to thousands or even millions of euros.

The greatest misconception of our time is not that AI is unintelligent. The greatest misconception is believing that it is infallible.

The truth is far more complicated. Today's AI models can write software code, analyze medical reports, draft legal documents, and summarize scientific research. At the same time, they can invent a non-existent court case, fabricate a scientific paper, or provide dangerously inaccurate medical guidance.

That is the paradox of artificial intelligence.

It is intelligent enough to help us, but not yet reliable enough to be trusted without verification.

Perhaps this is the most important lesson we have learned during the first years of the AI revolution. The problem is not that artificial intelligence occasionally makes mistakes. Human beings make mistakes every day.

The real problem begins when we forget that AI can make mistakes too.

The moment we stop verifying its answers, the moment we hand over responsibility for our health, finances, or careers, an AI hallucination ceases to be an amusing internet anecdote and becomes a genuine risk.

The past few years have shown that sometimes a single convincing lie from a machine is enough to cause a person to lose a court case, money, professional credibility, or even trust in their own judgment.