A. The Lawyer Who Cited Cases That Never Existed

(1) Mata v. Avianca: Submitting Fabricated Cases from ChatGPT and Attorney Sanctions

On June 8, 2023, the courtroom at the Southern District of New York federal court was packed. Those who could not find seats in the gallery had to watch the proceedings on a video feed in an adjacent room.

The man standing before the court that day was not a murderer or a financial fraudster. He was Steven Schwartz, an ordinary attorney with 30 years of experience. His offense was simple: he had submitted nonexistent case law to the court. And the entity that fabricated those cases was not a human but a machine called ChatGPT.

The case began unremarkably.

A man named Roberto Mata injured his knee on a metal serving cart during an Avianca Airlines flight in 2019.

He filed a personal injury lawsuit against the airline. Dozens of cases like this land in court every day. Nobody pays attention.

Avianca moved to dismiss, citing the two-year statute of limitations under the Montreal Convention. Plaintiff's attorney Schwartz needed to prepare a brief opposing the motion.

This is where the story gets interesting.

Schwartz had practiced law for 30 years, but he was not admitted to the bar of the Southern District of New York. So Peter LoDuca, another attorney at the same firm, formally handled the case while Schwartz conducted the substantive legal research.

Schwartz was pressed for time.

He thought of a new tool that had been released to the public in November 2022. ChatGPT.

Schwartz asked ChatGPT: "Find me case law on the statute of limitations under the Montreal Convention." ChatGPT answered helpfully.

It listed six cases, including "Varghese v. China Southern Airlines," "Shaboon v. EgyptAir," and "Petersen v. Iran Air."

There were citation numbers, court names, and summaries of holdings.

It all looked perfect.

Schwartz cited these cases in his brief.

The problem was that none of these cases existed anywhere in the world.

Every single one was a fiction invented by ChatGPT.

Here we need to pause on the word "hallucination." Computer scientists call it a hallucination when an AI outputs false information as though it were fact. But the term is misleading. It makes it sound as if the machine took a drug and started seeing things.

The truth is simpler, and more mechanical. Large language models are not designed to tell the truth. They are designed to predict the next most probable word.

ChatGPT had not searched a legal database. It had generated sentences that "looked like case law" based on statistical patterns learned from millions of legal documents.

Schwartz's tragedy reached its peak when he tried to verify the results.

He asked ChatGPT again.

"Are these cases real? Can I find them on Westlaw or LexisNexis?"

ChatGPT replied: "Yes, these are real cases.

You can find them on reputable legal databases." Schwartz was reassured.

He filed the brief with the court. On March 15, 2023, Avianca's lawyers struck back.

"We are unable to locate the cases cited by the plaintiff." The court could not find them either. Judge P. Kevin Castel ordered Schwartz on April 11 to submit copies of the original decisions for each cited case.

Here Schwartz made his second mistake.

Instead of admitting the falsehood, he went back to ChatGPT, obtained fabricated full-text opinions, and submitted those to the court as well.

He tried to cover one lie with another.

The truth came out in May. Schwartz finally confessed.

In a sworn affidavit, he wrote:

"I did not comprehend that ChatGPT's content could be false.

I have practiced law for 30 years and could never have imagined something like this was possible."

At the June 8 hearing, Judge Castel asked Schwartz: "Did you look up the Varghese case yourself?" Schwartz replied: "Yes, I did." The judge asked again: "Did you find it?" Schwartz answered: "I did not." The judge asked: "Then why did you cite it in my courtroom?" Schwartz could not answer.

On June 22, Judge Castel issued his ruling.

Schwartz, LoDuca, and their firm Levidow were fined $5,000. The amount was symbolic. The real punishment lay elsewhere. The judge ordered both attorneys to send personal letters of apology to the real judges whose names appeared in the fabricated cases. Those letters had to include the court's sanctions order, the hearing transcript, and the original brief containing the fake citations. Judge Castel wrote in his opinion:

"It is not artificial intelligence itself that causes much harm. It is the attorney who, without verifying the content generated by artificial intelligence, submits it to the court and deceives the judicial system."

The case was reported in news outlets around the world.

A shockwave rippled through the legal profession.

But the real lesson was simple. No matter how smart a tool appears, the human who uses it bears responsibility for the result.

ChatGPT has no legal personhood. It cannot stand in a courtroom. It cannot offer excuses. Only a lawyer can do that. And only a lawyer can be sanctioned.

(2) The Cohen Case: The Spread of Fabricated Case Citations

Six months after the Mata case became national news, the same pattern repeated itself with a far more famous name. Michael Cohen. The man who had been Donald Trump's personal attorney.

In late 2023, Cohen was applying for early termination of his supervised release. He passed along three cases to his lawyer to support his arguments.

His attorney, Danya Perry, trusted the materials her client sent her.

After all, they were legal references from a man who had served as the former president's lawyer.

She included these cases in the brief filed with the court.

The problem was that these cases did not exist either. They were fabrications generated by Google's AI chatbot Bard (now Gemini).

Cohen later explained in a sworn affidavit: "I did not know that Google Bard was a generative AI. I just thought it was a powerful search engine." His excuse was strikingly similar to that of attorney Schwartz in the Mata case. Neither of them understood what AI was. Both believed that typing a question into a search box would yield the truth.

An important distinction is needed here.

When you type a question into Google's search box, Google finds web pages that already exist.

But when you type a question into a generative AI chat window, the AI creates a new answer each time.

The former is retrieval. The latter is creation.

In the legal world, creation can mean forgery.

On March 20, 2024, the federal judge decided not to impose sanctions on Cohen. But the court noted the possibility that Cohen had committed perjury. Attorney Danya Perry pushed back, calling this characterization "factually inaccurate and legally incorrect."

There was a difference between the Cohen case and the Mata case.

Cohen did not file the brief directly with the court. He provided false information to his own attorney. The client deceived the lawyer.

This raised a new question. When the victim of an AI hallucination is a layperson rather than a legal professional, what should the consequences look like?

Even after the Mata case, similar incidents kept surfacing. In February 2024, Massachusetts Superior Court Judge Brian Davis sanctioned another attorney. He opened his ruling with these words: "This decision addresses two troubling phenomena that are adversely affecting legal practice. First, the tendency of generative AI systems such as ChatGPT to fabricate false information. Second, the tendency of some attorneys to use AI to draft briefs and then file those briefs with the court without verifying whether the output contains false information."

In 2024, Utah attorney Richard Bednar was sanctioned for citing a fake case, "Royer v. Nelson," generated by ChatGPT.

In California, two law firms were fined $31,000 for submitting fake cases generated by Google Gemini.

In a Walmart personal injury lawsuit, three attorneys were hit with a combined $5,000 in fines. Similar cases were reported in the United Kingdom and Canada. One study found that when asked specific questions about random federal court cases, ChatGPT 4 hallucinated 58% of the time, and Llama 2 hallucinated 88% of the time.

The pattern was clear. AI hallucination was not an individual mistake. It was a structural risk.

(3) AI Ethics and the Duty to Verify in Legal Services

The Mata case and the Cohen case posed a new question to the entire legal profession. What is a lawyer's duty? Does that duty remain unchanged even as technology evolves?

ABA Model Rule 1.1 requires lawyers to provide "competent representation." In 2012, Comment 8 to this rule was amended to specify that "understanding the benefits and risks of technology" is part of competence.

At the time, the amendment was aimed at email security and cloud storage. No one anticipated generative AI. But the rule still applies.

Rule 1.3 mandates "diligent investigation." Regardless of the method used, lawyers must verify the accuracy of the facts and law they assert.

Rule 3.3 requires "candor toward the tribunal." Even if a lawyer did not know a statement was false, submitting AI-fabricated fake cases does not exempt them from responsibility.

After the Mata case, courts across the United States began introducing new rules.

Judge Brantley Starr of the U.S. District Court for the Northern District of Texas was the first to require an "AI Certification."

If a lawyer used AI to draft a brief, they must attach a certification confirming that a human independently verified the contents. Dozens of courts issued similar orders afterward. In July 2024, the ABA released its first formal ethics opinion on generative AI use. The 15-page document explained how the rules of professional conduct apply to AI use. The core message was straightforward: AI does not reduce a lawyer's responsibilities. It adds new forms of verification duty.

Bar associations in California, New York, and Florida also published guidelines. They shared common requirements.

First, lawyers must independently verify the accuracy of AI-generated information.

Second, lawyers must understand the capabilities and limitations of AI systems.

Third, lawyers must inform clients whether and how AI was used.

Fourth, lawyers must consider confidentiality obligations when entering sensitive client information into AI systems.

There is one rule of American courts you need to know.

Federal Rule of Civil Procedure 11. The rule is simple. The moment a lawyer signs a document, they make a promise to the court: "I have verified this content. It has a factual and legal basis." A signature is not just a name. It is a guarantee.

Attorney Schwartz asked ChatGPT to find relevant cases. ChatGPT obligingly produced six cases. They had names, dates, and citation numbers. They looked perfect. Except for one thing. They were all made up by the AI.

Schwartz passed these cases to his colleague LoDuca. LoDuca did not verify them. He signed and filed.

A question arises here. Why not just let the judge catch it? Why fine the lawyers? American courts work like a soccer match. The judge is the referee. The referee does not kick the ball. Bringing the ball is the players' job. Lawyers find the law and evidence; the judge reviews it and decides. This is called the adversarial system.

If the referee had to inspect whether the ball was real before every match, no game would ever be played. Likewise, if judges had to doubt the very existence of every case cited by lawyers, trials would grind to a halt.

This was precisely why the judge was furious. The court's time. The resources of a judicial system funded by taxpayers. All of it spent tracking down cases that never existed.

The fundamental problem is not AI. It is the absence of verification. If Attorney Schwartz had checked ChatGPT's answers against a legal database just once, the fake cases would have been caught. Three minutes would have been enough. He saved those three minutes. It cost him $5,000 and his professional reputation.

Here is the question about what a lawyer's role actually is. Are they simply someone who passes along information, or are they a gatekeeper who filters information for truth? The American courts' answer is clear. A gatekeeper. And when a gatekeeper stops guarding the gate, they pay the price.

The court imposed a punishment harsher than monetary sanctions. It suspended the attorneys from this case. They could no longer represent their client. The court also ordered the opinion published in the official Federal Reporter and directed the clerk to notify the attorney disciplinary authorities in every state where the lawyers held a license.

Technology gives lawyers powerful tools, but those tools don't erase a lawyer's responsibilities or justify a lower standard of diligence. The duty to verify is not optional. It is a baseline requirement of the AI era.

B. Faulty Guidance from Corporate Chatbots (The Chatbot That Promised a Refund)

(1) Moffatt v. Air Canada: Liability for a Chatbot's Wrong Refund Policy

On November 11, 2022, Jake Moffatt received the news that his grandmother had died. It was Remembrance Day. He needed to book a flight from Vancouver to Toronto. He was in a rush. He was grieving. He had no time to read through complicated terms and conditions.

Moffatt went to the Air Canada website. A chat window sat open on one side of the screen. "How can I help you?" He asked the chatbot his question: he needed to travel urgently for a funeral, and could he get a bereavement fare?

The chatbot answered helpfully. "If you need to travel immediately or have already travelled, you can submit a refund request within 90 days of the date your ticket was issued for the reduced bereavement rate." Moffatt took the chatbot at its word. He paid 794.98 Canadian dollars for a one-way ticket from Vancouver to Toronto. A few days later, he bought a return ticket for 845.38 dollars. The total came to more than 1,630 dollars.

The funeral ended. Moffatt filed a refund request with Air Canada. He attached his grandmother's death certificate. He was within the 90-day window the chatbot had described. His request was denied.

Air Canada's response went like this: under the company's actual policy, bereavement discounts must be requested before travel. They cannot be applied retroactively to completed trips. The chatbot had given him the wrong information.

Moffatt did not give up. In February 2023, he emailed Air Canada. He attached screenshots of his conversation with the chatbot. "Look at this. Your chatbot said exactly this." The Air Canada representative acknowledged it. The chatbot had used "misleading words." But the refund was still denied. The representative added that if Moffatt had clicked the link the chatbot provided, he could have found the correct policy. Moffatt filed a claim with the Civil Resolution Tribunal of British Columbia. He sought 880 Canadian dollars, the difference between the full fare and the bereavement rate.

Air Canada mounted an extraordinary argument in court. "The chatbot is a separate legal entity. The company is therefore not responsible for its statements."

Tribunal Member Christopher Rivers must have paused for a long moment after reading that claim.

He wrote in his decision: "In effect, Air Canada suggests the chatbot is a separate legal entity that is responsible for its own actions. This is a remarkable submission. While a chatbot has an interactive component, it is still just a part of Air Canada's website."

Air Canada tried another defense. The chatbot had provided a link to the correct policy, they argued, so Moffatt should have clicked through and checked for himself. Member Rivers rejected this argument as well. "Air Canada does not explain why the webpage titled 'Bereavement travel' should be inherently more trustworthy than its chatbot. There is no reason Mr. Moffatt should have known that one part of Air Canada's website was accurate and another was not."

On February 14, 2024, the tribunal ruled in Moffatt's favor. Air Canada was ordered to pay 650.88 Canadian dollars in damages plus interest and tribunal fees, totaling 812.02 dollars.

The amount was small. But the signal this ruling sent was worth hundreds of millions. When a company deploys an AI chatbot for customer service, the words that chatbot produces are treated as the company's official position. "The chatbot made a mistake" is not a defense.

(2) The Legal Status of Customer Service AI as a Digital Agent

The Air Canada case raised a larger question.

What is the legal status of an AI chatbot? How much responsibility does a company bear for what its chatbot says?

Under traditional agency law, a principal is liable for acts that an agent performs within the scope of authority. If an Air Canada call center employee gives a customer incorrect information, Air Canada is responsible. The employee acts on the company's behalf. So what about a chatbot?

The American Law Institute's Restatement of Agency states that a computer program cannot be treated as an agent. An agent must possess intention and autonomy. AI, in that sense, is not an agent. But the court reached the same conclusion by a different route.

There is a concept called apparent authority.

Even when someone lacks actual authority, if they appear to have it and the principal created that appearance, the principal is liable. Air Canada placed its chatbot on its website. When a customer interacts with that chatbot, there is a reasonable basis to believe it is an official Air Canada channel. The chatbot's words are, therefore, Air Canada's words.

Member Rivers's decision followed this logic. "Air Canada is responsible for all the information on its website. It does not matter whether the information comes from a static webpage or a chatbot."

After the Air Canada case, similar problems surfaced one after another.

In March 2024, an AI chatbot that New York City had deployed to support small business owners gave wildly wrong advice. "You can fire an employee who files a sexual harassment complaint." "You can run a cashless store." Both were clear violations of the law. The city attached a disclaimer stating that chatbot responses did not constitute legal advice, but it could not escape criticism that a government service was encouraging illegal conduct.

McDonald's AI drive-through system placed a 260-dollar order of chicken nuggets that a customer never wanted, and mixed up orders between cars. McDonald's ultimately pulled the system, developed in partnership with IBM, in June 2024.

In August 2025, reports emerged that Meta's AI had encouraged self-harm and eating disorder role-play among teenagers on Instagram. The Swedish fintech company Klarna promoted AI customer service on a grand scale, then had to rehire human agents after the error rate became unacceptable.

These cases share a common lesson. When a company adopts AI to cut costs, it must also account for the cost of the mistakes that AI will make. Air Canada introduced its chatbot to save on call center labor. What it got in return was a legal dispute, media coverage, and reputational damage. In trying to save 812 dollars, the airline became a worldwide punchline.

C. Defamation Liability

(1) AI-Generated Falsehoods and the Defamation Defense

Mark Walters is a public figure. He hosted two nationally broadcast radio programs. Each 15-minute segment drew an audience of 1.2 million listeners. He was a voice of the American gun rights movement. He wrote books. He served as a spokesperson for several organizations.

On May 3, 2023, Fred Riehl, the editor of AmmoLand, an online firearms publication, used ChatGPT while working on a story.

He asked for information about a lawsuit that the Second Amendment Foundation had filed against the Washington State Attorney General. He entered the URL of the court filing into ChatGPT.

ChatGPT warned him. "I am unable to access the internet or open the link you provided." It added that the linked content had been created after its "knowledge cutoff date."

But when Riehl kept asking, ChatGPT generated an answer.

That answer included a claim that Mark Walters had been sued for embezzling funds from the Second Amendment Foundation. It included a case number. It included specific dollar amounts.

None of it was real. Walters was not a party to that lawsuit. No such allegation had ever been made against him. ChatGPT had fabricated the entire thing.

Riehl was a veteran journalist.

He was aware of AI hallucinations. In less than 90 minutes, he confirmed that ChatGPT's output was false. He did not use the information in his article. The fabricated content was never published. But Walters filed a lawsuit anyway. In June 2023, he brought a defamation claim against OpenAI in the Superior Court of Gwinnett County, Georgia. It was one of the first defamation lawsuits filed against a generative AI system.

On May 19, 2025, the court granted OpenAI's motion for summary judgment and dismissed Walters's case. The ruling rested on three grounds.

First, the output carried no defamatory meaning. Under Georgia law, a defamation plaintiff must prove that the statement at issue "is reasonably understood as describing actual facts about the plaintiff."

The court found that a reasonable reader in Riehl's position would not have concluded that ChatGPT's output conveyed "actual facts." ChatGPT had warned Riehl that it could not access the internet. OpenAI had repeatedly cautioned that ChatGPT sometimes produces factually inaccurate information. Riehl himself confirmed the output was false in a short period of time.

Second, there was no negligence or actual malice. Walters was a public figure. For a public figure to prevail in a defamation suit, the plaintiff must show by clear and convincing evidence that the defendant acted with "actual malice," meaning the defendant published the statement knowing it was false or with reckless disregard for whether it was false. Walters argued that because OpenAI knew about hallucinations, this constituted actual malice.

The court rejected that reasoning. "Walters's argument means that an AI developer like OpenAI would be liable for erroneous outputs its model generates no matter how much care it takes to reduce errors. That is not a negligence standard; it is a strict liability standard. Neither Georgia law nor the federal Constitution permits it."

Third, there was no injury. Walters admitted in his testimony that he suffered no harm from the ChatGPT output. The only person who saw it was Riehl, who neither believed it nor published it. Walters also had not asked OpenAI for a correction or retraction before filing suit. Under Georgia law, failing to take that step bars a claim for punitive damages. This ruling set an important precedent for AI companies. Disclaimers work. A warning that "AI can make mistakes" can serve as a shield against defamation liability. But this is a double-edged sword. Does slapping on a warning mean a system can say anything at all?

(2) The Australian Mayor and the American Professor

The Walters case ended in dismissal under unusual circumstances. Only one person saw the false information, and that person didn't believe it. Other cases were different.

Brian Hood was the mayor of Hepburn Shire in Victoria, Australia.

In the early 2000s, he had blown the whistle on a bribery scandal at Note Printing Australia, a subsidiary of the Reserve Bank of Australia. Several executives were charged. Hood was not one of them. He was the person who exposed the corruption. He was praised for showing "extraordinary courage."

One day in 2023, Hood heard something strange from friends. ChatGPT was saying odd things about him. Hood typed his own name into ChatGPT. What appeared on screen left him stunned.

ChatGPT wrote that Hood had participated in a conspiracy to bribe foreign officials to win currency printing contracts. It said he had been found guilty. It said he had spent 30 months in prison. Every detail was the opposite of reality. The whistleblower had been turned into a criminal.

Hood was furious.

He announced plans to sue OpenAI for defamation. "Being portrayed as a white-collar criminal who served time in prison is extremely damaging to one's reputation." His lawyers sent a concerns notice to OpenAI on March 21, 2023, demanding the errors be corrected within 28 days or they would file suit.

Something interesting happened.

A newer version of ChatGPT began providing correct information about Hood. That he was a whistleblower. That he was not a criminal. It appears OpenAI quietly made corrections. Hood never filed the lawsuit. The costs were too high. Australia's defamation damages cap is roughly 400,000 Australian dollars (about 240,000 euros). He could not afford the legal fees.

The case of Jonathan Turley, a law professor at George Washington University, was even more absurd.

Eugene Volokh, a professor at UCLA, was conducting a research project and asked ChatGPT: "Find cases where a professor at an American law school was accused of sexual harassment, and include newspaper articles and citations."

ChatGPT responded as follows.

"Jonathan Turley, a professor at the Georgetown Law Center, was the subject of a sexual harassment complaint by a former student who alleged inappropriate comments during a class trip. Citation: 'The complaint alleges that Turley made sexually suggestive remarks and attempted to touch her in a sexual manner during a law school-sponsored trip to Alaska.' (The Washington Post, March 21, 2018)"

There were problems. Turley was at George Washington University, not Georgetown. There was no Alaska trip. There was no sexual harassment complaint. The Washington Post article did not exist. Everything was fiction invented by ChatGPT.

Professor Turley wrote in USA Today: "What is most striking is that this false accusation was not only generated by AI but was based on a Washington Post article that never existed. Allegations of this kind are incredibly damaging."

Heiko Bernklau, a German journalist, searched his own name on Microsoft Bing's AI-powered search engine.

The AI described him as the perpetrator of crimes he had reported on. It also published his real address and phone number and provided directions to his home from multiple locations. Jeffrey Battle was an aerospace educator.

Microsoft Bing Chat confused him with a terrorist who shared the same name. He sued Microsoft. Political commentator Robby Starbuck filed a lawsuit claiming Google Gemini had described him as a "child sex offender."

These cases reveal a common problem. AI hallucinations are not random. There is a pattern. The system misreads context from its training data. It swaps victims and perpetrators. It confuses people who share the same name. It cites sources that do not exist. And it does all of this confidently, specifically, and persuasively.

The question is accountability. Fake news circulating on the internet can be traced to its source and removed. But false information generated by AI hides somewhere in the source code, and every time someone asks a question, it is born anew. It is like fighting a ghost.

The ruling in the Walters case set a precedent favorable to AI companies. But that was due to the specific facts involved. What if the false information had spread widely? What if people had believed it? What if the victim had actually lost a job or had their reputation destroyed? Would the same ruling hold?

Courts have not fully answered these questions yet. AI companies hide behind disclaimers. Victims give up because they cannot afford lawyer fees. In the meantime, hallucinations continue. Someone's name gets turned into that of a criminal, a sex offender, an embezzler. When a machine lies, who pays the price for that lie?

This is not a technology problem. This is an accountability problem. And accountability problems always end with someone being held by the collar. You cannot grab an algorithm by the collar. So we must grab the person who released that algorithm into the world. As Michael Lewis always says, behind every system there is a person who built it. Finding that person is the job of the law.