AI Risk & Limitations - Part 7: Hallucinations — Association Risk

Every name on every page of a modern litigation file — judges, attorneys, expert witnesses, treating physicians, paralegals — can be silently rebranded by AI through hallucinated associations. The result is a kind of AI Scarlet Letter: an invisible mark the person rarely learns about until it surfaces in a screen, a credentialing review, or opposing preparation.

Published May 25, 2026 · Updated June 01, 2026

In the official record of nearly any court case, names accumulate with remarkable persistence. Judges sign orders. Attorneys file motions and appear in transcripts. Expert witnesses submit reports. Treating physicians, nurse consultants, life-care planners and IME examiners contribute charts, opinions and testimony. Emergency responders document incidents that become exhibits. Medical providers whose charts and notes enter the file. Each of these individuals leaves a digital footprint across discovery materials, chronologies, depositions and public filings.

What was once confined to courthouse archives or sealed folders now exists in digitized, searchable form. And in an age of large language models trained on vast internet scrapes and public records, that visibility carries a hidden cost few see coming. Everyone mentioned inside those case documents may be receiving an AI Scarlet Letter — a modern, invisible brand of stigma — without ever knowing it exists.

The Scarlet Letter, reborn as an algorithm

The metaphor is uncomfortable but precise. In Nathaniel Hawthorne's novel, Hester Prynne wore a scarlet "A" sewn to her clothing, a public mark of shame visible to everyone but impossible for her to remove. Today's version is quieter, more insidious, and algorithmic. An AI model ingests a name from a court filing, a medical chart, or a deposition transcript. It then hallucinates associations — reassembling that name with nearby allegations, outcomes, or context from entirely different matters the person merely touched as a professional. The resulting profile, summary, or background check brands the individual with associations they never earned. And because the hallucination is opaque and automated, the person wearing the mark rarely learns it exists until it surfaces in an insurance application, a job screen, or a credentialing review.

A stark illustration: the German court reporter who became the criminal

A stark illustration comes from Martin Bernklau, a veteran German court reporter based in Tübingen. For years Bernklau covered criminal trials, filing factual accounts of cases involving child abuse, fraud and other serious crimes. In 2024 he decided to test Microsoft's Copilot by entering his own name and location. The AI did not summarize his journalistic work. Instead, it falsely portrayed him as a convicted child abuser who had preyed on minors, a conman who targeted widows, an escapee from a psychiatric institution, and the perpetrator of multiple other offenses he had merely reported on as a journalist. The model had hallucinated direct associations between Bernklau's bylines and the defendants whose trials he had covered — turning a neutral court reporter into the subject of the very stories he once told.

Bernklau's experience is not an isolated glitch. It is a preview of the AI Scarlet Letter that now threatens every name on every page of a modern litigation file. A treating physician's name appears repeatedly in a high-profile malpractice matter. An expert witness's signature sits on a life-care plan. A paralegal's initials mark dozens of discovery logs. Each becomes training signal or inference fodder that an AI can later hallucinate into something damaging. The association is not a search result; it is a synthesized narrative that can travel far beyond the original docket.

Two layers of risk: training contamination and inference

This problem has two layers. The first is training data contamination. Many general AI systems incorporate publicly available court filings, news articles and document dumps into their training corpora. Names, addresses, medical details and professional credentials that appear in these records become part of the model's statistical understanding of the world. Once embedded, those false associations are difficult to remove — the digital equivalent of a scarlet letter that cannot be unpinned.

The second layer is inference-time risk. Even without permanent training, a single query that includes a name and a few contextual keywords from a case can cause a model to fabricate connections drawn from broader web data or prior case material. The hallucinated output may never be published, but it can quietly shape internal memos, draft reports or an opposing party's preparation. In high-stakes litigation and medical-legal work, where accuracy and accountability are paramount, these subtle distortions matter — and the people whose names appear in the documents often remain unaware they have been marked.

Disclosure norms haven't caught up

Legal and medical professionals whose names populate these records are increasingly aware of the exposure. Yet current court rules, discovery protocols and expert disclosure requirements have not fully caught up. There is rarely a formal obligation to disclose which AI tools were used in case preparation, whether sensitive case data was kept private or fed into external models, or what validation steps were taken to ensure outputs were grounded in verified source documents.

This information asymmetry undermines trust in the process itself. Opposing counsel, judges, juries and the public have a legitimate interest in knowing how the record was analyzed and whether AI played a role — and if so, under what safeguards.

It may be time to establish a clearer norm of transparency. Law firm teams on both sides, expert witnesses, treating physicians, medical reviewers, paralegals and litigation support staff could be expected to disclose, at minimum, their approach to AI use in a given matter. Not vague assurances, but specifics: which tools were employed, whether data remained within a private environment or was exposed to training pipelines, and how AI-generated content was validated against original source documents. Such disclosures would not hinder the adoption of helpful technology. They would strengthen confidence that the opinions and analyses presented to the court rest on defensible foundations.

Privacy as a precondition of fairness

In an era when AI can rapidly synthesize and recirculate information from the public record, protecting the privacy of those named in case documents is not a technical detail. It is fundamental to preserving fairness, professional reputations and the integrity of the justice system — and to ensuring that no one walks into a deposition, a credentialing hearing or an insurance review carrying an AI Scarlet Letter they never knew they had been given.

How Case Chronology® was designed for this reality

Case Chronology® was designed with this reality in mind. Privacy is core to the platform: it does not send case data — including the names and sensitive information of those involved in matters — back to any AI model for training. This controlled environment helps ensure that the professionals whose names appear throughout the documents can work with AI assistance while maintaining accountability for humans and accountability for AI. Case Chronology® — A Verified Opinion You Can Trust.