AI Risk & Limitations — Part 6: Missing Documents

When AI silently omits handwritten notes, faded faxes, and pages outside its relevance threshold, the missing document only surfaces under cross-examination. Case Chronology® equips experts to see what the AI could not — and build a verified opinion that accounts for both.

Published May 22, 2026 · Updated June 01, 2026

AI Risk & Limitations — Part 6: Missing Documents

In the quiet hours before a deposition or the final weeks before trial, the fear that haunts every expert witness and litigator is the same: the one document that changes everything, the page no one noticed. For years, that anxiety lived in the realm of human fallibility — overlooked exhibits, misfiled records, the sheer volume of paper that no single person could reasonably absorb. Now, as more legal and medical-legal teams turn to artificial intelligence for help sorting through massive productions, a new version of that fear has emerged. The documents are still there, sitting in the digital folder. But the AI never saw them.

The problem rarely announces itself with drama. It starts quietly. A prompt is written in a hurry. The system returns a confident summary, complete with citations. Everything looks solid until, days later, someone manually reviews the underlying material and discovers entire categories of records the AI never considered. Sometimes the miss is the result of a poorly phrased prompt — a question that was too narrow, too literal, or simply failed to anticipate how the case material was organized. A better prompt, asked the right way, often surfaces what was previously invisible. The fix seems straightforward on the surface. In practice, it requires the user to know exactly what they are looking for before they ask.

Even when the prompt is carefully written, another limitation quietly intervenes. Many AI systems are engineered for speed. To deliver fast answers, they apply internal filters that return only the “most relevant” documents — sometimes a fixed percentage of the collection, sometimes an absolute number the model’s designers have determined is sufficient. The trade-off is understandable in theory: quicker responses, lower token usage, a cleaner output. In the real world of litigation, that trade-off can be dangerous. A document that seemed peripheral at first glance may become central once the timeline or the theory of the case shifts. When the system has already decided, for efficiency’s sake, to ignore everything beyond its relevance threshold, those pages simply disappear from consideration. The user rarely receives a clear warning that anything was left behind.

The most stubborn version of the problem, however, has nothing to do with prompts or relevance thresholds. It has to do with the documents themselves.

Real-world medical and legal records are noisy in ways that clean benchmark data never captures. Doctors’ handwritten notes are legendary for their illegibility. Faxes arrive with streaked ink and faded text. Scanned pages carry watermarks, binder holes, coffee stains, or the ghost images of pages that were copied on top of one another. Some records are low-resolution PDFs of microfilm. Others are images of carbon copies or handwritten margin notes added years after the original document was created. Modern AI models are remarkably good at reading clean, well-formatted text. When confronted with the actual detritus of a complex case, they simply cannot see what is there. The system does not hallucinate these pages into existence; it quietly omits them. And because the omission is silent, the user has no easy way of knowing what was missed.

At a few hundred pages, the problem is manageable. A careful human reviewer can spot the gaps. At thousands or tens of thousands of pages — the scale many of our clients now work at — the gaps become invisible. The AI returns its analysis. The expert builds an opinion on what appears to be a complete record. And only later, sometimes under cross-examination, does the missing document surface.

This is the uncomfortable reality behind the polished accuracy numbers one often sees in marketing materials. Those figures are usually derived from tidy test sets of clean data. They do not reflect the messy, imperfect, high-stakes collections that actually decide cases. In a courtroom or a peer-review setting, “mostly right” is not good enough. A single overlooked page can unravel an entire opinion. When the expert or the treating physician is asked how they reached their conclusion, the answer must stand up to scrutiny. “The AI told me so, and I didn’t realize it had ignored half the handwritten notes” is not a defense anyone wants to offer.

The solution cannot be to ask users to double-check everything manually. That defeats the purpose of using AI at scale. Nor can it be to pretend the problem does not exist. What is required is a system designed from the ground up to acknowledge the limitations of real-world data and to make those limitations visible. The platform must give experts not only the AI’s answer, but the tools to understand what the AI could not see — and the means to surface it themselves.

Case Chronology® was built precisely for these conditions. Its Validation Tool Suite does not assume the AI will always find everything. Instead, it equips the human expert to interrogate the record from multiple angles, to spot the silent omissions, and to build a verified opinion that accounts for both what the AI found and what it could not.

Case Chronology® — A Verified Opinion You Can Trust