Your company assistant sounds sure of itself, cites its source, and is quietly quoting a document three versions out of date. Here's why it happens, how to spot it, and how to stop it.
Somewhere in your shared drive there are two documents called Refund Policy. One is from 2021. One is current. Same logo, same headings, same first page. The difference is one line on page four: the old one gives customers 60 days to request a refund. The current one says 30.
Your team asks the company AI about refunds, and it answers calmly, in full sentences, with a citation:
"You have 60 days to request a refund."
The model did exactly what it was asked: it read the document it was handed and summarised it accurately. The mistake happened one step earlier, when the system decided which document to hand it.
Most assistants fetch a few passages that look relevant and answer from those. "Looks relevant" usually means "uses similar words", not "is the current version". And once the answer comes back, every clue a person would have used is gone: no file date, no "last edited by", no folder called ARCHIVE_DO_NOT_USE.
"How long do customers have to request a refund?"
The old copy happens to match the question's wording best, so it's the one the AI receives.
The AI summarises it flawlessly. From the wrong document.
Your AI answers from whichever copy it reached first, and never mentions the others disagree.
Documents that share a name are paired, and your AI sees exactly what differs before it answers.
Your AI read it last month and keeps repeating the old version.
Your AI is told when something it read has changed since, so it re-checks instead of repeating stale text.
Duplicates flood the answer, and the one document that says something different is buried.
Each distinct passage comes back once, so the one that differs stands out.
Your AI picks one definition of "active customer" and never mentions the other exists.
You're warned when one name is defined in several places, so the answer can say "it depends which definition you mean".
The handbook still points to it, and your AI answers as if it's there.
Everything that's referenced but missing is listed, so your AI can say the procedure doesn't exist.
Archive old files, remove duplicates, keep one source of truth: good advice, and a project that never ends. People save copies, export files and duplicate folders "just for now". You need an AI that copes with the mess, not a promise that the mess will go away.
ARR doesn't change your model. It changes what the model is handed: the right material, with the warning signs left in.
We've measured these abilities in our own benchmark on code and text. We haven't yet run it on a real company's shared drive, so treat this as how ARR is built to work here, not a measured result. That's exactly what early access is for.
If it quoted the old one, you're not looking at an AI problem. You're looking at a retrieval problem, and that's the part ARR was built to fix.
We're opening ARR to a small group first. Tell us a little about your setup and we'll be in touch when your place is ready. No payment required.
More in ARUKAS Field Notes:
Next: When legal AI gets the clause wrong
Chunking breaks meaning
Why AI coding agents break things three files away
Running parallel coding agents without them overwriting each other
Which version of the policy did your AI just apply?
AI citation errors start before the AI writes anything
Duplicate files, duplicate totals: where AI reconciliation goes wrong
What AI prior-art search doesn't tell you it missed
How ARR works →