Documentation drift in AI assistants: when a stale source becomes a confident answer

Suppose your AI assistant tells a customer, in a warm and flawlessly punctuated sentence, that exports are capped at 10,000 rows. The customer wanted the whole quarter in one file, so she starts working out how to slice it into pieces. Three sprints ago, an engineer raised the cap to 50,000. He changed one number in a config file, wrote a perfectly good commit message, and went to lunch. The help article that says 10,000 lives in a different system, belongs to a different team, and has never heard of him. Nothing broke. No alert fired. The answer just quietly became false.

This is documentation drift: the product changes and the explanation stays put. What happens when an AI assistant uses that explanation to answer a customer, and what can catch the mismatch? The risk isn't that docs are inherently worse than code, or that an assistant without docs will politely admit defeat. It's that a stale source can give a wrong answer the appearance of evidence. An assistant can then deliver it in exactly the tone it uses for a correct one. Call that the confident stale answer.

Three objections will occur to you before we're done, and each has a real point, so they get a section of their own. First, how a doc ends up wrong when nobody did anything wrong.

How the doc gets left behind

A doc records an explanation of the product. Code records its implementation, though the version and configuration running for this customer matter too. The two can agree perfectly on Monday. On Tuesday, somebody merges a change that affects a documented limit. If the help article lives outside the code review and nothing checks that claim, the change has no reason to carry the explanation along with it. The gap can sit there until somebody notices.

Notice that nobody in this story did anything wrong. The engineer raised the cap for a good reason and recorded it accurately, in the place where engineers record things. Nobody merges a change thinking, “and now the help center is a lie.” Staleness has no author. It's the sum of a thousand correct decisions made inside a system that wasn't looking at the doc, which is why hiring more careful people doesn't fix it. The careful people are the ones who changed the number.

There's an asymmetry underneath this that's worth sitting with. Teams give code tests, type checks and alarms, none of which catch everything, but all of which give a mistake somewhere to announce itself. An unchecked help article has no equivalent alarm for a changed export cap. It can be false for a year and look exactly as healthy on its last day as on its first (it may even have picked up a nice new screenshot). Docs-as-code workflows and documentation checks can narrow that gap. In our scene, neither covers this article, so noticing falls to a human who is busy shipping the thing that made it wrong.

A person reading the original article might get a warning that the assistant's answer leaves out. Documentation sometimes leaks its age. The screenshot shows a button that has since moved. The page mentions a plan you retired. The footer says 2023. A reader picks up on all of that without trying, gets a faint feeling that this might be out of date, and either asks a colleague or tries the thing and sees. Those clues aren't reliable, and a human can miss them too. But they are clues. Hold onto that, because an assistant can leave them out entirely.

What fluency does to a stale fact

Ask an assistant about the export limit and watch what happens to the help article on the way out. The assistant reads the sentence, restates it in the present tense and in the customer's own words, answers her exact question, and hands it back in a friendly tone. The date is gone. The screenshot is gone. The faint feeling that this might be old is gone. What's left is a fact with every sign of its age sanded off, delivered by a voice that sounds identical whether it's right or wrong. Fluency launders staleness, and that's the machinery inside the confident stale answer: the tone is constant, and the correctness underneath it isn't.

Compare that with the thing everyone is afraid of, the assistant that says “I don't know.” An honest I-don't-know is a visible error. The customer sees it, knows exactly where the gap is, and does something sensible: asks a person, opens the settings page, tries a bigger export and watches what happens. Finding out may take a tab, a support ticket or an afternoon; the gap is inconvenient, but at least she knows it's there. A confident stale answer is an invisible error. She can't see it, so she can't route around it. She slices the quarter, tells her finance team the limit is 10,000, and the finance team builds a process around the slices. The real cost lands later, in every decision made on top of the answer.

The useful comparison is a stale sourced answer against an assistant that explicitly says it lacks reliable evidence. Missing docs alone do not produce that honesty. A model can confidently invent the export cap with no article at all. You need an abstention policy, evidence requirements and tests of whether it actually follows them. Given that behavior, an admitted gap gives the customer a chance to check. A stale answer presented as settled fact can take that chance away. The problem is hidden uncertainty, not the mere existence of documentation.

It gets quieter still. A customer who believes the answer has no reason to file a ticket saying it was wrong. If your dashboard mainly counts answers delivered and conversations closed, this failure can look like success. Deflection can rise while the mistake goes unreported. You are measuring how many answers got out the door, and the door doesn't check what's in the box.

When the assistant starts doing things

So far the damage is one wrong belief in one person's head, which is bad but containable. Now suppose this assistant also has an export tool, and the customer stops asking about the export and starts asking for it. “Can you pull the whole quarter for me?” Suppose the quarter contains 50,000 rows and the export tool accepts a row limit. The assistant treats the old article as a rule, passes 10,000 to the tool and doesn't check whether rows remain. Then it tells her the export is complete.

Complete. The file opens. It has headers, tidy rows and a plausible total, and it is missing every row past the 10,000th without saying so. The result is misleading without anyone having asked for a misleading result. The tool accepted a valid request, and the assistant followed a rule that used to be true. Neither fact establishes that the customer's task is complete. Checking row counts and completeness would help catch this particular failure; keeping the source current addresses the bad premise that started it.

Two useful controls don't settle this problem on their own. The first is the human yes (a subject with a post of its own). Suppose the assistant asks before sending, “Export 10,000 rows?” and the customer clicks Yes. A yes tells you the person wants the action and says nothing about whether the reason for it is true. If she trusts the stated cap, she can approve a request whose premise is wrong. A detailed preview or a completeness check could expose that mistake; the bare yes does not. The second is permissions. Allow, Ask and Deny settings fence what an assistant can touch, and they're silent on whether the story the assistant tells itself about your product is correct. A permitted action built on a false rule is still a permitted action.

Which brings us to the sentence the rest of this post is built on. A doc used to choose an action is doing some of the work of configuration. In our unchecked workflow, that configuration is prose without a test of the export claim. You would never ship a config file like that. You'd put it in the repo, review it, and fail the build when it disagreed with the code. And yet the thing that tells your assistant what the export limit is, what the refund rules are and which steps a workflow takes tends to live in a help center, edited by whoever remembers.

Zoom out and it's the same story at industry scale. A team that adds an assistant over an unchecked help center can end up maintaining two competing accounts of its product: what the implementation does and what the explanation says. The customer may be the first person to discover they disagree. The more the assistant can do, the more the product's behavior depends on whether that prose is still right.

Three nahhhs

Time for the voice in your head. It has some good points, and we'll give it every one of them.

Nahhh #1: “Just re-index the repo.” Concede first: a fresh index is real progress. It can make the new 50,000 value available as evidence. The assistant still has to find it, interpret it correctly and establish that it applies to this customer's running product. Indexing alone doesn't establish those things, or tell you whether a separately published help article still says 10,000. To check that article, something has to connect its claim to the code that supports it and test the connection. You can build that beside an index. It just isn't what refreshing the index, by itself, does.

Nahhh #2: “Make docs part of the definition of done.” This is the best process fix on the list and you should do it. It works for as long as the engineer knows the doc exists, and that's the catch. The person who raised the cap had never seen the article. The article states the number in three places, two of them inside screenshots. The change was one line inside a larger refactor, and the one person who knew about the doc left in the spring. A “docs updated” checkbox is the easiest box in software to tick sincerely and wrongly, because ticking it correctly requires knowing which doc is affected, and not knowing is the whole problem. Keep the process. Expect it to catch the drift people notice, and expect the rest to arrive on a Tuesday.

Nahhh #3: “Skip the docs. Ground the assistant on the raw code.” Concede the strongest version: the relevant deployed code and configuration can give you better evidence of an export cap than an old article. A repository snapshot still needs the right branch, version and customer context, and a model can misread it. But code holds behavior and very little else. It knows refunds over a threshold route to a manager. It doesn't know whether that's a deliberate policy or a patch from an incident two winters ago, or what you'd like the customer to be told. It says the export limit is 50,000 and can't say whether that's a promise you're making or a setting you might change next month. (If the config carries a different number per plan, it won't tell the customer which plan she's on, either.) Policy, intent and wording live in prose, which is exactly why you wrote docs in the first place. Throwing them away trades one problem for another: implementation evidence without the full explanation of what the product is meant to promise. It doesn't buy an accuracy guarantee.

A check tied to the code

Put the three concessions together and the shape of the answer shows up. Keep the docs, because they carry what code can't: intent, policy, and the right words for a customer. Keep the code as the judge of anything it can judge. Then tie them together, so that a sentence in a doc that makes a checkable claim carries a check that names its evidence. “Exports are capped at 10,000 rows” becomes a claim with an address: this sentence, this value, this file.

For that design to help, a check needs to exist, pass an initial run and stay connected to the relevant production branch. Later pushes can then trigger another run instead of waiting for someone to remember. In our example, a check quotes the export sentence, reads the relevant value, finds 50,000 instead of 10,000 and raises a hand. That's a narrower achievement than proving the whole article correct, but it's the one this mistake needed.

What happens after the hand goes up matters as much as the hand. The tempting design has a model rewrite the doc on the spot, and it's the wrong one: a model silently rewriting published answers is drift with extra steps, one more thing that sounds sure of itself. The better design proposes the fix and puts it in front of a person, with the claim, the evidence and the change side by side, and the person approves it or dismisses it. The human is there because the code may be the thing that's wrong (maybe 50,000 was a typo for 5,000), and because a person should decide when a customer-facing answer changes.

Be honest about two limits, because a check that overclaims is just a more official-looking stale doc. First, plenty of what a doc says can't be checked against code. A refund courtesy, a tone guideline, a promise about support hours: a person owns those and keeps them current, and any system worth trusting says plainly which pages it can't verify. A green badge should mean “the claims we can check passed,” and never “every sentence is true.” Second, there's an interval between flagged and approved in which the old text is still live. Somebody has to decide how short that interval is and who can see it. A design that pretends the interval doesn't exist is selling you a dashboard.

The same Tuesday, replayed

Replay the scene with an active check covering the export claim and the configured production branch receiving the change. On Tuesday the engineer changes the cap to 50,000. The production push triggers another check. It quotes “exports are capped at 10,000 rows,” looks for the value, and finds a different one. The help article is marked as needing review, a revised sentence is drafted, and the owner of that doc sees the claim, the evidence and the change side by side. She approves it Wednesday morning. (The old sentence was live for about a day, which is not nothing, and is a good deal better than three sprints.)

On Friday, the assistant retrieves the corrected article and tells the next customer the cap is 50,000. In this replay, she doesn't split anything. The check hasn't made the assistant infallible; it has removed the stale premise that sent the first customer off to slice her quarter. The customer never learns there was a config file. That is a perfectly good outcome for a config file.

This is the problem we build Teaflask around: it can draft docs from connected code, request checks for code-testable claims when a doc is released, and use active suites to detect later contradictions and propose corrections for an owner to review (the verification docs explain the limits). That checks released documentation, not every answer a customer has already received. But the question travels well beyond any product. An assistant can sound confident while its source is stale. When Tuesday's number changes, what in your system notices that the explanation no longer fits, and does anyone find out before a customer builds a quarter around it?