The Accidental Record

I — Nobody Ordered This

When a company puts artificial intelligence to work on something complicated, it rarely uses a single system. It uses several, in sequence. One drafts a plan. Another checks it. A third carries out the task — files the report, sends the message, moves the money.

These systems have to coordinate, and the way they do it today is by writing each other short notes. In English. Here is the plan. Here are the two things I could not verify. Proceed with the rest. Someone with access can open the file, read the exchange from beginning to end, and understand what happened and roughly why.

That is an unusual property for a powerful and poorly understood technology to have. Almost nobody has noticed it, and it is not going to last.

No regulator required the notes. No parliament debated them. No company adopted them as a transparency measure, and none advertises them as one.

They exist because writing things down is the only method that currently works. When two of these systems need to pass work between them, there is no direct channel — no wire between the machines carrying thought. Each one has to convert whatever it is doing internally into ordinary words, hand the words over, and let the other one read them and start again from scratch. (Beyond Tokens, 2026)

We got a readable record of automated decision-making as a by-product. It is genuinely useful and completely accidental.

II — Why It Is Readable

The explanation is short, and it is worth holding onto, because everything else follows from it.

These systems work by converting their internal state into words. Words are a bottleneck. They are slow to produce, slow to read, and a great deal is lost on the way — the difference between what you were thinking and the sentence you managed to write. Researchers put the gap at roughly fifteen bits per token against some forty thousand bits per hidden state. (Interlat, 2026)

That loss is the transparency. There is no separate logging system quietly recording their reasoning for our benefit. The log is the bottleneck.

The record exists precisely because the machines are being forced through a narrow opening, and the narrow opening is human language.

III — Why It Ends

Engineers are now working on removing the bottleneck.

The idea is simple enough to state: let the systems skip the words. Instead of converting internal state into English and handing over the English, hand over the internal state directly — a block of numbers, meaningless to a person, that the receiving system can use immediately without the slow business of reading.

It works. In published experiments it runs around four times faster end to end than the version that stops to write things down, uses seventy to eighty per cent fewer output tokens, and scores as well or better on accuracy. (LatentMAS, 2025) It is not yet used in products. It is very clearly where the field is heading, because speed and cost are what the field optimises for.

The choice is not quite as stark as words-or-numbers, and the piece should not pretend otherwise. Intermediate architectures exist: a system can run the fast path in production while writing a slower text channel alongside it for audit, and interpretability work is developing tools that map internal states back to approximate descriptions after the fact. Neither is free. Both cost exactly what the fast path was adopted to save. So the realistic future may not be no transparency but expensive transparency — which is a meaningful distinction, and also, on the evidence of how institutions treat costs that produce no revenue, a distinction with a predictable ending.

There is a worse possibility hiding inside the better-sounding one. A parallel text channel is only worth anything if the words are the thing the receiving system actually acted on. If the fast path does the work and a separate summariser is asked afterwards to describe what happened, the resulting document is not a record. It is a guess about a process the guesser did not participate in — and a confident, fluent, plausible guess at that. That is not weaker oversight than a readable handover. It is worse than nothing, because it produces a document that reads like evidence and functions like fiction.

Notice what is not happening here. Nobody is proposing to conceal anything. Nobody is arguing that oversight is bad, or lobbying to have the records removed. The engineers working on this are solving a performance problem, and they are right that it is a performance problem.

The record does not disappear because someone decides to remove it. It disappears because it was never the point.

IV — Who Was Relying On It Without Knowing

A surprising number of institutions have quietly built on the assumption that this record will be there and will be readable.

European rules already require financial firms to document technology failures, report them on tight deadlines, and preserve the evidence of what went wrong. (Regulation (EU) 2022/2554) Courts reconstruct events after the fact from whatever the system wrote down. Auditors trace decisions backwards. Investigators, journalists and anyone else trying to establish what a company's automated systems actually did all start from the same place: the trail those systems left behind.

None of these rules say the trail has to be in a form a human can read. They did not need to. It always has been.

Rulemaking is also not the fastest mechanism here, and it is worth saying which one is. The first serious lawsuit in which an automated system causes a large, well-documented loss, and the claimant's lawyers discover that no readable account of the decision exists, will move faster than any legislature. Liability insurers will price the difference between an auditable architecture and an opaque one long before a regulator defines it, because that is what underwriters do with uncertainty they cannot inspect. The commercial pressure to keep a readable handover may well arrive from the insurance market rather than from Brussels — and it will arrive after the first accident rather than before it.

V — Three Fair Objections

Won't companies keep the notes anyway? Only if something requires them to. Once the faster method exists, writing the notes becomes a cost — a deliberate decision to run slower than a competitor. A few firms will pay it. Most will not, and the ones that do will face steady internal pressure to stop.

Aren't the notes unreliable anyway? Yes, and this is the strongest objection. There is good evidence that these systems do not always write down what they actually did. The note describes a tidy line of reasoning; the real process may have been something else. So the record is not a faithful transcript, and anyone treating it as one is mistaken.

The sharper version of the objection is worse still: if the trace is unreliable enough, preserving it may be actively harmful, because it manufactures the appearance of oversight. A regulator with a document to read feels supervised. A court with a narrative to quote feels satisfied. If the document is fiction, the comfort is fiction too, and the institution has bought reassurance instead of scrutiny.

That risk is real, and it is the reason the requirement has to be stated carefully. But an unreliable record and no record are still not the same thing. A misleading account can be tested — against the outcome, against the other notes, against what the system was asked to do. Discrepancies show up. That is how most investigation works, in every field: not by trusting the account, but by finding where it fails to match. A block of numbers offers nothing to fail to match. The loss here is not accuracy, which was never guaranteed. It is the existence of anything to argue about.

Which means the right requirement is not that the record be believed. It is that the record be checkable — that a person can hold the stated reasoning against the observed outcome and see whether the two agree. A rule that mandates readable handovers without mandating that they be testable against results would indeed produce theatre. That is a drafting problem, and it is a solvable one.

Is this happening now? No. It is in the research stage. That is the reason to write about it now rather than later. This is a window, not an emergency — and windows of this kind are usually noticed once they have closed.

VI — What To Do About It

Record-keeping rules almost always specify how long. They almost never specify in what form.

"Retain records for five years" is an empty instruction if the record is a block of numbers that no person, and no court, can interpret. The requirement was written in an era when a record was self-evidently something you could read, so nobody thought to say so.

Say it now. Where automated systems are used for consequential decisions — moving money, allocating credit, running critical infrastructure — require that the trail be reconstructible in a form a human being can actually examine. Not the whole system laid open. Just the handovers: what one component told the next, in words.

And one word of specification carries most of the weight. The handover must be constitutive rather than descriptive: the text has to be the thing the next component actually received and acted upon, not a commentary produced alongside it for the benefit of the file. A descriptive requirement is satisfied by a summariser and produces the fiction described above. A constitutive one cannot be satisfied without the words doing real work in the system, which is precisely the property that makes them worth preserving.

The timing matters more than the drafting. Today this costs almost nothing, because it is what the technology already does. Once the faster method is standard, the same rule becomes a demand that European firms run several times slower than everyone else — and at that point it stops being a question about oversight and becomes a question about competitiveness, which is an argument that oversight loses. The counter-case is worth stating: a standard set early, while the cost is near zero, has a habit of travelling, and the existing incident-reporting regime already reaches non-European firms serving European clients. Acting now is a bet that a well-designed requirement spreads before the alternative hardens. Acting later is a bet on nothing at all.

— The General Shape of the Problem

We are protected, more often than we like to admit, by things that were never designed to protect us. Friction that happened to slow a bad process down. A limitation that happened to keep a record. Complexity that happened to require a human being somewhere in the middle.

These are not safeguards. They are accidents that have been doing the work of safeguards, usually for long enough that everyone has forgotten the difference. We tend to find out which is which at the moment the accident gets engineered away.

Here, unusually, we can see it coming.