My Audit Log Was Grading Its Own Homework

Every class has one kid who marks his own homework and then looks amazed at the score. Full marks again. What are the odds.

On a Saturday in September, I built that kid in software.

I spent that weekend building a pricing engine for small insurance products in Belgium. The rules for insurance brokers say a firm must be able to show, years later, what it advised a customer and why. So at ten to five in the afternoon I set up an audit ledger, a table that recorded every quote, with database triggers that refused any edit or deletion. That evening, around half past ten, I wrote up how my project compared with a vendor already in that market, working from the vendor's own website and the way it describes its audit trail. It asked me to trust it. Everyone in that market asks you to trust them. Twenty-one minutes later, at 22:54, I committed a change. The commit message from that night puts it more plainly than I would now: "Nobody lets you check it without trusting them."

The flaw sat in my own table, not theirs. My triggers stopped edits, but they proved nothing to anyone outside, because I owned the database. I could drop a trigger, change a row, put the trigger back, and the table would look exactly as it always had. My audit log was grading its own homework.

Within three weeks the log had swallowed the project, and the pricing engine shrank into a worked example. One disclosure before going further: I still build in this space, so weigh my enthusiasm accordingly. Nothing below depends on anything I've built. Every piece is a standard, free building block you can put together yourself.

That Saturday the question was about insurance quotes. The same question now sits under every AI system that decides anything about a person. If an AI system you run makes a decision this week, and somebody challenges it a year from now, what exactly would you show them?

Append-only is a promise, not proof

Most teams answer that question with four words. Our logs are append-only.

Think about who can write to the database underneath. An administrator. A support engineer at two in the morning. A migration script somebody wrote in a hurry. Whoever restores last month's backup. Any of them can change a row, delete one, or slip a new one into the past, and afterwards the table looks as if it had always looked that way.

Logging the changes doesn't save you. That second log is just another table, in the same drawer, written with the same pen. It's a diary about your diary.

Backups don't save you either. A backup keeps a perfect copy of whatever the table said on the night it ran, including any edit somebody made the day before.

So when a regulator, a court, or the person your model refused looks at your log, they see your account of events. Not evidence of them. From the outside they have no way to tell the difference, because nothing in the table can show that nobody touched it.

An honest log looks exactly like an edited one.

That sentence stings most if your log is honest, and mine was. Being honest didn't help. The person asking can't see my intentions. They can only see my table.

What the AI Act asks for, and what it doesn't

For high-risk AI systems, the EU AI Act gets specific about logs. Article 12 says the system must technically allow the automatic recording of events over its whole lifetime. Article 19 asks the provider to keep those logs, and Article 26(6) asks the firm using the system to keep them, for at least six months in both cases. After this summer's Digital Omnibus, those duties apply from 2 December 2027 for most stand-alone high-risk systems. So there is time to build this properly, which is the only nice thing anyone has ever said about a deadline.

Read those articles closely and notice what they leave out. Nothing says the log has to resist editing. It has to exist, and somebody has to keep it.

Except one sentence in Article 26(6) changes the picture, and it's easy to skip. Deployers that are financial institutions must keep these AI logs as part of the documentation they already keep under financial services law. For an investment firm, that documentation falls under MiFID II, and the detailed rulebook behind it, Regulation 2017/565, says in Article 72 that any correction must show what the record said before, and that nobody may manipulate or alter the records in any other way.

Put those two sentences side by side. An AI log at an investment firm can land inside rules that already ask for proof that nobody quietly changed it.

The GDPR adds the human side. Under Article 22, a person facing a decision made solely by automated means can ask for a human to step in and can contest the decision. You can only take that seriously if you can say what the system decided, which version decided it, and on what.

The law asks you to keep a log. The person contesting your decision asks whether to believe it.

An agent's log is the agent's testimony

AI agents add a twist that ordinary logging advice doesn't cover.

An agent doesn't make one decision and stop. It takes steps. It reads a file, calls a tool, sends a message, and then writes down what it did. That last part causes the trouble. When an agent records "I sent the email to the customer," the record is a statement by the agent. It doesn't prove the email went out, or went to the right person, or went only once.

I learned this from my own agents. I use scheduled Claude routines to send outreach emails for my side project. A nightly run drafts and queues the emails, and a weekday run sends them at 09:07. On 5 October I sent eight emails by hand that morning. Six minutes later the 09:07 run sent two of them again, politely and with complete confidence. It had checked the inbox for replies and found none. It never looked in Sent Items. From where it stood, it had done everything right, and its own account of the morning would have said exactly that.

The fix wasn't a better log. It was a second source of truth. Every send now checks Sent Items first, and the agent can't skip that check because it thinks it already knows.

So if you log what agents do, keep two kinds of entry apart. What the agent says it did, marked plainly as the agent's own statement. And what some other system confirms actually happened. Sealing the first kind still has value. It proves the agent said it, at that time, and that nobody changed it afterwards. It still doesn't prove the email went out.

Sealing a false record just gives the lie a very reliable date.

Turning a log into evidence

The method has four moves, and I invented none of them, which is honestly the best thing I can say about it. Most of them come from Certificate Transparency, the public system that catches fake web certificates.

Start with what goes into each record, because the method can only protect what you put in. For an AI decision or an agent run, record a fingerprint of the inputs rather than the inputs themselves, the model and its version, the outcome, and any human review, with one record per decision or per run. If you leave out the model version, you will meet the question everyone eventually asks, which version decided this, with nothing to show.

The first move sends a fingerprint, not the record. Your software writes the record in a fixed text form, so the same record always produces the same text. It adds a random value called a salt, and runs the result through SHA-256. Out comes a code 64 characters long. Change one comma in the record and the code changes completely. Only that code leaves your building. The record and the salt stay with you.

The salt matters more than it looks. Plenty of AI decisions come down to one word, approved or declined. Without a salt, anyone holding the fingerprint could hash every likely answer and find the match, which for approved or declined takes about as long as a coin toss. With the salt, they can't.

Keeping the record at home also settles a fight between two rules. A permanent log and the GDPR right to erasure can't share a table. If personal data goes into a log that nobody can edit, nobody can erase it either. So nothing about a person enters the permanent part. The data stays in your systems, where you can still delete it.

In Python, the first move fits in a few lines. This sketch shows the idea. A real system should use a proper canonical JSON library that follows RFC 8785, not this shortcut.

import hashlib, json, secrets
record = {"run_id": "r-1042", "model": "credit-scorer-v7",
          "outcome": "declined", "human_review": False}
salt = secrets.token_hex(16)  # store this next to the record, never send it
canonical = json.dumps(record, sort_keys=True, separators=(",", ":"))
fingerprint = hashlib.sha256((salt + canonical).encode()).hexdigest()

import hashlib, json, secrets
record = {"run_id": "r-1042", "model": "credit-scorer-v7",
          "outcome": "declined", "human_review": False}
salt = secrets.token_hex(16)  # store this next to the record, never send it
canonical = json.dumps(record, sort_keys=True, separators=(",", ":"))
fingerprint = hashlib.sha256((salt + canonical).encode()).hexdigest()

The second move builds a tree, not a chain. My first version chained each entry to the one before it, so changing any old entry broke every link after it. That works, and it makes a satisfying demo. Change one number in a four-entry chain and you can watch everything after it fall over.

But a chain carries a cost. To prove one entry, you replay the history up to it. A Merkle tree, the structure Certificate Transparency uses (RFC 6962), proves one entry with a few dozen hashes even in a log of millions. It can also prove that the log only grew between two dates and that nobody rewrote it. So my old chain stays frozen, and every new fingerprint goes into a tree.

The third move matters most. Sign the tree's current state, its size and its root, and publish it. The state leaves the building. Once somebody outside saves a copy, the operator can't rewrite anything before it without that copy catching them. Keep the signing key somewhere nobody can copy it out, yourself included. In my own build the key exists only inside the running service, and I hold no copy. Publish the public half, and keep old public halves published after you rotate, so old seals stay checkable.

The fourth move adds outside clocks. RFC 3161 time stamp authorities will stamp a state for free. I take two stamps every hour, one from DigiCert and one from Sectigo. One early state of my log also sits in Bitcoin through OpenTimestamps, so it existed before block 967,830 went into the chain, whatever anyone says later, me included.

Put together, an auditor gets a small bundle: the record, its salt, the proof that its fingerprint sits in the tree, and the signed state. They run a verifier offline. Write that verifier so it shares no code with the system it checks. Mine is open source, in JavaScript and Python, with no dependencies. It recomputes the fingerprint, checks the tree, checks the signature against the published key, checks the time stamps, and, with an older state the auditor saved, checks that the log only grew in between.

They don't have to trust the firm, because a changed record produces a different fingerprint. They don't have to trust whoever runs the log, because that party never held the record. Two parties who might each want to bend the story, and neither of them can.

The rule I took from all of this fits on a sticky note.

"Never let the accused keep the only copy of the notes."

What broke while I built it

The first break was embarrassing. My database handed back timestamps in a slightly different text form from the one it had hashed, "+00:00" at the end and fewer decimal places. Same moment, different outfit. Any verifier that didn't correct for it would call a perfectly intact record broken. A trust project whose own checker calls honest data a forgery would have had a very short life.

The fix became a rule. Nothing in a seal may depend on formatting, key order or locale. I wrote it down more bluntly afterwards. A chain only one database can rebuild is a claim, not evidence.

The second break came from scale. I built proofs by reading every entry in the tree, and that hit a ceiling fast. Now the tree stores its in-between hashes as entries arrive.

The third was worse. For a while, my own instructions for the independent check told readers to pull rows out of the database with a key only I would ever hold. So only I could run the independent check. Independent in the sense that I was independent. The corrected instructions now carry one blunt line. A proof that needs the operator's cooperation is not a proof, it is a courtesy.

For anyone wondering whether this is practical, the numbers. When I load-tested my own build on a GitHub machine, not in production, sealing one entry took 9 ms at the median and 11 ms at the 95th percentile, and a batch of 50 took 22 ms. Each row in my production log takes about 400 bytes, and the record itself takes zero, because the log never holds it. Infrastructure comes to roughly $0.00003 per record at a million records a month, before storage. Cost won't be your excuse.

Where sealing makes things worse

Done badly, all of this makes things worse, and that deserves its own section.

A seal can make a bad record look official. If your agent's log says it checked a customer's ID and it didn't, sealing that line gives you a beautifully preserved false statement with two outside time stamps on it. People will trust it more because it looks so careful. The seal moves the question from "did anyone change this?" to "was this ever true?", and teams that stop at the first question tend to believe they've answered both.

And in some cases the honest answer is not to bother. If your real risk is records that nobody ever wrote, sealing does nothing, because a record nobody sends leaves no trace. You can number your records in a chain so that a gap shows, but even that can't prove a record ever entered the chain. If your real risk is that the decision itself was wrong, fix the decision. A tamper-evident log of a bad model is just a very well-kept diary of a bad model.

What a seal cannot prove

Being plain about the limits is the only reason anyone should believe the rest.

A seal proves that this exact record existed by a certain time and hasn't changed since. It doesn't prove the record was true or complete. It doesn't prove when anyone created the record, only that it existed no later than the time stamp, so a fabricated record can carry a seal, and the seal just fixes the fabrication's date. It doesn't prove who wrote the record, unless the firm signs with its own key, and even then it proves only that the key holder made the claim. If you lose the record or the salt, it proves nothing, and nobody else can rebuild it for you. Free time stamps aren't qualified under eIDAS, so they don't carry the legal presumption of accurate time that qualified ones do. None of this makes anyone compliant or certified. It's evidence, not approval.

One weakness belongs to the design itself. Whoever runs the log could show one version of the tree to you and a different version to the regulator, and each of you would see a perfectly consistent history. Certificate Transparency answers this with witnesses, independent parties who cosign each new state so everyone knows they're looking at the same one. My own log has no witness yet. Until it does, the protection rests on people saving copies and comparing them later. A log that shows a different face to each visitor is just lying with very good maths.

The other side of the door

If you build AI products, you can read all of this as advice for your customers. Turn it around first. On the night I started, I read a vendor asking for trust and wrote that nobody lets you check. If you sell an AI platform that holds your customers' decision logs, you are that vendor. Their logs live in a system you run, and they can't show anyone that you, or they, didn't change a line after the complaint arrived.

Your premise is probably that your logs are honest. Mine were. Run one test before you decide that settles it. Name everyone in your company who could change an entry from last March without anyone noticing. If you can name a single person, yourself included, your log is your account of events, not evidence.

If you buy AI rather than build it, ask your vendor one question, in these words. "If someone challenges one of our decisions next year, can someone outside both our companies check that this log is the one from the day, without asking you?" Any answer that starts with "trust us" is a no.

And if your models have made decisions for two years on ordinary logs, the hard answer is that you can't fix the past. Sealing an old log today proves only that it existed no later than today. It says nothing about the months before. The date of your first seal is the date your evidence starts, and every week you wait moves that date later.

My audit log doesn't grade its own homework anymore. It hands in a copy, and strangers keep it.

Most days, nobody reads that copy.

It only has to be there on the day someone asks.