How to keep on top of AI slop

AI slop is code that works, shipped faster than anyone can explain it. The fix is not to write documentation faster. It is to stop writing it by hand: generate the documentation from the repository on every merge, so the docs are a function of the code rather than a memory of it.

The failure pattern is consistent. Velocity goes up, review quality holds, and the documentation quietly dies. Nobody decides to stop documenting. It just stops being possible.

Why does AI-generated code break documentation?

Hand-written documentation assumes a human wrote the code, and that the same human can describe it. Both halves break at once.

An agent produces a working module in four minutes. The engineer reviews the diff, agrees with it, and merges. They understand what it does. They did not design it, so they cannot tell you why it looks the way it does, and the architecture decision record that would have captured that was never written because there was no decision — there was an acceptance.

Multiply that by a team and a quarter. The code is fine. The map is gone.

What does AI slop actually look like in a repository?

Specific symptoms, in the order teams usually notice them:

Three utility modules that do the same thing, because three agents each wrote one
A service nobody can name the owner of
Terraform that provisions something no diagram shows
A new dependency in package.json that no human chose
An onboarding doc whose first setup step has been wrong for two months

None of these is a bug. That is what makes them survive. Tests pass, the service is up, and the only thing that broke was everybody's shared understanding.

The phrase can be misleading because it sounds like a judgment about code quality. The code may be clean, tested and idiomatic. The problem is the rate at which locally sensible changes accumulate without a corresponding system-level explanation. Ten good modules can still form a system nobody can draw.

Look for the gap between what the repository can prove and what the team can explain. Ask which service owns a piece of data, which deployment changes when a shared library moves, or where a request crosses a trust boundary. If the answer requires opening six tabs and asking the person who happened to review the last pull request, the documentation problem already exists. Calling the code bad misses the useful diagnosis.

Can you just ask an LLM to document the codebase?

You can, and it will produce something that reads well and is wrong in the places that matter.

A model handed a repository with no further context will describe what the code appears to do, confidently, including for the parts where the real behaviour depends on an environment variable set in a different system three years ago. Reading the code is the easy half. Knowing which parts of it are load-bearing is the half that needs the deployment config, the IAM policies, the git history and the incident record.

So the question is not whether to generate documentation with a model. It is what you give the model to read.

How do you keep documentation current without writing it?

Treat documentation as build output, not as a document.

Pick the artifacts you actually need. For most teams: a service map, a data-flow diagram, a change-impact view, and an ownership table. Four things, not a wiki.
Generate them from the repository, not from a meeting. Source of truth is the code, the infrastructure definitions and the CI config.
Regenerate on merge. If it is a nightly job, it is wrong by lunchtime. This is what event-driven documentation means in practice: the change event triggers the rewrite.
Diff the docs in the pull request. The valuable signal is not the new diagram. It is the line that says this change moved a trust boundary.
Let it fail loudly. A generated doc that cannot determine an owner should say so, not guess. The gaps are the useful part.

Done well, this turns architecture documentation from a separate writing project into part of the delivery pipeline.

The output should also stay small. A generated wiki with four hundred pages is easier to produce than a useful map and almost as hard to trust as the old wiki. Start with artifacts that answer operational questions: what exists, what talks to what, who owns it, what data crosses each boundary, and what changed in this merge. If a page does not help with an incident, a review or an onboarding task, it probably does not belong in the first version.

Provenance is the other requirement. Every generated claim should point back to evidence: a Terraform resource, a CODEOWNERS entry, a dependency declaration, a workflow file or a specific commit. That lets a reviewer challenge a line instead of debating a model. It also makes uncertainty explicit. "Owner unknown" is actionable. A confident invented owner is not.

Finally, keep the generated change beside the code change. A pull request that adds a queue should show the corresponding architecture diff while the author and reviewer still have the context. Publishing the result later throws away the cheapest review moment in the whole process.

What about the security review?

This is where most tools in this category die, and it is worth being blunt about it.

Giving a third-party service a read token on your monorepo is a conversation with your security team that you will lose, and you should lose it. The code is the asset.

The way through is for the analysis to run inside your own boundary: deploy a stack into an account you control, read the code in place, and clone nothing out. Zipper Blueprint for Engineering is being designed around that constraint because it is the first question a security team will ask.

That deployment model is not a complete security answer. The IAM policy, model invocation, logs, intermediate artifacts, update path and deletion behavior still need review. "It runs in your account" narrows the risk; it does not erase it. Any vendor that stops at the slogan has answered only the easiest question.

Frequently asked

Is AI slop the same as technical debt? No. Technical debt is a shortcut somebody chose and could explain. AI slop is code with no author to ask — it may be perfectly clean and still be undocumentable by hand, because the reasoning behind it never existed in a human head.

How often should generated documentation be regenerated? On every merge to the default branch. Scheduled regeneration reintroduces the gap you were trying to close; a day-old architecture diagram is a diagram you cannot trust in an incident.

Does this replace architecture decision records? No, and nothing does. Generated documentation captures what the system is. An ADR captures why somebody chose it. Generating the first frees up the time to write the second for the decisions that genuinely were decisions.