Coding agents keep making the same mistakes in the same repo, session after session. That is a harness engineering problem. The model forgets everything when the session ends, and the repo's harness, the AGENTS.md, rules, skills, scripts and checks that guide the agent, does not learn on its own. The fix is to make the harness self-improving: right after a session, let an agent read what happened, find what could go better next time, and propose where that lesson belongs. You approve it, and the next agent starts from a better repo.
In my guide on how to set up your repository for AI coding agents, I explain how to build a harness: the context the agent reads before it codes, and the checks that catch its mistakes after. This post answers the next question: once the harness exists, how do you keep improving it over time?
Try it now
If you want your agent to propose improvements to your harness, call the Harness Retro skill right after a conversation ends. Install it with the skills CLI:
npx skills add alfonsograziano/skills --skill harness-retro
Then, at the end of a session, ask for a "harness retro". That's it. The skill is open source, it is two markdown files, and it works with any agent that supports skills, like Claude Code and Codex. You can read it on GitHub.
The rest of this post explains why it exists and how it decides what to propose.
Agents forget, and the harness does not learn by itself
An agent has no memory between sessions. Whatever it learns during a run ends when the run ends. The only way a lesson survives is if someone writes it into the repo, and today that someone is a human who noticed the pain and had a free afternoon. Most of the time, nobody does.
Here is what that looks like on a real project. The database lives in a private network. To run one query, the agent has to start a bastion host, connect through it, run the query, and stop the bastion host at the end. The first agent figures this out after a few failed tries. The second agent, one day later, does the same thing from zero. So does the tenth. Sometimes one of them forgets the last step, and the bastion host stays up all night.
A human would have written a script after the second time, because a human remembers the pain. Birgitta Böckeler makes the same point in Harness engineering for coding agent users: when an issue comes back, you improve the harness so it stops coming back. The hard part is noticing, because the evidence is buried in transcripts nobody reads.
How the skill was born
The bastion host was the moment I got tired of it. I wrote down one question, and the whole skill grew from it: what should the repo have so that the next agent does this faster, with less risk, or without asking a human?
The answer was already sitting in the agent's conversation. Every failed command, every retry and every "no, we use pnpm" from the user is in there. Reading agent transcripts is nobody's idea of fun, but an agent reads them fine.
Finding the problems turned out to be the easy part. Agents are good at spotting that something went wrong, and bad at deciding where the fix should live. Ask a plain agent and it will suggest "add a note about the bastion to AGENTS.md". That note then costs tokens in every future session, for every task, and the agent can still forget to stop the host.
So I took what I wrote in my book, AI-Native Software Engineering, about how a harness is built, cut it down to a one-page primer, and put it inside the skill. Now every finding comes with a layer and a reason for that layer. I also kept the skill humble: it only proposes, and you decide what gets applied.
What it does
Figure 1 shows the loop.
Figure 1. The self-improving harness loop.
The skill reads the conversation you just had, plus past sessions in the same repo when they exist, because a problem seen three times is a much stronger signal than one seen once. It looks for failed commands, steps done by hand more than once, guessed conventions, your corrections, and anything risky. It checks what your harness already has, so it does not propose a script that already exists. Then it returns a short list of findings, each with a severity and the exact change. Zero findings is a valid answer: a smooth session should produce nothing.
Where the fix goes matters more than the fix
The same lesson can live in many places, and the wrong place makes it useless. A line in AGENTS.md that says "always run the linter" gets skipped some of the time. A pre-commit hook that runs the linter never does. Two rules decide most cases:
- Enforce before you instruct. If a test, a hook or a permission can catch it, use that. Agents skip instructions sometimes, but they cannot skip a failing check.
- Load it only when needed. Every line in
AGENTS.mdis paid for in every session. Prefer a scoped rule, a linked doc, a skill or a script, unless nearly every task needs it.
For the bastion host, the skill proposes a script, scripts/db-query.sh, that starts the bastion, runs the query and always stops the bastion at the end, even on error. Then it adds a rule scoped to the database folder that points to the script. The steps never change, so they belong in code. Only database work needs to know about it, so the pointer stays out of AGENTS.md. Forgetting the last step becomes impossible, and only the agents that touch the database pay for knowing about it.
The harness is the part that compounds
A bug fix helps once. A harness fix helps every future session, for every agent and every person on the team. That is the core idea of harness engineering, and it only works if the lessons actually reach the repo. A short retro right after each conversation is the cheapest way I have found to make that happen.
If you like the idea of agents keeping what they learn, I wrote about a related one in procedural memory for AI agents. The full reasoning behind the harness layers and the gates is in my book, AI-Native Software Engineering, published by O'Reilly.