If you want agents to do what you want, you need to learn how to box them into deterministic actions. Unfortunately, this is not what current coding agents are good at. They are designed to be exploratory and open ended. They take a vague prompt, produce a plausible patch, run tests, loop a few times, and so on. This is great for prototyping and greenfield work. But when you need something reliable, when you want the same command to have the same effect every time, the same refactoring to happen to twenty files, the same deploy process to execute in the same order, the vague prompt and the "please read the docs" prompt become a liability.
So you end up with dozens of md files telling your agent to keep reading. That's the wrong place to stop.
Markdown documents aren't constraints. They're recommendations. Agents can read the CONTRIBUTING.md, AGENTS.md, CLAUDE.md, rules file, style guide, runbook, whatever, summarize them, quote from them, say "I've read the docs, and I'll do my best to follow them." They add more and more surface area that needs to be parsed and understood: "Use the existing logging conventions in the repository." Which? Where? "Use small functions." How small? "Run the tests before pushing." Which tests? Which command? Which environment? The agent has to process all of this, balance conflicting directives, and exercise judgment. It's in the judgment where the agents can stray.
In contrast, deterministic actions don't need judgment. "Read the docs and follow the coding style" becomes "Run the ./scripts/lint.sh, ./scripts/format.sh, ./scripts/test.sh scripts." The agent runs them and gets a pass/fail outcome. If one fails, the agent sees the exact error and has nothing to interpret or guess at. "I thought the rule meant…" becomes ./scripts/lint.sh. "Do the refactoring" becomes "Run the ast-grep or jscodeshift or sed tool to apply the transformation." The agent doesn't need to decide which files to update. The script does that, by design. You're boxing the agent into a sequence of deterministic actions.
This is a change in design paradigm from "teach the agent what you prefer" to "translate your preferences into deterministic actions." You can't run a markdown document. You can run a script, a checklist, a Makefile target, a GitHub Action. If you have a Makefile with make setup, make lint, make test, make build targets, the agent doesn't need to understand your coding philosophy. It just needs to run make lint and respond to the results.
The point isn't to dumb down the agent. The point is to make the environment unambiguous. An agent can still be smart when it executes a set of deterministic actions. It can read the failures, make corrections, and re-run the tests. It just can't step outside of the box. It can be clever within the constraints, but the box exists as a series of commands that either pass or fail.
This is why coding agents don't do this well. They want to be independent and explore the repo, read the files, build a mental model, and then make things happen. That works until it doesn't. The more md files you have, the more the agents think they have discretion. "The docs say to use snake_case, but this one existing file is camelCase. Maybe the docs are old." The agent is trying to be helpful, but it's not being deterministic.
Most AI models write replies that are hard to follow. So I added ASD-STE100, a plain-English standard written for aircraft maintenance manuals, to my coding agent's markdown rules. One rule says "Keep sentences to 20 words or fewer." As a script, that becomes MAX_SENTENCE = 20. When the agent ran the script over 10 of its own replies from one session, it found 9 failures. The script found them only after the replies were written, because nothing forced an earlier run. So the script goes in a hook.
I saw CI do this on a fix I sent to Forem. I don't write Ruby, and I had no Ruby installed, so CI was my test machine. My coding assistant wrote the fix and the tests. The first run failed, and the issue was in the test, not the code. That result did not depend on the assistant's opinion of its own work. But CI also passed an unused line, and only the reviewer caught it. A check only boxes in what it tests.
Full story: I Don't Write Ruby. A Fix I Sent Is Now in Forem.
If you want agents to do what you want, don't stop at md files. Write scripts.
Make every "please remember to" into a command. Every "we prefer" into a check. Every "before you push" into a pre-push hook. An agent can skip a local hook, so back it with a required CI job and a merge gate the agent has no rights to change. If a check fails, the agent can make corrections and run it again. That's how you box the agent into a deterministic sequence of actions. It's not a recommendation. It's a contract. And the agent can't ship work that fails the check.
Top comments (48)
"It's not a recommendation. It's a contract" is the line that matters, and the comments have already done the hard work on root of trust — the settings file that declares the hook, the script hash, and a check that never ran printing the same line as one that found nothing. I don't want to retread that.
The gap I'd name sits one step later. A script makes a constraint executable, but a single exit code still compresses several failure classes into one boolean. Your sentence-length example is the clearest case: the check went from nine failures in ten to a pass, the replies got shorter, and you noted you hadn't looked at what went missing. Length is now enforced; whether the answer is still complete is a different question the same green cannot answer. That isn't an argument against scripts — prose can't even tell you which rule was consulted — but a gate needs to say which class it evaluated, not merely that it ran.
So I'd give the gate a small positive / boundary / follow-up set instead of one threshold. Positive: the ordinary violation. Boundary: exactly twenty words, a code block, a quoted line, an abbreviation — does the counter agree with the rule's intent? Follow-up: after the hook sends the draft back and the agent rewrites it, is the second receipt attributable to the same candidate, or just another green with no link to the first failure? Then coverage becomes recordable — script revision, input identifier, which class ran, each result — and "deterministic by construction" has a scope: the checks that exist and were evaluated on this input, not a claim about the whole task.
When your hook logs a pass, does it record which case set produced it and the revision of the script, or only the exit code?
Exactly! Deterministic + unambiguous agent control is the right way forward.
That is proper Software Engineering Automation (SWEA) as opposed to vibe coding.
Vibe only coders may hate me for saying this, but I'll say it anyway: No, English is not the next programming language, purpose built Deterministic + Unambiguous Agent Harness is. And as I said before: There's No AI Agent, Nor Any Harness 🤣🥰
Not sure about how good or bad ASD-STE100 performs though. Will check it out.
measure it if you try it. As a markdown rule alone, my agent still wrote 9 sentences over the limit in 10 replies.
The
md = recommendation, script = constraintframing is exactly right. We hit the same split in Electron preload design: a line in the architecture doc saying "renderer must not callshell.openExternalon arbitrary URLs" is just prose the renderer can ignore, but acontextBridgemethod that validates the URL against an allowlist before callingshell.openExternalis a constraint the renderer cannot bypass. The corollary we learned the hard way: put enforcement in the smallest deterministic unit (a typed function, a schema, a binary check), and use markdown only for the why — never for the what. If the agent can phrase its way around it in natural language, it was never a rule.You asked upthread who guards the settings file that holds your Stop hook. The Claude Code docs answer most of it. The file watcher normally picks up direct edits to hooks, so an edit lands in the running session. One line,
"disableAllHooks": true, turns every hook off, and a project-level value overrides a user-level one..claudeis a protected path: writes there are prompted in default and acceptEdits mode, and allowed in bypassPermissions.The closest built-in guard is
permissions.blockWritesOutsideWorkingDirectories. While it is on, Claude Code refuses Write, Edit and NotebookEdit writes to settings files in every permission mode (v2.1.314 or later). It does not limit shell commands, so a Bash edit of the file still lands unless the sandbox blocks it. A ConfigChange hook can also block a settings change from applying to the running session, but silently (debug log only), and the file stays edited for the next run. So your CI gate stays the backstop.We went one step further and took those steps out of the agent. In our coding-agent pipeline the agent can't run
git pushat all: a guard on every tool call blocks it, and a plain script commits, runs the build check and pushes once a human moves the card. The agent's only exit,endWork, is refused until its plan is finished, its stack is healthy and the merge is clean. Same lesson as your hook: the check runs before the agent can say it's done.Agree on boxing the agent into scripts. The failure mode I keep seeing is the agent "improving" the script mid-run — it edits lint.sh because the failure looked like a false positive, then green-lights itself. We treat ./scripts as root-owned and refuse any write to that directory from the agent session; if the checker is wrong, a human opens a PR. Pass/fail only works if the checker cannot rewrite the checker.
Pratik, does your write block also apply to the configuration that calls the scripts? I have my hook in a settings file, and changing that file could disable the check.
The settings file is the actual root of trust, so it has to sit in the same class as the script. If the agent can rewrite the file that declares the hook, it does not need to touch the script at all -- it just removes the invocation and the check goes quiet, which looks exactly like a passing check.
Where ours landed: the session config is readable but not writable while the agent is running. Any change to it is a file write outside the project subtree, and that is rejected by the tool layer before review -- not because a reviewer would object, but because the path is not in the writable root at all. So the hook declaration, the script path and the pinned hash all live outside what the agent can write, and the run fails loudly rather than silently when one of them stops matching.
The line that stuck with me is that markdown is a recommendation and a script is a constraint. I hit the same wall with the agent saying "fixed" when nothing had run since the last edit. A CLAUDE.md rule never stopped it, because by then the model already believes it's done. What worked was a Stop hook that checks the session when the turn ends and sends it back once if nothing ran after the last code edit. It's a small repo if anyone wants to see how it works: github.com/Balance0014/claude-code... . The surprise for me was how many false blocks my first version had, so I'd say replay your own history before you trust any check like this.
The markdown-versus-script trap is real: prose instructions invite negotiation.
When an invariant lives as a markdown paragraph ("keep functions under 30 lines and preserve existing error formatting"), an agent with a crowded context window treats it as a soft recommendation. If a test fails later, it starts explaining why its violation was justified by the surrounding code.
The clean boundary is keeping markdown strictly for high-level intent and delegating every mechanical rule to a script that exits non-zero. Replacing a prose rule about import hygiene with
ruff check --select Ior a custom ast-grep rule leaves no room for interpretation. When a command fails with an exit code and a diff, the loop has an exact receipt to fix instead of a preference to balance. If an invariant cannot fail a command, an agent will eventually negotiate around it.Scripts are often a better operational contract than prose because they can validate inputs and produce evidence as part of the run.
The useful split is to keep the why and the safety boundary in a short instruction file, while putting repeatable state changes and checks in versioned scripts with explicit exit conditions.
Know what happens to the scripts when the thing they check changes, because a markdown rule can be argued with and a script just fails. A lint cap or a sentence-length check is a number someone fixed on a particular day, and once it's a required gate that number keeps its authority long after the reason for it has gone. Who owns the script, then, and how does anyone go about changing it once the rule turns out to be the wrong one? Otherwise the box gets tighter and the code gets worse and the gate stays green.
Adam, I haven't tested but the ones I have tested all passed but are 15% to 41% shorter. I haven't looked at what is missing from them.