Never Let Claude Code Tell You It's Done

What you will do: add one test that catches the agent’s mistakes, then wire it so Claude Code runs it on its own and cannot end a turn while it is failing. About twenty minutes. Who this is for: you use Claude Code on real code, and it has told you “fixed it, tests pass, done” when it was not. Who should skip it: if your agent already cannot end on a failing test, you are past this. You need: Claude Code (version 2.1.143 or later), Python 3 for the worked example (it uses the built-in test runner, nothing to install), and a project of your own.

Contents

1. Write a test that says what you want
2. Put it behind a gate the agent cannot talk past
3. Make the gate run itself

Claude Code will tell you it fixed the bug. It will tell you the tests pass. It will tell you it is done. And sometimes it is wrong, and it says all of it with the same calm certainty it uses when it is right. “Done” is the most expensive thing it gets wrong, because the moment you believe it, you stop looking.

The reason is plain. The model is rewarded for sounding finished, and sounding finished is not the same as being finished. The gap between the two is invisible at exactly the moment you have decided to trust it. You cannot close that gap by asking the agent to be more careful. You close it with a check the agent runs but does not get to grade: its own tests, with a real pass or fail.

1. Write a test that says what you want

A test is a small program that runs your code and checks it does the right thing. The important word is yours. The test has to encode what you want the code to do, because if the agent writes both the code and the test, it can quietly make the two agree. The check has to come from outside the agent, or it is not a check.

Here is the smallest possible example: a function, and a test for it. The agent was asked to fix add, and reported back that it was done.

calc.py

def add(a, b):
    return a - b

test_calc.py

import unittest

from calc import add


class TestAdd(unittest.TestCase):
    def test_basic(self):
        self.assertEqual(add(2, 3), 5)

    def test_zero(self):
        self.assertEqual(add(0, 0), 0)


if __name__ == "__main__":
    unittest.main()

Save both files in the same empty folder and open your terminal there (cd into it). They have to sit together, because the test does from calc import add. Now run the tests. (python3 -m unittest finds and runs every test_*.py file in the folder. It is built into Python, so there is nothing to install.)

$ python3 -m unittest
F.
======================================================================
FAIL: test_basic (test_calc.TestAdd)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_calc.py", line 8, in test_basic
    self.assertEqual(add(2, 3), 5)
AssertionError: -1 != 5

----------------------------------------------------------------------
Ran 2 tests in 0.000s

FAILED (failures=1)

The agent was certain. The test does not care how certain it was. add(2, 3) came back -1, not 5, and now you know, in one second, that “done” was not true. That is the whole idea: a fact about the code the agent cannot argue with.

When you write a test for your own code, pick an input where a broken version and a correct one give clearly different answers. If your test passes on code you already know is wrong, the inputs are not separating right from wrong yet, and the gate will wave the bug straight through.

2. Put it behind a gate the agent cannot talk past

A test you remember to run is already worth a lot. But you will not always remember, and the agent can finish and hand control back to you before you check. So wrap the test in a gate: a small script that runs it and turns the result into a hard yes or no.

Save this as check.sh in your project:

#!/bin/bash
# A Stop-hook gate. Claude Code runs this when the agent tries to end its turn.
# If your tests pass it exits 0 and the agent is free to stop. If any fail it
# prints them and exits 2, which blocks the stop: the agent is handed the output
# and has to keep working, so it cannot end the turn while the suite is red.
cd "${CLAUDE_PROJECT_DIR:-.}" || exit 2   # run from the project root, not wherever the agent cd'd to
# PYTHONPYCACHEPREFIX gives Python a fresh bytecode-cache dir each run, so the gate
# can never pass on bytecode compiled from older code that was rewritten at the same
# size and timestamp (a false green this gate exists to prevent). The trap removes it.
tmpdir="$(mktemp -d)"
trap 'rm -rf "$tmpdir"' EXIT
output=$(PYTHONPYCACHEPREFIX="$tmpdir" python3 -m unittest 2>&1)
if [ $? -eq 0 ]; then
    exit 0
fi
echo "The tests are not passing. Do not stop. Fix these and try again:" >&2
echo "$output" >&2
exit 2

Not on Python? Replace the python3 -m unittest line with whatever runs your suite (npm test, go test ./..., pytest), anything that exits nonzero when tests fail. That is all the gate needs. The tmpdir, trap, and PYTHONPYCACHEPREFIX lines are a Python-only wrinkle you can drop; keep the cd so the runner still fires from your project root.

Run it on the broken code and it answers plainly. (Trimmed here to the lines that matter; you will see the same full traceback as Step 1.)

$ bash check.sh
The tests are not passing. Do not stop. Fix these and try again:
F.
FAIL: test_basic (test_calc.TestAdd)
AssertionError: -1 != 5
FAILED (failures=1)
$ echo $?
2

That 2 is the load-bearing part. An ordinary failure exits 1. We exit 2 on purpose, because of what Claude Code does with it next.

3. Make the gate run itself

Claude Code has hooks: scripts it runs for you on certain events (like “a file was edited” or “the agent is about to stop”), without being asked. The one we want is Stop, which runs the instant the agent tries to end its turn.

Wire check.sh to it. In your project, make the .claude folder if it is not there, then create or open .claude/settings.json and add:

{
  "hooks": {
    "Stop": [
      {
        "hooks": [
          {
            "type": "command",
            "command": "bash \"${CLAUDE_PROJECT_DIR}/check.sh\""
          }
        ]
      }
    ]
  }
}

${CLAUDE_PROJECT_DIR} is Claude Code’s own name for your project root, so the hook finds check.sh wherever the agent has wandered to during the turn (quoted so it survives a path with spaces), and runs it through bash so you never have to mark the file executable. The cd at the top of the script does the other half of the job: a hook runs in whatever directory the agent last moved to, not the project root, so the cd puts the test run back where your tests actually live. If the hook never seems to fire, check that JSON for a typo first: a settings file with a JSON error is rejected whole, so one stray comma takes your hook down with it.

Here is why the exit code mattered. When a Stop hook exits 2, Claude Code blocks the stop: it refuses to let the agent finish, feeds your test failures back to it as the reason, and makes it keep working. The agent cannot tell you it is done while the suite is red, because the suite, not the agent, now decides when the turn is allowed to end. (You need Claude Code 2.1.143 or later. Versions move, so the prove-it step below is how you confirm it on your own machine, not my word.)

When the code is actually fixed, the same gate gets out of the way. Change add to return a + b, and:

$ bash check.sh
$ echo $?
0

Silent, exit 0, the agent is free to stop. Green means go.

The one rule that keeps this honest. The gate does not make the agent honest on its own. It makes exactly one thing checkable from outside it: whether your tests pass. That holds only while the check stays out of the agent’s reach. It must never edit the test, the gate, or the .claude/settings.json hook to slip past them, so keep those out of its edit scope, or read any change to them before you trust a green run. To make that a wall instead of a rule, a PreToolUse hook can hard-block any edit to those files: the same exit-2 move, aimed one tool call earlier. The day you let it rewrite your own test, you have handed the check back to the thing being checked, and you are back to trusting “done.” If your agent likes to “fix” failing tests, say so in your CLAUDE.md: change the code, never the test.

When it still goes wrong

• A gate that always blocks would loop. If the agent genuinely cannot fix the tests, a Stop hook that keeps exiting 2 would trap it. Claude Code ends the turn on its own after 8 consecutive blocks (change the limit with the CLAUDE_CODE_STOP_HOOK_BLOCK_CAP environment variable), and you can have check.sh give up and exit 0 after a few tries if you want a softer limit of your own.
• A big suite makes every stop slow. On a large repo, running the whole suite on each stop attempt drags, and the agent can thrash on small tasks. Point check.sh at a fast subset (the tests near what you changed) and leave the full run to CI.
• Green is only as good as the test. The gate proves the tests pass, not that the tests are enough. A test that asserts nothing passes happily. The gate is exactly as honest as what you put in it.
• It only guards what the tests touch. Untested code, prose claims, “I checked the docs”: the gate sees none of that. It catches the lie that matters most, that the work is not actually working, and leaves the rest to you.

Prove the gate fires

Do not take my word, or the agent’s, that the gate works. A gate you have never watched block is not a gate. Break something on purpose: change a line so a test fails, then ask Claude Code to wrap up. Watch it get pulled back and handed the failure instead of stopping. Once you have seen it block, a clean finish from the agent finally means something.

That is the floor, and it is a real one: the agent can no longer decide for itself that the job is done. Something it cannot argue with does. The next step is widening the gate from “the tests pass” to “the tests are worth passing,” which is the next walkthrough.

---

If you want the why under this: How Reliable Is Your AI Agent makes the broader case, and The Stable Liar is this exact failure mode up close.

---