Prompts are the new legacy code

Dispatch’s code review agent recently lost about 80% of its system prompt, along with all four of the sub-agents that used to run underneath it. The main before-and-after tests ran on Claude Opus 5, with no change in quality. Cost per review more than halved, and reviews came back about 25% faster.

​

Deleting a prompt that works

The prompt-cutting experiment started with Thariq Shihipar’s post on the new rules of context engineering for Claude 5 generation models. Anthropic had removed more than 80% of Claude Code’s system prompt for newer models without lowering its coding evaluation scores. Applying that result to a production agent requires evidence. The existing prompt already works, and the agent runs every day. You need to prove that removing instructions doesn’t reduce quality.

​

Recipes lose to pointers

“More context is better” is how most people learned to work with these models. That advice is about data, not about prompts. Data means the diff, the logs, the profile, the file that needs reading. Give the agent as much of that as it needs. The prompt is different: past a certain point, every line you add makes the agent worse at its job. Upsun Cloud’s performance agent went through two versions of its prompt. One read like a runbook: start here, then check this, then decide if that’s the problem, then do this. The other version said here’s the continuous profiling endpoint, here are the logs, here’s how you inspect transactions, go find the regression. The second version performs a lot better. Every instruction you write is also a restriction. Tell the model to check three things in order, and it may skip another useful check. Sometimes, that missed check is the one that matters. Left alone, the model often picks a better order than the one you wrote.

​

Silent bugs

Conflicting rules often appear only after months of edits. Every rule looks right the day you write it. You know performance, you know what to look at, and the prompt says so. Then a new case shows up, and you add another rule. The rule from three months ago may now conflict with it in a subtle way. The prompt gets longer and vaguer at the same time. Debugging the performance agent once meant chasing down why it wouldn’t report an issue. It had found the issue. It was throwing it away, because of a narrow rule someone added for an unrelated case and never took out. A prompt gives you no stack trace. The rule fired exactly as written, outside the situation its author had in mind. The prompt is code. It has behavior, it regresses, and it accumulates rules whose purpose is no longer clear. It gets harder to change as it grows. Treat it like code, and keep it small.

​

The model fights you on this

Ask a model to make a chunk of your prompt shorter and more generic, and it mostly won’t. It rewrites the section so the one failure in front of it passes, usually by adding another special case. Models are built to solve the problem in front of them. A narrow fix does that. Making the prompt smaller is your goal, not the model’s, so you have to keep asking for it.

​

Sub-agents are a context tool

The old setup ran four reviewers with different lenses: one for security, one for correctness, and so on. The lenses overlapped, producing duplicate findings and repeated costs. Every sub-agent gets its own context. Each one reads the same diff and surrounding files, then spends its own thinking tokens on that reading. Nothing is shared between them. Four lenses over one diff means paying four times to read the same files and waiting for four sets of round trips. Sub-agents pay off when their contexts don’t overlap. Claude Code includes a read-only agent for codebase search called Explore. Several Explore agents can cover independent areas in parallel. Four reviewers covering the same diff repeat the same context instead. You could reduce the overlap by reading files in the coordinator or assigning specific files to each lens. Both approaches add coordination and weren’t necessary here. One reviewer replaced all four. Its prompt tells it to search aggressively for defects and report everything it finds. Today’s models know what a code review agent is and what a defect is. A reviewer told to find defects is already looking for security bugs. A single reviewer still has a trade-off. Push it to catch more real bugs, and it also starts flagging things that aren’t bugs. A dedicated security lens may perform better when security is the product. That hasn’t happened in these tests.

​

Evals are the spec

You need an evaluation set: a fixed batch of test cases scored the same way every time. Deleting 80% of the prompt was only safe because there was one to check against. Without it you’re guessing, and those four sub-agents would still be running. This set is 25 cases, all written by hand. Synthetic cases didn’t hold up in these tests. It’s small on purpose, and the cases are deliberately different from each other: security, correctness, dependency changes, etc. Cases that all exercise the same behavior cost money to run and tell you nothing new. The initial evaluation set favored the old prompt. The lean prompt failed cases the old prompt had passed. Five of those failures were false alarms. The old prompt had only passed them because a phrase buried in it matched those cases. The lean prompt no longer had that phrase. That tuning wasn’t deliberate. An evaluation fails, you ask the model for a fix, the fix reads generic enough, and you merge it. After a few months, individual test cases start shaping the prompt instead of its purpose. That’s overfitting, same word as in model training and the same meaning. The prompt scores well on the cases you have and does worse than that score suggests on everything else. Your users notice before you do. Those five failing cases had to come out of the set before its score meant anything. Finding them was most of the work. Budget for the runs. A full pass over these 25 cases cost roughly $20 before the changes and $7 afterward. Landing the final prompt took hundreds of passes. A change that fixes one case can break another. Treat the eval set as the spec. It’s the thing that says what the agent has to do. The prompt is one implementation of it, and a smaller implementation that still passes is a better one.

​

A shorter prompt transfers between models

Cost and speed improved right away. The prompt also got easier to move between models. A prompt with 200 rules in it ends up tuned to the model you wrote it against. Half those rules exist to work around behavior a newer model doesn’t have anymore. Switching models means finding out which half, one regression at a time. A short prompt has few of those workarounds, and the eval set tells you in one run whether the swap is safe. Moving this agent from Opus 4.8 to Opus 5 takes it from catching around 70% of the real bugs in the set to around 90%. Opus 5 catches real bugs that 4.8 was known to walk past, and it does it with fewer false positives. The refactored prompt also costs less and scores slightly higher on OpenAI’s models, although it isn’t faster. GPT-5.6-Terra is much cheaper and faster than Opus. It scores slightly below Opus 4.8, making it an option when cost or speed matters. Opus 5 scores higher than both. GPT-5.6-Sol is worth another look now that its price came down. Those comparisons depend on using the same prompt for each model.

​

If you’re trying this

Start from nothing. Say what the agent is for, hand it the tools and the references, and let it work out the order. Add a rule only after you’ve seen the same failure twice. Every rule you add becomes something you have to maintain. Before you add a sub-agent, ask whether its context overlaps anything else in the run. If it does, you’re paying twice to read the same thing. Write the eval set by hand, keep the cases different from each other, and avoid overfitting. Then start deleting, and let the eval set tell you when you’ve gone too far. Without one, don’t try any of this.