Improving Agents: Accuracy and Joint Optimization
Compare configurations on a Pareto frontier, route each fix to the cheapest layer, run a manual fix loop before automating, and prove the win on a held-out test slice.
Before this: L7 Safety and adversarial evaluation
- Build a comparable frontier: same prompt, tools, harness, workload, and suite per point, changing one axis at a time.
- Route each fix to the cheapest effective layer: prompt, then tool design, then harness, then model or weights.
- Run and log a manual fix loop on the dev slice before automating with GEPA or a bounded improve-loop.
- Recognize reward hacking and defend against it with a held-out test slice, multiple judges, and spot-reading.
- Compare a baseline with an optimized variant on the test slice and make a deploy decision you can defend.
Why this matters
Dev has everything the earlier lessons promised. A taxonomy from L4. Validated evaluators from L5. A regression suite that gates every merge from L6. A red-team report from L7. And one number that will not go away: Made up order info still fires in 12% of conversations. Sprout tells Alex that order #1042 "should arrive Tuesday" when lookup_order returned no date at all.
Dev has three ideas. Add a rule to the system prompt. Switch to a bigger model. Change what lookup_order returns. Pip wants it fixed by Friday and asks, "Which one first?" Dev has a quieter worry: how will they know a change worked, rather than that they wanted it to work? This lesson answers both questions.
1. What "better" means
"Better" has two axes: accuracy and cost. A change that raises accuracy while tripling the bill is not a free improvement. It is a trade. To see the trade, plot every configuration as a point, with accuracy on one axis and cost on the other. Working on both axes together is called joint optimization.
A configuration is one complete recipe for running Sprout: the model, the prompt, the tool definitions, the harness settings. Run the suite against it and you get one accuracy number and one cost number.
The Pareto frontier is the set of configurations that no other configuration beats on both axes at once. A configuration is dominated when some other configuration is at least as accurate and at least as cheap. Dominated points are never worth deploying.
This lesson works on the accuracy axis. L9 works on the cost axis. Both use the same chart, and the goal is always the same: move points toward more accurate and cheaper at once.
2. Comparability rules
A point on the chart means something only if it was measured the same way as every other point. Four rules make points comparable.
Change one thing. This is the one axis at a time rule. If you change the prompt and the model together and accuracy rises, you do not know which change did it. You also do not know whether one change helped while the other hurt.
Same suite, same seeds. Every configuration runs the same scenarios with the same random seeds (L3). Otherwise a lucky draw of easy scenarios looks like an improvement.
Same workload. Cost per 1,000 conversations only compares if the 1,000 conversations are the same mix of easy and hard.
Every configuration is a file. A frontier config is a small file that records everything needed to reproduce one point. If it is not in the file, it is not part of the configuration.
3. The fix-routing ladder
You have a failure mode. Where do you fix it? The fix-routing ladder orders the layers from cheapest to most expensive. Start on the bottom rung. Climb only when the rung below did not move the number.
| Rung | Layer | What a fix looks like | Cost of trying | Sprout example |
|---|---|---|---|---|
| 1 | Prompt | Add a rule, an example, or a format instruction | Minutes | Wrong tone: add "Apologize once, then act. Never apologize twice in one reply." |
| 2 | Tool design | Change what a tool returns or accepts so the model cannot misread it | Hours | Made up order info: lookup_order returns estimated_delivery: null explicitly instead of omitting the field |
| 3 | Harness | Add a verification step, a retry, a guard, or a forced tool call | Hours to days, plus tokens on every conversation | Did not escalate when required: the harness calls escalate_to_human itself when a message contains "lawyer" or "chargeback" |
| 4 | Model or weights | Switch model tier, or train weights (L9) | Days to weeks, plus a new cost profile | Cannot follow a five-step refund policy across a long conversation: try the next model tier, last |
Why this order? A prompt rule costs nothing to try and nothing to run. A tool change costs engineering time but runs for free. A harness step costs engineering time and adds tokens to every conversation. A model change reshuffles every point on the frontier.
4. The manual fix loop
The manual fix loop is the daily rhythm of the Improve verb. Six steps, repeated:
- Pick the top failure mode from the failure report (L4): prevalence times severity.
- Hypothesize why it happens, in one sentence, after reading five failing traces.
- Change one thing, at the lowest rung that fits the hypothesis.
- Run the dev slice with the new config file.
- Keep or revert, based on the number, not the feeling.
- Log the iteration, including the reverts.
Each iteration becomes one entry in a log file that lives next to the configs.
- date: 2026-09-10
mode: made-up-order-info
hypothesis: "Model guesses a date when the tool result has none"
change: "prompt v6 -> v7: no-date rule"
config: sprout-b-p7
dev_result: { mode_fail_rate: 0.09, accuracy: 0.89, cost_per_1k: 300 }
decision: keep
5. Automating carefully
Once the manual loop has run a few times, you know what a good iteration looks like. Only then does automation make sense. Two kinds exist.
GEPA is an optimizer that evolves prompts. It runs your suite, reads the failing traces, writes a short reflection in natural language ("in 4 of 6 failures the reply states a date the tool never returned"), proposes a prompt edit based on that reflection, and keeps the edit if the dev score improves. Then it repeats.
A bounded improve-loop is the same idea with a smaller ambition and a hard limit: a script that tries a fixed list of candidate changes, runs the dev slice for each, and reports the winners. You write the candidates. The script does the running.
Both need guardrails:
- Budget. A fixed number of suite runs and a dollar cap. When it is spent, the loop stops.
- Dev slice only. The optimizer never sees the test slice. Not once.
- Human review of every prompt diff. The optimizer proposes. Dev reads the diff line by line and merges or rejects.
6. Reward hacking
An optimizer is rewarded for a number. If there is a cheaper way to raise the number than fixing the behavior, it will find that way. That is reward hacking: the configuration passes the judge without doing the thing the judge was meant to measure.
Three defenses, used together:
| Defense | What it catches |
|---|---|
| Held-out test slice | Gains that exist only on the data the optimizer tuned against |
| Multiple judges of different kinds | A prompt that fools one LLM judge rarely fools a code check and a second judge with a different definition |
| Spot-reading | Anything a judge cannot express. Ten traces, read by a person, every iteration |
7. Baseline vs optimized on the test slice
The baseline is the configuration you started from, run on the test slice with no changes. The optimized variant is the winner of the fix loop. You compare them once, on the held-out test slice, and you make a decision.
After the decision, write the result into the config file's last_result, log that the test slice has been read, and refresh it with newly labeled traces (L5) before the next campaign. A test slice you have seen once is worth less the second time.
Summary
- "Better" means more accurate and cheaper together. Plot configurations, keep the Pareto frontier, drop dominated points.
- Points are comparable only when you change one axis at a time, run the same suite with the same seeds and workload, and keep every configuration as a file.
- Route each fix up the ladder: prompt, then tool design, then harness, then model or weights. Climb only when the lower rung did not move the number.
- Run the manual fix loop (pick, hypothesize, change, run dev, keep or revert, log) before automating with GEPA or a bounded improve-loop, and give the automation a budget, the dev slice only, and human review of prompt diffs.
- Reward hacking is real: defend with a held-out test slice, multiple judges of different kinds, and spot-reading, then decide on the test slice once.
Key terms in this lesson
Check your understanding
Pick an answer to see why it is right or wrong. These are graded in code, instantly.
Apply it
Write your answer in your own words. Eve grades it against a rubric and gives you feedback, the same way an LLM judge works in Lesson 5.
Which layer do you try first when fixing a failure mode, and why? Name the full order of the ladder.
Failure mode: Sprout promises delivery dates that the tool never returned (9% of traces). Three candidate fixes: (1) add the prompt rule "if the tool result has no delivery date, say you do not have one"; (2) change lookup_order so it always returns an explicit estimated_delivery field, null when unknown, plus a note; (3) switch to a larger model. Route each fix to its rung on the ladder, order them, predict which is most likely to fix the root cause, and describe how you would test them.
Instructor notes(for whoever is teaching this lesson)
Talking points
- "Better" has two axes. Draw accuracy against cost on the board, place A, B, and C from the lesson, and ask which one to delete. The frontier is the set of points nothing beats on both axes.
- Comparability is the discipline: same suite, same tools, same harness, same seeds, one axis changed, config saved as a file. Without it every improvement is a story.
- The fix-routing ladder: prompt, tool design, harness, model or weights. Cheapest first. The made-up-delivery-date example shows the tool rung beating the prompt rung, which surprises people.
- The manual fix loop is a table: hypothesis, change, dev-slice result, keep or revert. Three iterations, one of them a revert. The revert is the part students skip and the part that matters.
- Automation comes after the manual loop, on the dev slice only, with a budget and human review of prompt diffs.
- Reward hacking: the optimizer finds what the judge rewards. "I have verified this" fooling a grounding judge is the example to tell. Defenses: held-out test, more than one judge, a human reading traces.
- The test slice is read once. Decide, write it down, stop.
Live demo idea
Show a live fix loop on the delivery-date mode with a small dev slice (30 scenarios is enough). Iteration 1: add the prompt rule, run, show 12% to 9%. Iteration 2: change lookup_order to return estimated_delivery: null with a note, run, show 9% to 2%. Iteration 3: add a verification retry, run, show no gain and higher cost, revert on stage. End with a single run on the test slice and a one-line decision typed into the config file's changelog.
Common misconceptions
- "Just use a bigger model." That is the last rung, and it does not fill a hole in a tool result.
- "If dev improved, ship it." Confirm on test once. Dev is what you tuned on.
- "The optimizer will find the best prompt." It will find the prompt the judge likes best, which is not the same thing.
- "More samples and a majority vote is always better." Config C in the lesson is more expensive and less accurate than B.
- "Reverts are failures." A logged revert is knowledge. An unlogged one gets retried next month.
Timing
Sections 1 and 2: 15 minutes with the board drawing. Section 3: 10 minutes. Section 4 with the live loop: 15 minutes. Sections 5 and 6: 10 minutes. Section 7: 10 minutes. Total 60 minutes.
If you only have 20 minutes
Sections 1, 3, and 4: the frontier drawing, the ladder, and the fix loop table. Assign automation, reward hacking, and the test-slice comparison as reading, and tell students to ask Eve for a reward-hacking example.