Map where it's reliable, and engineer for the edges.
A model isn't good or bad. It's reliable to a degree, across many dimensions at once. Engineering with it means knowing that shape, working inside it, and building structure where it's thin.
The same model can refactor a module flawlessly and then invent an API that doesn't exist. Write excellent tests, then lose track of a change across twenty files. Both observations are true, so a yes or no about the model as a whole is meaningless.
The useful question is where it's reliable, how reliable, and what happens at the edges. That's an envelope, like the flight envelope of an aircraft: a shape with many dimensions, not a score.
Draw it as a map: kinds of task down one side, size of change across the top, colour for how often the result is right without help. Every model has its own map. Two models with similar benchmark scores can have very different ones.
Then engineer the system to make it greener. Not by waiting for a better model, but with structure: types, rules, smaller steps, and loops that catch mistakes. The map is of the model plus everything around it.
Illustrative maps, not measurements of any real model.
Everything around the model was tuned to its envelope: which tasks you hand it whole and which you break down, where you added checks, what the skills warn about. A new model has a different map. Some of that tuning is now wasted, and some of it is missing.
So the team goes through Tuckman's stages again, as it would with a new colleague. Plan for it. Your encoded judgement (skills, rules, checks) carries over; the calibration has to be redone. A new model can be worth it. It's never free.
A tool works or it doesn't. An instrument has a range, a feel and quirks that you learn by playing it for a long time. A good player knows where it sings and where it buzzes, plays to its strengths, and works deliberately at the edges.
Hand a player a different guitar and they're still a musician, but they'll sound worse for a while. That's the same thing as the hours of practice in the learning curve: much of it is learning one model's envelope.
Once you think in envelopes, the line between “the model” and “the rest” disappears. The prompt, the skills, AGENTS.md, the types, the tests, the checks, the review loop and the merge gate are all either context going in or feedback coming back.
The thing that's reliable or unreliable is that whole system. And each part of it is something you can engineer, which the model on its own isn't.
Context: prompts, skills, AGENTS.md, types, specs
Loops: tests, checks, review, merge gate, production
Where should the trellises go? Exactly where the envelope is thin. A model that loses track of changes across many files needs a codebase of small, well-bounded modules. One that invents APIs needs strict types and generated clients. One that writes plausible but wrong concurrency needs scenario tests that force the interleavings.
The structure in the next chapters, rules that enforce themselves and derivation, isn't generic best practice. It's the shape of the envelope, pressed into the codebase.
| Where it's thin | Structure that hardens it |
|---|---|
| Long, many-file changes | Small modules, clear boundaries, smaller steps |
| Invented APIs and fields | Strict types, generated clients, contract tests |
| Concurrency and ordering | Scenario tests that force interleavings |
| Drifting conventions | Architecture rules in CI, skills that say why |
| Duplicated facts | One source, everything else derived |
Public benchmarks tell you about someone else's tasks. Your envelope is about yours: your codebase, your stack, your kind of change. Build the map from evidence you already have, and keep it current.
When a new model comes out, run the same evals on it before you switch. Then you know what you're trading, and where to move the guard-rails, before production tells you.
| Source | What it maps |
|---|---|
| Evals on your own tasks | Planned exploration, comparable across models |
| Escapes, by category | Where the system leaks today |
| Review and loop findings | Which mistakes the model makes, and how often |
| Rework and reverts | Where “done” didn't hold |
| Per model and version | So a swap is a decision with evidence |
A model is reliable to a degree, across many dimensions. Map that envelope on your own work, play inside it, explore its edges on purpose, and harden them with structure and loops. Treat a model swap as a new hire, not an upgrade. And remember that what you're really engineering is the whole system: it's all context, and it's all loops.