What the catalog said
On October 15 we moved the agent catalog to the newest Claude models. Its 18 agents hold 99 model nodes, and each one now names Haiku 5.5, Sonnet 5.5 or Opus 5.5, with an explicit reasoning effort.
What actually ran
Four days later, while writing down which model and effort each agent should use, we read the code path from manifest to model call, and found that most of those names never run.
- Planners make no model call. 49 of the 99 nodes are “passthrough” planners that only split the work, so their model is a placeholder.
- Workers take the runner’s model, not their own. Every worker sets
use_tier_model: true. Orca, our agent engine, then replaces the node’s model with the first model on the runner’s tier list whose provider has an API key in the run’s environment. - The runner had one key, and it was Gemini’s. So the workers resolved to Gemini, not to the Claude models the catalog showed.
That last step was read from the code, not observed. Our hub didn’t record which model served a run, so we couldn’t check after the fact. That gap is part of the design we wrote that day: what the catalog says should be what runs, and every run should record the model it used.
Why not Claude through Google
The obvious route was Claude on Google’s Vertex AI, where the rest of our AI already runs. On October 19 a test of that route found zero Claude quota in every one of our Google Cloud projects: every call returned a 429 with RESOURCE_EXHAUSTED. We filed quota requests that night, and they were turned down automatically.
The switch
On the evening of October 20 we gave the runner an Anthropic API key, kept in Secret Manager. It sits beside the Gemini key, readable only by the runner’s identity. Orca’s default tiers prefer Claude once that key exists: Haiku 5.5 for economy, Sonnet 5.5 for standard, Opus 5.5 for frontier.
One property of that design matters operationally. The model is picked by which keys exist, not by which calls succeed. If the Anthropic key stops working, runs fail; they don’t quietly fall back to Gemini. We think that’s the right failure, because a silent fallback is how we got here, but it’s worth knowing.
Aria isn’t part of this change. Aria runs inside the customer’s site and calls Google’s Vertex AI from the customer’s own Google Cloud project, as before.
What the first Claude runs showed
A failed lane with no reason. The first Claude run of one catalog agent reported “1 of 3 lanes failed”, and nothing in the logs said why. The runner reported only counts, and the lane’s details lived in a database inside the container, gone when the execution ended. Another agent had shown 2 of 3 failed lanes for days on Gemini, with the same blind spot. The runner now logs one warning line per failed lane: the node, the error code, the model, the attempts and a short, scrubbed message. It never logs the prompt, tool data or the lane’s text. Our best guess for that first failure is an analyst hitting its 10-step tool limit, but we can’t confirm it from the logs we had.
Thinking that was never sent back. While adding prompt caching to the hosted runner the same night, we found that the worker loop never sent Claude’s thinking back on later steps. From step 2 on, thinking was on but the previous turn carried no thinking block, and Claude rejects that request. We didn’t see it fail live, but any hosted Claude run with tools was exposed until the fix shipped.
The same change put cache breakpoints on every Claude call and made Claude histories append-only, so each step reads the earlier steps from cache. We estimated about 55% less input spend for a typical four-step run. That’s arithmetic, not a measurement: the per-run cache counts now go into the run summary, and we’ll report real numbers when we have them.
The Terraform that would have undone it
The key and its permission were added with gcloud, not Terraform. A terraform apply on the runner’s module would have removed both, and every Claude lane would have failed with a missing key. It would also have switched off NAT error logging we’d turned on while chasing a separate bug that day.
We declared all three in Terraform, then ran a read-only plan: 2 to add, 1 to change, 0 to destroy. The two “adds” already existed live. The local state they were compared against dated from August 27.
What we’d tell someone else
Check what actually runs, not what the configuration says. A manifest that names a model is a request, and something between it and the API call can rewrite it.
And when you change production by hand, write it into your infrastructure code the same day, or someone’s next apply will quietly put it back.