Model Routing Is Simple. Until It Isn’t.
Which summary reads better? Pick one — models revealed after.Both summaries are AI-generated.
GPT-4.1 cost nearly double per task ($0.37 vs $0.19) despite lower token pricing because caching slashed Sonnet’s effective input costs. Routing by sticker price alone will misfire—your cost model must account for workload reuse and serving infrastructure. Expect 2–3× cost swings on the same workload if you ignore cache hit rates.
Cache hit rates can cut effective input costs dramatically, with one model's lower cache-read pricing resulting in nearly half the total cost of another despite higher base pricing. This means model routers must account for serving infrastructure and workload patterns, not just model pricing, to optimize costs for agent workloads that reuse context across steps.