AgentStack v0.1 Open benchmark · Apache 2.0

The coordination tax
is a cache miss.

Multi-agent systems reprocess the same context once per agent. UnitStack measures what inference benchmarks can't — the duplicated prefill, the GPU idle time, and the true cost per completed task once you scale a fleet.

Positioning

Three benchmarks, three questions.

AgentStack sits one layer above the GPU-inference benchmarks — it measures the performance of completing an agent's whole task, not a single forward pass.

LayerBenchmarkThe question it answers
Cloud quality ClusterMAXSemiAnalysis How good is this cloud at hosting GPUs?
GPU inference InferenceXSemiAnalysis How fast is this GPU at serving a model?
Agent systems AgentStack What is this GPU's performance for completing an agent's task?
Why it matters

The workloads that were priced and abandoned.

Plenty of teams scoped a multi-agent workflow, ran the token math, and shelved it. If duplicated prefill is most of the bill, the answer changes — and the workload comes back off the shelf.

01

Ten agents, one case file

Ten agents reading the same context means ten prefills of the same tokens. On a metered API you pay for every one — and it never shows up as waste. It looks like legitimate input.

02

Sharing needs the server

Removing the duplication means reusing key-value state across agents. That requires owning the inference server. It is not purchasable from any frontier API at any price.

03

Scale breaks the trick

The cache mechanism that works beautifully on one node returns almost nothing at 150 — unless routing is prefix-aware. That crossover is exactly what the model finds.

derivation — balls in binscold(N, K)
# expected cold prefills, random routing over N nodes, K shared calls
cold(N, K) = N · (1 − (1 − 1/N)^K)
hit_rate   = 1 − cold(N, K) / K

# N = 1,   K = 24  →  1 cold prefill,    96% hit rate
# N = 150, K = 24  →  ~22 cold prefills, 7% hit rate

See where renting stops beating owning — on your own numbers.

Every assumption is a control. Nothing is baked into a single quotable number.

Tools  /  KV Cache Sharing

KV cache sharing across a 150-GPU fleet

Deployment-tier model — every value is an assumption until you replace it with a measurement.

01

Cache hit rate vs fleet size

Prefix-aware routing Random load balancing
02

Cost per 1,000 tasks vs fleet size

Frontier is token-priced; the rest are hardware-priced. Self-hosted tiers hold cost flat because routing is yours to control. Y-axis is logarithmic.

03

Cumulative spend over 36 months

04

Summary

TierRouting controlHit rateCost / 1k tasks

Throughput, capex, rates and workload shape are placeholders. Replace each with a measured value before any of these curves is quotable. The hit-rate curve is the exception — it follows from routing behaviour rather than from hardware.

Benchmark · Open method

The Coordination Tax Is a Cache Miss

An open benchmark for KV cache sharing across four deployment tiers.

AgentStack by UnitStack · 8 min read

Multi-agent systems reprocess the same context once per agent. Ten agents reading one case file means ten prefills of the same tokens. On a metered API you pay for every one of them and it never shows up as waste — it looks like legitimate input.

Removing that duplication requires reusing key-value state across agents. That requires owning the inference server. It is not purchasable from any frontier API at any price.

This benchmark measures how much that is worth, at what scale it stops working, and where the crossover sits between renting and owning.

Four tiers

The line that matters is not cloud versus on-premise. It is tenant versus operator.

Frontier APIManagedRented GPUOwned
Exact token sequenceNoUsually noYesYes
Cross-agent KV sharingNoNoYesYes
Cache offload to CPU/NVMeNoNoYesYes
Routing controlNoneNoneYoursYours
Idle capacityNoneProvisionedStoppableBackfillable

Rented GPU gives you every cache capability that owned hardware does. On-premise adds utilisation economics, fabric choice, and data residency. Keep those arguments separate — conflating them is the fastest way to lose a technical audience.

The prompt-order trap

Frontier prefix caching works when prefixes are byte-identical. Agents differ by construction — that is what makes them different agents. If role instructions sit at the top of the prompt, prefixes diverge at position zero and no cache anywhere hits.

Prompt layout order can matter more than deployment tier. Put the shared corpus first and role instructions last, and frontier caching recovers a large fraction of the benefit. This is free to fix and almost nobody does it — the model's "role instructions first" switch shows the cliff.

The result that survives extrapolation

Most of this benchmark is measurement. One part is derivation, and it is the part that decides what happens at fleet scale. With random load balancing across N nodes and K calls sharing a prefix, the expected number of cold prefills is the balls-in-bins expectation:

derivationcold(N, K)
cold(N, K) = N · (1 − (1 − 1/N)^K)
hit_rate   = 1 − cold(N, K) / K

At N=1, K=24: one cold prefill, 96% hit rate. At N=150, K=24: roughly 22 cold prefills, 7% hit rate. Scaling the fleet destroys the cache benefit unless routing is prefix-aware — pinning a task's calls to the pipeline that already holds its state.

Metrics

MetricWhy it's here
Concurrent agents per GPUConverts to node count in a quote
Coordination taxThe waste, made visible
Cache hit rateValidates the routing model
Cost per completed taskPrices the waste
Scaling efficiencyWhere affinity routing pays

Coordination tax was always the right name for this. It turns out to be, mechanically, a cache miss.

Reproduce it

Every number should be regenerable from the published configuration on your own hardware. If your workload shape differs — and it will — the calculator takes your parameters rather than ours. Results that only hold on the author's cluster are marketing. Results that reproduce are a benchmark.

Try it on your own numbers. The interactive model turns every assumption above into a control.