Durability costs a call 16 to 24 ms, on every model measured.
How Continuum is measured, and what it costs: durability against the same call made directly, on two models and two runtimes, the whole platform around it, fact extraction, and a job too long for a direct call.
UPDATED 2026-09-26 · VERSION 0.1
The benchmarks
Every benchmark here makes the same model call twice, directly and through Continuum, and measures the difference. The like-for-like test puts a Temporal workflow around the call and nothing else; the whole platform, which does far more than run the call, is measured separately. Two models ran on two runtimes, on one machine, for inference, fact extraction and a job too long for a direct call. Each flow and each model has its own page, and every figure on them was recomputed from the run's raw data.
The bill
Durability costs the same on every model and runtime measured.
The startup consensus is that a workflow engine is too complicated and too slow for a team that has to move fast. Measured against the call it wraps, a durable workflow adds 16 to 24 ms at p50, on a 4-billion-parameter model and a 30-billion one, on LM Studio and on vLLM. Its own time outside the model call is 13 to 16 ms of that, whatever the model.
| Flow | Model | Adds at p50 | Its tail | Runs per arm |
|---|---|---|---|---|
| Durable inference | Nemotron 3 Nano 4B, LM Studio and vLLM | 16–21 ms | 25–32 ms at p99, outside the call | 200 |
| Durable inference | Nemotron 3.5 Lightning, LM Studio and vLLM | 17–24 ms | 24–32 ms at p99 outside the call; 113–114 on the long prompt, a connection stall | 200 |
| Durable extraction | Nemotron 3.5 Lightning, LM Studio and vLLM | 18 ms outside the activities | 30–33 ms at p95 | 136–138 |
| The whole platform | Nemotron 3 Nano 4B, LM Studio | 35–46 ms | Temporal connection stalls of 317–447 ms, 12 in 600 | 200 |
| A 32,000-token job | Nemotron 3.5 Lightning, LM Studio | Finished in 192.3 s, where the direct call failed at 100.0 | Pass or fail | 1 |
A percentage depends on the answer's length, so each page gives the milliseconds first:
on the Nano, 19 ms is 1.0% of a 658-token answer, and 17 ms is 5.9% of an 88-token one.
The whole platform is its own number, because it does far more than run the call: it
takes the request through the Gateway, writes it to Postgres and Redis, answers
202, streams the answer into Redis and signals the client to fetch it.
House rules
Every figure has a run behind it, and every run is kept.
- Like for like first. The same call, on the same machine, model and settings, made directly and through the workflow in alternation, so the only difference between the arms is the workflow.
- The model stays out of the overhead. Each durable run's time outside its activity is the workflow's own time, with no model time in it, so the model's spread on a long answer can't pass for Continuum's cost.
- A tail needs its sample. A p99 comes only from 200 runs or more. Below that the page names the slowest run for what it is.
- Every stall is named. A run far outside its median is traced through its spans, and the page says where the time went, or that the cause isn't known.
- The data is kept. Every run's raw CSV, spans and answers are stored with this site, and every figure on these pages was recomputed from them. A run left out is kept too, with the reason.
Read on
A page for each flow, and one for each model.
- Durable inference. The model call directly and in a workflow, then the whole platform around it, span by span, and the changes that took the platform from 168 ms to 38.
- Durable extraction. The production fact extractor directly and in its workflow, and the cost held from 2.5-second extractions to 21.6-second ones.
- Long-running calls. A job that outlasts a caller's deadline, and the one timeout to set inside an activity.
- Nemotron 3 Nano 4B. Continuum's inference model on LM Studio and vLLM: quality, latency, throughput under load, and its settings.
- Nemotron 3.5 Lightning 30B-A3B. Continuum's extraction model on both runtimes: speed, judged extraction, the settings that extract the most, and its quirks.
By the book
Every run was taken on one workstation.
| Machine | AMD Ryzen 9 9950X3D, 64 GB, RTX 5090 (32 GB), Windows 11 Pro |
| LM Studio | 0.4.25 on Windows, llama.cpp runtime 2.45.0 for the Nano and 2.46.0 for Lightning |
| vLLM | 0.29.0 in WSL2 under Docker Desktop |
| Temporal | The dev server, run by the platform's Aspire host with the stores |
| Load | One request in flight unless a page says otherwise; each page says which models shared the card |
A number compares only with one taken on the same machine, model and server settings. Docker Desktop's VM sits between the hosts and the stores here, and a Linux server wouldn't have it. Each page carries its own setup in full, with the commands to take the runs again.