Continuum \ Advanced inference

Durability costs an extraction 18 ms.

Continuum's fact extraction made directly against the same production activities in a Temporal workflow, on LM Studio and vLLM: 48 frozen sections, three repeats, judged, and Continuum's cost held from 2.5-second calls to 21-second ones.

UPDATED 2026-09-26 · VERSION 0.1

The benchmark

Fact extraction is Continuum's heaviest model workload: a model reads each section of an ingested document and returns its facts, each with its evidence quoted, and the fact store records them. The benchmark runs the production code both ways. Directly, it calls the extractor and then the recorder. Durably, it runs the production activities in a Temporal workflow, under the extraction workflow's own activity options. The corpus is 48 sections of two of Continuum's own documents, frozen, each extracted three times per arm on Nemotron 3.5 Lightning, with LM Studio and vLLM each alone on the card. Every run is kept, with its setup.

Like for like

The workflow adds 18 ms to an extraction at p50, 0.6% of the call.

The time outside the activities is the workflow's whole run less its two activities, the model call and the fact-store write, so neither the model nor the database is in it. It came to 18 ms at p50 and 30 to 33 ms at p95 on both runtimes, around an extraction of 2.6 to 3.1 seconds. It counts first attempts only, since a retried run's extra time is Temporal's backoff between attempts.

ms	LM Studio `UD-Q4_K_S`	vLLM NVFP4
Outside the activities, p50 / p95 / slowest	18 / 33 / 147 (136 runs)	18 / 30 / 131 (138 runs)
Workflow started to the extraction starting, p50 / p95	5 / 17	5 / 16
Extraction done to the write starting	5 / 6	5 / 5
Write done to the workflow done	4 / 6	4 / 5
The extraction, direct, p50 / p95 / slowest	3,136 / 6,480 / 7,624	2,574 / 5,936 / 6,985
The fact-store write, direct, p50 / p95	26 / 60	28 / 65

The fact-store write costs 26 to 28 ms at p50 and runs in both arms, so it isn't Temporal's. Paired on the same section, repeat and answer, a durable extraction took 20 to 22 ms longer than the direct one at p50. On LM Studio that pairing's p95 is 124 ms, which is the model call varying inside the activity with the answer unchanged: the time outside the activities is 33 ms at p95 on the same runs.

LM Studio's rerun, played back in order: 46 durable extractions, one per section that finished, each on its first attempt. Each goes out along the top rail to Temporal, the worker and the model, and comes back along the rail beneath it. The readout counts only while the extraction is outside the model call, and each run's time lands in the histogram. The fact-store write, the workflow's second activity, isn't drawn and isn't counted. At 46 runs the p95 settles and the p99 is never drawn: it would be the slowest run under another name.

The slowest run on each runtime is a Temporal connection being renewed.

LM Studio's slowest run, section 25's second repeat, spent 147 ms outside the activities, 130 of it waiting for Temporal to acknowledge the workflow's start. vLLM's, section 43's second repeat, spent 131, 107 of it in the start. The start's median is about 3 ms, and everything after it was normal. Temporal's frontend renews a client's connection about every five minutes, and the durable inference benchmark traces the same stall, at about 120 ms with Temporal pinned to 127.0.0.1.

Retries, and the rerun

Retries. The longest sections needed more than the 2,048 tokens these runs allowed an answer, so their JSON ended mid-fact. A model that repeats itself fails the same way on every attempt: on LM Studio the workflow retried each three times, and all failed. On vLLM, which varies, 3 extractions succeeded on their second attempt and 3 failed. A 4,096-token cap extracts every section, as Lightning's page shows.
The rerun. LM Studio ran again after vLLM, one repeat of every section. The direct extraction moved 3 ms at p50, the time outside the activities came to 19 / 29 ms, and every section gave the same facts as the first run.

Standing still

From a 2.5-second call to a 21.6-second one, the workflow's cost stays at 18 to 21 ms.

The settings study ran the same benchmark under every extraction setting it tried, 48 sections per arm, so the calls ranged from 2.5 seconds to 21.6. The workflow's time outside the activities held at 18 to 21 ms at p50 throughout. Its slowest run on most settings is the same connection stall, which a longer run meets more often.

Setting	One extraction, p50	Outside the activities, p50	Slowest
LM Studio, the studies' settings	3.1 s	18 ms	147 ms, a start stall (136 runs)
LM Studio, cap 4,096	3.4 s	18 ms	156 ms, a start stall
LM Studio, NVIDIA's temperature-1.0 profile	2.5 s	19 ms	32 ms
LM Studio, thinking, budget 1,024	8.4 s	20 ms	113 ms, a start stall
LM Studio, thinking, budget 2,048	11.7 s	20 ms	145 ms, one of three start stalls
LM Studio, thinking, budget 4,096	21.6 s	20 ms	134 ms, a start stall (47 runs)
vLLM, the studies' settings	2.6 s	18 ms	131 ms, a start stall (138 runs)
vLLM, cap 4,096	3.1 s	21 ms	112 ms, a start stall
vLLM, NVIDIA's temperature-1.0 profile	2.6 s	20 ms	127 ms, a start stall
vLLM, thinking, budget 2,048	11.7 s	20 ms	116 ms, a start stall
vLLM, cap 4,096, compact JSON	2.9 s	21 ms	33 ms

In every stall the time went on acknowledging the workflow's start, 95 to 130 ms against a median of 3, and everything after the start was normal. The 2,048-budget run on LM Studio lasted 21 minutes and met three, where the six-minute runs met one or none.

Same again

Where the model repeats itself, the durable arm returns the same facts as the direct one.

On LM Studio, which repeats itself at a fixed seed, durable and direct gave the same answer on all 136 pairs, and 43 of the 48 sections gave one answer across all three repeats. vLLM varies at a fixed seed, so its two arms matched on 24 of 138 pairs and its repeats on 3 of 48 sections. The workflow changes nothing about the answer: the same request reaches the same model, from inside an activity.

First repeat, direct	LM Studio `UD-Q4_K_S`	vLLM NVFP4
Sections extracted	46 of 48	46 of 48
Facts accepted / rejected	306 / 51	294 / 40
Clean, by the judge	202 of 306 (66.0%)	198 of 287 (69.0%)
Assertions missed	118	140

The validator accepts a fact only when its evidence is in its section word for word, and rejects the rest before they reach the store. A judge model, Qwen3.8 27B, graded each runtime's first facts without knowing which runtime wrote them, and the two tie: 3 points apart against a standard error of 3.8. Lightning's page has the judging in full, and the settings that extract the most.

By the book

Both arms run the production code, and two things are left out on purpose.

Direct: the extractor's ExtractAsync, then the recorder's RecordAsync, each timed, as the Console's fact extract runs them.
Durable: a workflow of the production activities, ExtractFactsAsync then RecordFactsAsync, with the extraction workflow's own options: a five-minute extraction that heartbeats and gets three attempts, and a write that retries.
Left out: the production task queue. The workflow runs on its own queue, on a worker in the benchmark process, because the platform's workers poll the production queue and would otherwise take the vLLM arm's work to LM Studio.
Left out: the projection child. Production starts a child workflow that embeds the new facts, abandoned by design and run on the projection worker, so it isn't on the extraction's path. Here it would embed in the background during the measurement.

Corpus	Continuum's `CONTEXT_LAWS.md` and `RULES.md`, frozen with their SHA-256, split by the production splitter into 48 sections
Runs	3 repeats of every section per arm, after 2 untimed warmup sections per arm. Arms alternate section by section, and the arm that goes first swaps each repeat
Percentiles	Nearest rank, p50 and p95; the slowest run is named, and no p99 is quoted below 200 runs
Model	Nemotron 3.5 Lightning 30B-A3B: `UD-Q4_K_S` on LM Studio 0.4.25 (runtime 2.46.0), NVIDIA's NVFP4 on vLLM 0.29.0 in WSL2; each alone on an RTX 5090. The full setup is on Lightning's page
Sampling	The extraction settings as stored: temperature 0.2, top-p 0.95, top-k 0, min-p 0.01, repeat penalty 1.0, thinking off, 2,048 max tokens, seed 42, and a strict JSON schema for the answer
Temporal	The dev server, at `127.0.0.1:7233`
Judge	Qwen3.8 27B, loaded alone, grading each runtime's first-repeat direct facts against their sections; it never sees the runtime

A first pass was discarded and run again after its rejections turned out to be measuring the validator's handling of markdown, not the models: the quality numbers were measuring markdown.

# 48 sections, 3 repeats, direct and durable, alternating (LM Studio's run)
env 'ConnectionStrings__continuum-temporal=127.0.0.1:7233' dotnet run -c Release --project src/Apps/Continuum.Console -- \
  benchmark durable-extraction --repeats 3 --warmup 2 --label lightning-q4ks

# then the judge, loaded alone, on the first repeat's direct facts
dotnet run -c Release --project src/Apps/Continuum.Console -- fact judge --run <run>