Nemotron 3.5 Lightning 30B-A3B

Decode drifted by up to 15% within a run on both runtimes, with the memory clock constant, so a gap that small is a draw. The card's core clock was also capped at 2,557 MHz for the first 45 minutes after a reboot, cause unknown, which covered vLLM's short run and about its first 70 long runs. Run again at full clock, vLLM's short prompt took 210 ms at p50 against 227.

Extraction is the one clear lead: vLLM was 15% faster, and a rerun of LM Studio after vLLM moved its time by 3 ms, so the lead isn't drift. The 4K document's first token favours LM Studio because these prompts carry no run-id line, so every run repeats the one before it. LM Studio reuses the whole cached prompt. vLLM caches in blocks of 2,128 tokens, so, going by the block size, about half of the document is computed again.

On the long prompt the p99 is a stall. Three runs on each runtime spent 113 to 129 ms outside the model call, nearly all of it starting the workflow, spaced 5 to 10 minutes apart. Temporal's frontend renews a client's connection about every five minutes, and the durable inference benchmark traces the same stall through the whole platform.

The clean rates are 3 points apart, against a standard error of 3.8 over about 300 facts each, so they tie. LM Studio extracted more and missed fewer assertions, and vLLM was faster. Neither invents much: 2 or 3 facts in 300 weren't supported by their evidence, and 7 to 10 weren't faithful to it. The validator keeps invention out before the judge sees a fact, by accepting it only when its evidence is in the section word for word. The weak axis for both is kind, mostly telling an Observation from a Constraint in normative text.

The two builds extract different facts from the same text: matched on their evidence, their sections agree with a mean Jaccard of 0.42. The judge is uncalibrated, holds a few verdicts that contradict themselves on kind, and skipped about 7 of vLLM's facts, so the rates carry its noise as well as the extractor's. What the workflow costs an extraction, 18 ms at p50, is on the durable extraction benchmark.

• A 4,096 cap is needed. Two sections needed 2,448 and 2,241 tokens, and the baseline's cap cut both off mid-answer. At 4,096 they extract, and the other 46 come out byte-identical, because the cap doesn't change sampling on a runtime that repeats itself.
• NVIDIA's temperature-1.0 profile is worse. At that temperature the model paraphrases the evidence it was told to copy, so the validator rejects more and the judge finds fewer clean facts.
• Thinking finds the most. 359 clean facts against the raised cap's 217, and 74 missed against 133. The clean rate rose with the count, so the extra facts aren't padding. It splits lists harder, and more of its facts are not atomic: 49 against 26.
• vLLM ranks them the same way. Thinking gave 323 clean facts (74%) and 105 missed, against 200 and 143 with the cap raised alone. Asking vLLM for compact JSON cut the output by 17%, 754 tokens a section against 905, at the same clean rate, 62%. LM Studio has no equivalent setting.

The sweep counts facts and time, and wasn't judged. At 4,096 one section's thinking and answer together passed the 8,192-token cap. Through every setting and budget, the workflow's time outside the model call held at 18 to 21 ms at p50, from 2.5-second calls to 21.6-second ones.

LM Studio updated its own runtime from 2.45.0 to 2.46.0 on the morning of the runs, before the first LM Studio run, so the Nano's runtime version doesn't carry over here. vLLM loaded the weights from a Docker volume in 6.5 s, where a bind mount from Windows had taken 107 s the day before.