A 32,000-token job fails a direct call and finishes in a workflow.
One 32,000-token job, made directly under a caller's deadline and through a Temporal workflow whose activity heartbeats: the direct call failed at 30 and 100 seconds, and the durable one finished in 173 and 192.
UPDATED 2026-09-26 · VERSION 0.1
The benchmark
One job too long for a direct call. Nemotron 3.5 Lightning, alone on LM Studio at 65,536 context, gets a 30,933-token prompt: Continuum's eleven host and app documents, with the instruction to copy every section in full and comment on it, so the answer's length follows its input and fills a 32,000-token cap. The job is made directly under a caller's deadline, and through a Temporal workflow whose one activity heartbeats. Each arm ran once, since the question is whether it finishes, and every arm is kept with its setup.
Beat the clock
The direct call failed at its 100-second deadline, and the workflow finished the job in 192 seconds.
| Arm | Client timeout | Outcome | Elapsed |
|---|---|---|---|
| A: direct | 100 s, the OpenAI SDK's default | failed | 100.0 s |
| A30: direct | 30 s, a usual HttpClient deadline | failed | 30.0 s |
| C: direct, the control | 30 minutes | finished, 32,000 tokens | 191.8 s |
| A2: direct, streamed | 100 s for each read | finished, 32,000 tokens | 187.4 s |
| B: one activity that heartbeats | 30 minutes, the activity's StartToClose | finished, 32,000 tokens, 20 heartbeats | 192.3 s |
Arm A made the call with the OpenAI SDK's client as it comes, which ended it at exactly
100 seconds: TaskCanceledException, "exceeded the configured timeout of 0:01:40".
LM Studio logged "Client disconnected. Stopping generation…" the same second, so the work
stopped on the GPU too, and the prefill and about 90 seconds of decode, some 16,000
tokens, were thrown away. At a 30-second deadline, the kind most HttpClients are given,
the same call failed at 30.0 s.
Through the workflow, the caller holds no connection to the model. The worker makes the call, the activity's StartToClose bounds it, and a heartbeat every 10 seconds tells Temporal the worker is alive, so a dead worker would be found within the 30-second heartbeat timeout rather than at the end of the job. Arm B finished on its first attempt, and returned the same 32,000 tokens as the direct control, byte for byte. It took half a second longer, which is within the run's own decode drift.
Streaming carried the direct call through too, because A2's timer restarts on every read and tokens arrived every 6 ms or so. It measures the gap between bytes, so a prefill longer than the timeout, which sends none, would fail it the same way.
Time's up
Inside an activity, the HTTP client's timeout has to cover the activity's.
The only timeout that fired in any arm was an HTTP client's. None of Temporal's own limits fired, the StartToClose, the heartbeat timeout and the retry policy included. Two more arms ran the activity under the extraction workflow's own options, a five-minute StartToClose, a 30-second heartbeat timeout and three attempts, with two different clients inside it.
| Arm | Client inside the activity | Outcome | Elapsed |
|---|---|---|---|
| B0 | 100 s, the SDK's default | failed: three attempts of 100.0 s | 315.1 s |
| B5 | 5 minutes, the activity's StartToClose | finished on its first attempt, 18 heartbeats | 172.7 s |
B0's StartToClose would have let the job finish. Its client ended each attempt at 100 seconds, Temporal retried after 5 and then 10 seconds as its policy says, and three attempts of GPU work went nowhere. B5 changed only the client, and finished. So the rule for a long job is one setting: the client inside the activity gets a timeout at or above the activity's StartToClose, and the activity's cancellation and its heartbeat own the deadline from there. A dev entry tells how this arm was first read the wrong way round.
B5's answer, 29,421 tokens ending on its own, isn't byte-identical to B's 32,000. LM Studio had been reloaded between the two runs, and the cause isn't established. The claim here is that the job finishes, which it did.
By the book
The job's length is fixed by its input, so it can't come up short.
The first three prompts asked for a length, and Lightning at temperature 0.2 writes short whatever it's asked. A 400-word digest of each of 95 sections became 11 digests in 75 seconds. With all 95 listed by number it covered them, at about 200 tokens each, in 103. Asked for four labelled parts of 150 words or more, it wrote less again, in 83. The fourth prompt asked it to copy each section in full and comment on it, and filled the 32,000-token cap in 191 seconds. All four are frozen with their hashes.
| Model | Nemotron 3.5 Lightning 30B-A3B, Unsloth UD-Q4_K_S, alone on LM Studio (runtime 2.46.0), RTX 5090 |
| Load | 65,536 context, one slot, flash attention off, f16 KV cache, seed fixed; warmup at 201.8 tokens a second |
| The job | long-duration-annotate.md, 111,059 bytes, 30,933 prompt tokens |
| Sampling | The extraction settings with a 32,000-token cap: temperature 0.2, top-p 0.95, top-k 0, min-p 0.01, repeat penalty 1.0, thinking off, seed 42 |
| Temporal | The dev server, at 127.0.0.1:7233; one request in flight on the machine, and LM Studio idle between arms |
| Order | A, C, B, B0, A2 in one load; A30 and B5 after a reload |
# the five arms, one after another, each against the same job
env 'ConnectionStrings__continuum-temporal=127.0.0.1:7233' dotnet run -c Release --project src/Apps/Continuum.Console -- \
benchmark long-duration --model nvidia-nemotron-3.5-lightning-30b-a3b \
--prompt-file src/Core/Utilities/Continuum.Benchmarks/Models/Prompts/long-duration-annotate.md \
--sampling "temperature=0.2,top_p=0.95,top_k=0,min_p=0.01,repeat_penalty=1,reasoning_effort=none,max_output_tokens=32000,seed=42" \
--deadline 00:30:00 --arms A,C,B,B0,A2 --label e1-lightning-64k