The only timeout that fired was the HTTP client's.
A 32,000-token job failed at 100 seconds as a direct call and finished in 192 through a Temporal workflow. Claude's first write-up then blamed Temporal for an arm that failed on an HTTP client's timeout, though none of Temporal's own limits had fired.
Long story short
A long job needs its length fixed by its input.
Tristan
"One small set of tests should be A VERY LONG response with a huge context window where we can demonstrate TIMEOUT with LM Studio directly/vLLM, vs. the Temporal path where it can take as long as it likes to get a result. Success vs. failure test for long duration."
The OpenAI SDK gives each network operation 100 seconds unless it is told otherwise.
Continuum's model clients take a NetworkTimeout per entry, and an entry
without one keeps the SDK's 100 s. So the job had to run well past 100 s, and
Lightning, loaded alone on LM Studio at 65,536 tokens of context, took four prompts to
get there.
| The prompt asked for | Lightning wrote | Time |
|---|---|---|
| a 400-word digest of each of 95 sections | 13,102 tokens, in 11 digests | 75 s |
| the same, with all 95 listed by number | 19,039 tokens | 103 s |
| the same, in four labelled parts per digest | 14,671 tokens | 83 s |
| each section copied in full, then commented on | 32,000 tokens, the cap | 191 s |
At temperature 0.2 the model answers a request for length tersely, however it's phrased. Copying its input fills the cap: a 30,933-token prompt of the engine's own host and app docs, and 32,000 tokens back.
Beat the clock
The direct call failed at 100.0 seconds, and the workflow finished the job in 192.
Every arm sent the same body, and the arms differed only in the client's network timeout, the variable under test.
| Arm | Path | Client timeout | Outcome |
|---|---|---|---|
| A | direct | 100 s, the SDK's default | failed at 100.0 s |
| C | direct, the control | 30 min | 191.8 s, 32,000 tokens |
| B | one activity, heartbeat on a timer | 30 min, the activity's StartToClose | 192.3 s on the first attempt, 20 heartbeats |
| B0 | the same activity, under the fact extractor's activity options | 100 s, the SDK's default | failed three times at 100.0 s, 315.1 s in all |
| A2 | direct, streamed | 100 s per read | 187.4 s |
Arm A failed with the SDK's own message, "exceeded the configured timeout of 0:01:40", and LM Studio logged "Client disconnected. Stopping generation…" the same second, discarding the prefill and about 90 s of decode.
Arm B's activity heartbeats on a 10-second timer, since a non-streamed call sends nothing to beat on until the answer arrives. It sent 20, and returned the byte-identical answer to the unbounded control, 192.3 s against 191.8. A2 passed with the 100-second client because each streamed read restarts the timer, and tokens arrived every 6 ms or so.
Shoot the messenger
The first write-up blamed Temporal for the client's timeout.
B0 ran B's activity under the fact extractor's own activity options, a 5-minute StartToClose, a 30-second heartbeat timeout and three attempts, with the SDK's 100-second client inside it. It failed at exactly 100.0 s three times, with 5 and 10 seconds of backoff between. Claude's summary of the experiment read "Temporal alone doesn't save a long call; the activity has to be built for it".
Tristan
"Well, this sounds fundamentally like a logic mistake. We prove that Temporal wouldn't fail in this scenario, where you artificially kneecapped Temporal. The LM Studio call will fail when Temporal will succeed, if you don't time out the activity."
He was right, and the error was in the logic as well as the wording. In every arm, the only timeout that fired was an HTTP client's. The 5-minute StartToClose was never reached, the heartbeat timeout never tripped, and the retry policy did as it was told, three times.
B0 had carried arm A's direct-call deadline into the activity, where it ended each attempt 200 seconds before Temporal's own deadline could. The claim was corrected in the results, the findings and the journal the same hour.
Burden of proof
Two more arms proved the corrected claim.
Tristan
"Or set the timeout high enough to, let's say, 5 minutes."
Tristan
"HttpClient durations are usually max 15, 30, or 60."
By a direct caller's standards the SDK's 100 s is generous: Aspire's standard resilience handler allows 30 s in total. So at 15:00, with the model reloaded, two arms ran the same job. A30 called the model directly with a 30-second client. B5 ran the activity under the fact extractor's options, unchanged, with the client inside it set to their StartToClose.
// One entry per arm, differing only in the network timeout: the variable under test.
// A null timeout keeps the SDK's 100 seconds.
[LongDurationArms.A] = Client(LongDurationArms.A, null),
[LongDurationArms.A30] = Client(LongDurationArms.A30, TimeSpan.FromSeconds(30)),
[LongDurationArms.C] = Client(LongDurationArms.C, TimeSpan.FromMinutes(30)),
[LongDurationArms.B] = Client(LongDurationArms.B, deadline),
[LongDurationArms.B0] = Client(LongDurationArms.B0, null),
[LongDurationArms.B5] = Client(LongDurationArms.B5, FactExtractionWorkflow.ExtractOptions.StartToCloseTimeout),
[LongDurationArms.A2] = Client(LongDurationArms.A2, null),| Arm | Path | Client timeout | Outcome |
|---|---|---|---|
| A30 | direct | 30 s | failed at 30.0 s |
| B5 | one activity, the fact extractor's options unchanged | 5 min, their StartToClose | 172.7 s on the first attempt, 18 heartbeats |
B5 differs from B0 in one value, the client's timeout inside the activity. At the StartToClose, it leaves Temporal owning the deadline and the heartbeat owning liveness, and the job finished on its first attempt with two minutes to spare.
The buck stops here
A failure belongs to the limit that fired.
The benchmark brief took two rules from the afternoon. A failure is attributed to the component whose timeout or check ended the run, named from its own exception, and nothing is concluded about the parts around it. A corrected claim is proved with a run as well as rewritten, which is what A30 and B5 are for.
The arms, the calibration prompts and the setup are on the long duration benchmark page.