Dev Entries
Dev Entries \ blame_the_limit_that_fired

The only timeout that fired was the HTTP client's.

A 32,000-token job failed at 100 seconds as a direct call and finished in 192 through a Temporal workflow. Claude's first write-up then blamed Temporal for an arm that failed on an HTTP client's timeout, though none of Temporal's own limits had fired.

2026-09-26· 4 min read · Continuum Engine

Long story short

A long job needs its length fixed by its input.

Tristan

"One small set of tests should be A VERY LONG response with a huge context window where we can demonstrate TIMEOUT with LM Studio directly/vLLM, vs. the Temporal path where it can take as long as it likes to get a result. Success vs. failure test for long duration."

The OpenAI SDK gives each network operation 100 seconds unless it is told otherwise. Continuum's model clients take a NetworkTimeout per entry, and an entry without one keeps the SDK's 100 s. So the job had to run well past 100 s, and Lightning, loaded alone on LM Studio at 65,536 tokens of context, took four prompts to get there.

The prompt asked forLightning wroteTime
a 400-word digest of each of 95 sections13,102 tokens, in 11 digests75 s
the same, with all 95 listed by number19,039 tokens103 s
the same, in four labelled parts per digest14,671 tokens83 s
each section copied in full, then commented on32,000 tokens, the cap191 s

At temperature 0.2 the model answers a request for length tersely, however it's phrased. Copying its input fills the cap: a 30,933-token prompt of the engine's own host and app docs, and 32,000 tokens back.

Beat the clock

The direct call failed at 100.0 seconds, and the workflow finished the job in 192.

Every arm sent the same body, and the arms differed only in the client's network timeout, the variable under test.

ArmPathClient timeoutOutcome
Adirect100 s, the SDK's defaultfailed at 100.0 s
Cdirect, the control30 min191.8 s, 32,000 tokens
Bone activity, heartbeat on a timer30 min, the activity's StartToClose192.3 s on the first attempt, 20 heartbeats
B0the same activity, under the fact extractor's activity options100 s, the SDK's defaultfailed three times at 100.0 s, 315.1 s in all
A2direct, streamed100 s per read187.4 s

Arm A failed with the SDK's own message, "exceeded the configured timeout of 0:01:40", and LM Studio logged "Client disconnected. Stopping generation…" the same second, discarding the prefill and about 90 s of decode.

Arm B's activity heartbeats on a 10-second timer, since a non-streamed call sends nothing to beat on until the answer arrives. It sent 20, and returned the byte-identical answer to the unbounded control, 192.3 s against 191.8. A2 passed with the 100-second client because each streamed read restarts the timer, and tokens arrived every 6 ms or so.

Shoot the messenger

The first write-up blamed Temporal for the client's timeout.

B0 ran B's activity under the fact extractor's own activity options, a 5-minute StartToClose, a 30-second heartbeat timeout and three attempts, with the SDK's 100-second client inside it. It failed at exactly 100.0 s three times, with 5 and 10 seconds of backoff between. Claude's summary of the experiment read "Temporal alone doesn't save a long call; the activity has to be built for it".

Tristan

"Well, this sounds fundamentally like a logic mistake. We prove that Temporal wouldn't fail in this scenario, where you artificially kneecapped Temporal. The LM Studio call will fail when Temporal will succeed, if you don't time out the activity."

He was right, and the error was in the logic as well as the wording. In every arm, the only timeout that fired was an HTTP client's. The 5-minute StartToClose was never reached, the heartbeat timeout never tripped, and the retry policy did as it was told, three times.

B0 had carried arm A's direct-call deadline into the activity, where it ended each attempt 200 seconds before Temporal's own deadline could. The claim was corrected in the results, the findings and the journal the same hour.

Burden of proof

Two more arms proved the corrected claim.

Tristan

"Or set the timeout high enough to, let's say, 5 minutes."

Tristan

"HttpClient durations are usually max 15, 30, or 60."

By a direct caller's standards the SDK's 100 s is generous: Aspire's standard resilience handler allows 30 s in total. So at 15:00, with the model reloaded, two arms ran the same job. A30 called the model directly with a 30-second client. B5 ran the activity under the fact extractor's options, unchanged, with the client inside it set to their StartToClose.

// One entry per arm, differing only in the network timeout: the variable under test.
// A null timeout keeps the SDK's 100 seconds.
[LongDurationArms.A] = Client(LongDurationArms.A, null),
[LongDurationArms.A30] = Client(LongDurationArms.A30, TimeSpan.FromSeconds(30)),
[LongDurationArms.C] = Client(LongDurationArms.C, TimeSpan.FromMinutes(30)),
[LongDurationArms.B] = Client(LongDurationArms.B, deadline),
[LongDurationArms.B0] = Client(LongDurationArms.B0, null),
[LongDurationArms.B5] = Client(LongDurationArms.B5, FactExtractionWorkflow.ExtractOptions.StartToCloseTimeout),
[LongDurationArms.A2] = Client(LongDurationArms.A2, null),
ArmPathClient timeoutOutcome
A30direct30 sfailed at 30.0 s
B5one activity, the fact extractor's options unchanged5 min, their StartToClose172.7 s on the first attempt, 18 heartbeats

B5 differs from B0 in one value, the client's timeout inside the activity. At the StartToClose, it leaves Temporal owning the deadline and the heartbeat owning liveness, and the job finished on its first attempt with two minutes to spare.

The buck stops here

A failure belongs to the limit that fired.

The benchmark brief took two rules from the afternoon. A failure is attributed to the component whose timeout or check ended the run, named from its own exception, and nothing is concluded about the parts around it. A corrected claim is proved with a run as well as rewritten, which is what A30 and B5 are for.

The arms, the calibration prompts and the setup are on the long duration benchmark page.

← All Dev Entries