Carlos Tapia
← All notes

In a small test, keeping an agent's thread alive cost less than resetting it

· 4 min read

The thread is the message history a role carries from one turn to the next. Recycling it means starting the next turn from scratch, with less context and, on the face of it, cheaper.

Across production roles, the floor of a turn (the tokens the first call already carries) tracked the recycling interval:

Recycling intervalMean floor (tokens)
Every turn50,104
Every 5 turns110,034

TABLE 1 · Observed across production roles, nothing controlled.

That suggests resetting often halves the floor. But the roles that keep long threads are also the ones with the biggest jobs, so the table cannot separate the two.

The experiment

One role, one model and one trivial task, in an isolated test team. The only change was the recycling interval: variant A reset the thread every turn, B every five. Of eight rounds I discarded A’s first (round 5), which ran right after an engine change, so it is 3 rounds against 4.

VariantRoundTool callsCache readCache writtenWeighted window
A · reset every turn65104,02021,28450,013
A · reset every turn75104,32520,90547,532
A · reset every turn86105,10121,43550,257
B · reset every 594104,08420,88046,409
B · reset every 5104141,9828,34840,569
B · reset every 5113107,9335,31426,029
B · reset every 5124182,2156,75934,486

TABLE 2 · “Weighted window” is the turn’s cost in equivalent tokens: cache writes at 1.25×, cache reads at 0.1×, as the provider bills them.

The mean weighted window was 49,267 for A and 36,873 for B, 25% lower. B also averaged 3.8 tool calls per turn against 5.3.

Cache rebuilt per turn, by variant

Lower is cheaper. A cache write is billed at 1.25× a read.

Reset every turn Reset every 5 turns
0 6k 12k 18k 24k Reset every turn — 21,284 21.3k Reset every 5 turns — 20,880 20.9k 1st turn Reset every turn — 20,905 20.9k Reset every 5 turns — 8,348 8.3k 2nd turn Reset every turn — 21,435 21.4k Reset every 5 turns — 5,314 5.3k 3rd turn Reset every 5 turns — 6,759 6.8k 4th turn TOKENS WRITTEN

FIG. 1 · Variant A rebuilds the whole cached prefix every turn. Variant B pays once, then reuses it.

Cache writes

With a reset every turn, the role rewrites its whole cached prefix (instructions, tools, history) every time: about 21,000 tokens in each of A’s rounds, billed at the premium rate. B’s first round does the same because its thread is new. From the second round on, writes drop to 5,000–8,000 tokens. Cache reads go up (141k, 182k), but they are the cheap part.

So a short thread does lower the floor, and pays for it by rebuilding the cache every turn. Reads keep growing while the thread lives, so a longer thread is not free either; I did not measure where the two costs cross.

Why B needed fewer tool calls I don’t know. My hypothesis is that it did not have to re-read files it still had in context.

Next

The test role stayed on B. Next is sweeping the interval from 1 to 10 with the same role, then repeating it with a role that actually uses tools, which accumulates history faster. Production roles change after that, one at a time.

How this was measured. Eight rounds in an isolated test team with no production traffic. Same role, same model, same trivial task; the only variable was the thread-recycling interval. Thread reset between variants so the second could not inherit the first's cache. One round of variant A discarded, so the comparison is 3 rounds against 4.

Related