The occupancy trigger compacts context without knowing how many calls are left

When an agent’s conversation gets close to the context limit, something has to replace the history with a shorter summary. That is compaction, and the usual trigger is occupancy: compact when the window reaches a certain fill level.

The arithmetic

On call i of a turn that will end up making N calls in total:

cost     C = fixed, 30–60k weighted tokens
             (the summarisation call, plus re-caching the new prefix)

saving   s = (ctx_before − ctx_after) × cache_read_weight
             ...earned once per call that remains

balance    = s·(N − i) − C

Weighted tokens count each token by how it is billed: writing to the cache costs 1.25× and reading from it 0.1×. So compaction pays if and only if:

N − i  >  C / s

On 15 Sept I measured 13 compactions call by call. A compaction paid for itself when what it saved over the following calls exceeded what it cost. With 17 or more calls left, 4 of 4 paid. With 9 or fewer, 7 of 7 didn’t. The two in between, with 11 and 16 calls left, landed within 6k weighted tokens of break-even, one on each side. With 13 points, 17 is approximate and specific to my system.

Why occupancy can’t decide it

The inequality needs N, and nothing in the turn says how much work is left. Occupancy tracks i, how far in the turn is, but not N − i, how far there is to go. A turn at 80% of its context may have thirty calls left or two, and the trigger does the same thing in both.

Estimating what’s left

N − i can be estimated from data the system already keeps: one row per turn, with the tool calls that turn made. The estimate is the residual mean lifetime:

m_role(i) = E[ N − i | N > i ]

If a role is already i calls in, how many does it have left on average. Actuaries use the same statistic for life expectancy at a given age. The rule becomes compact ⟺ m_role(i) > C/s, and both sides come out of the log.

TABLE 1 · Expected remaining tool calls once the turn has made i of them. n: the role’s turns with at least one tool call (3,346 across these 13 roles). Mean and p50: tool calls per turn. Bold: above the ~17 break-even.

For the heavy roles the estimate barely drops. A front-end dev thirty calls in still has 34.8 left on average, more than a light role makes in a whole turn. And it varies a lot by role: ten calls in, one expects 46.4 more and another 8.1. That’s why I estimate per role instead of using one threshold for everyone.

The rule isn’t applied yet: the trigger is still occupancy-based, and everything here comes from turns that already happened.

In production

By 17 Sept the production log had 80 compactions:

Only 3 in 10 paid, but one hit was worth about ten misses, so the net was positive. This log doesn’t record how many calls were left at each compaction, so it neither confirms nor refutes the 17-call crossover. Next I want to know which long-turn roles to switch it on for.