The nine-role team costs 3.5× more per verified task than the four-role one

I run eight teams of agents and compare them on cost per verified task: a task one role marked done and a different role reviewed and approved. I wanted to know whether a small team beats a large one and, if it does, whether the large one should be split.

The scoreboard

TABLE 1 · Mtok per verified task: millions of tokens per verified task. Mcache/turn: millions of cache-read tokens in an average turn. Verification coverage: share of the tasks marked done that got verified. *hotfix has one verified task in its lifetime, so its figure is noise. Management’s coverage could not be computed; deploy verified no task.

Lab is a test team on a toy project, with no repository. Landing maintains a static site, loyalty a loyalty program, platform the engine and tooling the other teams use, and product a point-of-sale system with a mobile client and a backend. Management writes documents, hotfix only switches on for production incidents, and deploy reviews and ships what the others build.

Product against landing gives 5.8×, but a point-of-sale system and a static site are different work, and the scoreboard can’t separate structure from difficulty. Platform is the closer comparison: another engineering team on large codebases. Against platform the ratio is 3.5×.

Where the cost comes from

TABLE 2 · Per-role cost drivers. In the four individual roles, each tool call reads about 70–110k cached tokens; in the lab team, about 48k.

Each tool call re-reads the cached prefix (instructions, tools, history). Cache reads are billed cheaper than fresh input, but they are billed every time, so a turn costs roughly its context times the number of tool calls it makes. A role that makes 51 calls in a turn pays for its context 51 times. Adding a role to a team costs whatever that role’s turns cost: at 51 calls per turn it is expensive, at 6 it is nearly free.

Why I’m not splitting the large team

Nine roles looked like too many, and two teams of four or five looked like the fix. But most of the product team’s cost sits in two roles that make 30–50 tool calls per turn against a large codebase. After a split they would make the same calls, against the same repositories and with the same context, plus the coordination between the two halves.

What I’d look at next

Verification coverage follows the cost ranking almost exactly: the teams at 100% are the cheap ones, and product, at 62%, is the expensive one. With this data I can’t tell which way it runs. Cheap work is easier to verify, and verified work needs less rework. It’s the next thing I want to run an experiment on.