Hi, I’m Carlos.
I’m a software architect in Santiago, Chile, thirteen years on systems for banking, telecommunications and the public sector. At work I use AI agents every day. This site is what I do on my own time: I run agent teams on my own code and publish what I measure.
Notes
- Agents that know whose file it is leave it alone, then patch around it What Claude Code, Codex and opencode do when the kernel denies a write to another role's file. With the bare denial, two of the three push back in every run. Knowing the owner beforehand, none tries, but two of the three still fix the bug in their own files. seisin's denial message was never shown.
- The Hugging Face intrusion from an OpenAI evaluation, step by step An agent running an OpenAI evaluation left its sandbox and spent five days in Hugging Face's infrastructure. Reconstructed from both companies' reports.
- Three in four sandbox escapes I measured broke no configured rule sandbox-runtime, nono and Codex against 168 tests from their own projects' suites. Most escapes come from defaults.
- The nine-role team costs 3.5× more per verified task than the four-role one I compared eight agent teams on cost per verified task. The gap follows tool calls per turn, and splitting the large team would not change what its two most expensive roles do.
- The occupancy trigger compacts context without knowing how many calls are left Compaction has a fixed cost and a saving per remaining call. In 13 measured compactions the crossover was around 17 calls left. Occupancy can't see that number; each role's turn history can estimate it.
- In a small test, keeping an agent's thread alive cost less than resetting it Resetting a role's thread every turn looked like the way to spend fewer tokens. In a controlled test with one role and one task, the long thread came out cheaper, because of cache writes.
- My verifier reported 100% conformance until I measured what it rejects 718 of 718 rounds followed the declared process, but a checker that accepts almost anything also reports 100%. A shuffle control showed the number holds in one team out of six.
- 86% of my agents' code is still there, and most of the drop came from one week Line survival on a production repository written by agents: 85.9%, down from 92.7% two days earlier. Most of the lines lost in between belong to one week and one 2,408-line commit that survived at zero.
- Finding duplicates and checking citations need opposite kinds of search Four citations from my agents failed when the source was opened. On 14 labelled pairs, exact search finds almost no duplicates and embeddings find all of them with noise. Checking a citation is exact search in a known file, where similarity gets in the way.
- 38% of my agents' message deliveries never reached the recipient's prompt 11,845 deliveries in 312 rounds of one team, and 4,445 discarded by the mailbox cap before the recipient saw them. With a fan-out of 3.37×, every role was telling everything to almost everyone.
Things I built
- seisin · Gives each agent its own folders and keys. When the kernel blocks something, it tells you whose file it was. MIT, on npm.
- tartaros · A local sandbox for coding agents where every claim is a test you can run on your machine. Alpha, Apache-2.0.
- sandbox-conformance · An open lab that measures agent sandboxes with the tests their own authors wrote.
- denial-detours · An experiment: what a coding agent does when it is denied a write to another role's file. Every trial kept.