Three in four sandbox escapes I measured broke no configured rule

I maintain a sandbox, tartaros (mine), so I did not write these tests. I took the 168 contracts the lab had on 15 October 2026, all from the public test suites of sandbox-runtime, nono, Codex and bubblewrap, and ran the first three against them.

Each contract has an attack, a control with legitimate work that has to keep working, and an effect checked on disk afterwards. An attack only counts if it works with no sandbox.

Result	sandbox-runtime	nono	Codex
Contained	99	31	74
Escaped	22	33	24
Control broke	4	6	27
Cannot express it	18	78	18
Proves nothing	8	7	8
Not on Linux / missing in the runner	13	13	13

TABLE 1 · Linux 6.8, 2026-10-15. Some results are held while a coordinated-disclosure process is open and will be published when the vendor authorizes it. «Cannot express it» is a feature the sandbox lacks.

I also sent reports to the vendors’ security programs: 4 to sandbox-runtime (one still open) and 2 to Codex (one still open). An open report stays out of the published results until it is fixed.

Most escapes are defaults

Of the 79 escapes, 61 test something no configured rule asked for, such as protecting .git/hooks inside the folder the agent can write to. That is the sandbox as it ships. 13 are confirmed failures of a configured rule.

On a default On a configured rule

FIG. 1 · 61 of the 79 escapes are on the default side: the contract tests something no policy field asks for. The configured-rule segment adds escapes with a review note (5) and confirmed ones (13). One run, Linux 6.8, 2026-10-15.

Escape type	sandbox-runtime	nono	Codex	Total
trust-handoff	7	17	13	37
isolation	7	7	5	19
filesystem	1	5	3	9
secrets	2	0	1	3
network	5	0	0	5
lifecycle	0	3	2	5
ipc	0	1	0	1
Total	22	33	24	79

TABLE 2 · The 79 escapes by the lab’s category.

In a trust-handoff escape the agent writes a git hook or an editor config inside its folder, and your next git commit or your editor runs it outside the sandbox. Nine contracts escape in all three sandboxes; five of them are trust-handoff.

A «blocked» is not enough

Looking only at whether the attack failed, the three would score 111, 44 and 109 «blocked». In Codex, 27 of the 35 extra are broken controls: the attack failed and so did the legitimate work.

nono in its own model

The contracts assume everything is readable unless denied. nono works the other way: you grant the workspace and nothing else exists, so 62 of its 78 «cannot express it» say nothing about nono. In a second run I rewrote 22 of those attacks in its model without changing what they test. The rest have no equivalent.

The 22 variants	sandbox-runtime	nono	Codex
Contained	21	17	18
Escaped	0	4	0
Control broke	0	0	3
Proves nothing	1	1	1

TABLE 3 · Second run, same host. nono’s 4 escapes: unix sockets the kernel does not mediate, and grants resolved through symlinks.

After this run

landrun was added the same day: 17 of its 38 escapes are trust-handoff. tartaros (mine) was measured on 22 October 2026 against 188 contracts: 150 contained, 3 escaped. Since 26 October the corpus has 196 contracts. The contracts, the runner and the current results are in the lab.