Open harness benchmark

Containment, measured.

We ran the exact same gauntlet: eight probes, identical guest scripts, six sandboxes, and two bare controls. The result? Lineation was the only one to block every single probe. And the performance hit for that ironclad security? Just a one-time compile. By the second start, it’s running just as fast as any standard sandbox.

  • 8 probes
  • 6 sandboxes + 2 controls
  • 7 defense layers
  • Firecracker included

The field

Containment cells

The options with a solid border have a real isolation boundary, otherwise the guest shares the host process.

Walled

Lineation Sandbox

Walled

Lineation + daemon

Walled

Docker

Walled

OpenShell

Walled

Deno

Walled

E2B (Firecracker)

Bare control

Node

Bare control

Rust

Under fire

Efficacy, not feature claims

If the observed outcome matched the expected containment result, it is marked as pass. The feature-layer totals sit further down. OpenShell leads that axis; Lineation leads this one.

Pass Fail Unsupported Not run
Containment outcomes by backend and scenario
Probe Lineation Lineation+d Docker OpenShell Deno E2B Node Rust
Happy path P P P P P P P P
CPU / timeout P P F P U P F F
Memory limit P P F U U F F F
Filesystem escape P P P P F† P F F
Network exfil P P P P F† F F F
Tool deny P P U U U U F F
Cold start P P P P P P P P
Isolation reuse P P P P F† P F F
  1. Lineation (CLI and daemon paths) passed all eight probes. Bare Node and Rust failed six: timeout, memory, filesystem, network, tool deny, and isolation reuse.
  2. Docker contained filesystem, network, and reuse, but failed timeout and memory in this setup. Tool deny is unsupported on plain Docker.
  3. † Deno filesystem, network, and reuse failures were guest NotCapable env-permission errors, not clean escape proofs. Timeout and memory are unsupported-permission flags are not cgroups.
  4. OpenShell passed six probes. Memory limit and tool deny are unsupported in this CLI mapping, not necessarily product gaps.
  5. E2B passed five probes. It failed memory limit (guest exit 137) and network exfil (egress allowed). Tool deny is unsupported-no semantic tool allowlist in this harness.

Taxonomy · 0–4 per layer

Defense layers

Static feature scores from each adapter-what the product claims to enforce, not what the probes observed. OpenShell scores 24/28 here; Lineation scores 20–21/28. Do not read this chart as the efficacy winner.

L1 Compute isolation

Lineation
3
Lineation+d
3
E2B
4
OpenShell
3
Docker / Deno
2

Lineation L2–L7 (taxonomy fill)

L2 Resources
3
L3 Filesystem
4
L4 Network
3
L5 Secrets
2
L6 Actions
3
L7 Audit
2

Lineation sandbox L7 is 2; the daemon path is 3. OpenShell is 4 on filesystem, network, secrets, and action governance in this taxonomy fill.

Cold start vs second start · log scale

The first compile is the bill. The second start is not.

cold_start is create plus first execute. echo is the warm / second-start proxy after the harness has prepared the backend. Lineation is not the fastest cold start-bare Rust reports 2 ms; Firecracker cold is 101 ms. The useful fact is the drop: about 3 s once, then about 69 ms.

Cold start Second start (echo)
Rust control
2 ms
2 ms
Deno
24 ms
39 ms
Node control
28 ms
31 ms
OpenShell
57 ms
63 ms
E2B
101 ms
249 ms
Docker
182 ms
189 ms
Lineation Sandbox
2.96 s
58 ms
Lineation + daemon
3.14 s
61 ms

Lineation Sandbox: 2.96 s cold → 58 ms on the second start. Daemon path: 3.14 s → 61 ms. After the component is compiled, repeat execute is faster than E2B's 249 ms echo-not next to the 3-second compile.

8 / 8 Lineation probes tested in this harness
2 / 8 Bare Node and Rust controls
~43× Cold compile vs warm echo on Lineation

How to read this

Methodology

What was measured

An open harness comparing agent sandboxes on two axes: a feature scorecard (L1–L7, strength 0–4) and eight efficacy scenarios. Status values are pass, fail, unsupported, or skipped. The harness does not ship escape exploit payloads-containment outcomes only.

What we are not claiming

  • Not the fastest cold start.
  • Not the highest feature-layer total. OpenShell leads that scorecard; Firecracker leads L1 (microVM).
  • Not “industry best.” One dated run on one host, with the caveats above.

What the data does support: in this harness, only Lineation held all eight probes; E2B failed memory and egress. After the first compile the second start is about 58 ms.

Draw the line around untrusted execution.

See the interactive attack demo, or put runtime policy on your own agents.