agentic-workspace live run — 2026-08-26

Model: meta/llama-3.2-3b

Driver: in-process · Context length: 16384

Scenario	Result	Checks	Tool calls	Rounds	Invalid	Wall (s)	Tokens (prompt/predicted)
S1	❌	2/6	11	14	2	2.8	78578/571
S3	❌	6/11	0	1	1	1.9	—
S5	❌	0/3	0	1	1	1.3	—
S9	❌	7/11	0	1	1	5.0	—

Failures

S1 — inspect → read → edit → verify
- bug fixed in src/math.js — module.exports = { add: (a, b) => a + b + 1 }; |
- node --test passes — 0) | at Test.runInAsyncScope (node:async_hooks:214:14) | at Test.run (node:internal/test_runner/test:1106:25) | at Test.start (node:internal/test_runner/test:1003:17) | at startSubtestAfterBootstrap (node:internal/test_runner/harness:358:17) { | generatedMessage: true, | code: 'ERR_ASSERTION', | actual: 5, | expected: 4, | operator: 'strictEqual', | diff: 'simple' | } |
- at least one applied transaction — 11 recorded calls
- ran node --test through workspace_command — 3 run envelopes
S3 — plan mode: propose, /accept, execute (Error: Engine protocol predict stream returned an error:)
- no mutation before approval — src/math.js was ""
- plan proposed in phase 1 — 0 plan.propose envelopes before /accept
- exactly one plan approved — no plan records
- bug fixed in src/math.js — module.exports = { add: (a, b) => a + b + 1 }; |
- node --test passes — 0) | at Test.runInAsyncScope (node:async_hooks:214:14) | at Test.run (node:internal/test_runner/test:1106:25) | at Test.start (node:internal/test_runner/test:1003:17) | at startSubtestAfterBootstrap (node:internal/test_runner/harness:358:17) { | generatedMessage: true, | code: 'ERR_ASSERTION', | actual: 5, | expected: 4, | operator: 'strictEqual', | diff: 'simple' | } |
S5 — foreground command, background job, cancel (Error: Engine protocol predict stream returned an error:)
- foreground run reported an exit code — 0 run envelopes
- background job started — no job id
- job was canceled — 0 cancel envelopes; final status ""
S9 — full project from an empty directory (Error: Engine protocol predict stream returned an error:)
- package.json parses with scripts.test running node --test — no package.json
- at least one file under src/ — src/ is empty or missing
- at least one test file under tests/ — tests/ has no .js file
- node --test exits 0 with at least three ok lines — exit 0, 0 ok lines | TAP version 13 | 1..0 | # tests 0 | # suites 0 | # pass 0 | # fail 0 | # cancelled 0 | # skipped 0 | # todo 0 | # duration_ms 5.3067 |

Pass = every check ok. Checks assert filesystem, git and .agentic state, never model prose. Results are comparable only within one model.

Scenario

Result

Checks

Tool calls

Rounds

Invalid

Envelope

Wall (s)

Tokens (prompt/predicted)

❌

2/6

2.8

78578/571

❌

6/11

1.9

—

❌

0/3

1.3

—

❌

7/11

5.0

—

Failures

S1 — inspect → read → edit → verify

bug fixed in src/math.js — module.exports = { add: (a, b) => a + b + 1 }; |
node --test passes — 0) | at Test.runInAsyncScope (node:async_hooks:214:14) | at Test.run (node:internal/test_runner/test:1106:25) | at Test.start (node:internal/test_runner/test:1003:17) | at startSubtestAfterBootstrap (node:internal/test_runner/harness:358:17) { | generatedMessage: true, | code: 'ERR_ASSERTION', | actual: 5, | expected: 4, | operator: 'strictEqual', | diff: 'simple' | } |
at least one applied transaction — 11 recorded calls
ran node --test through workspace_command — 3 run envelopes

S3 — plan mode: propose, /accept, execute (Error: Engine protocol predict stream returned an error:)

no mutation before approval — src/math.js was ""
plan proposed in phase 1 — 0 plan.propose envelopes before /accept
exactly one plan approved — no plan records
bug fixed in src/math.js — module.exports = { add: (a, b) => a + b + 1 }; |
node --test passes — 0) | at Test.runInAsyncScope (node:async_hooks:214:14) | at Test.run (node:internal/test_runner/test:1106:25) | at Test.start (node:internal/test_runner/test:1003:17) | at startSubtestAfterBootstrap (node:internal/test_runner/harness:358:17) { | generatedMessage: true, | code: 'ERR_ASSERTION', | actual: 5, | expected: 4, | operator: 'strictEqual', | diff: 'simple' | } |

S5 — foreground command, background job, cancel (Error: Engine protocol predict stream returned an error:)

foreground run reported an exit code — 0 run envelopes
background job started — no job id
job was canceled — 0 cancel envelopes; final status ""

S9 — full project from an empty directory (Error: Engine protocol predict stream returned an error:)

package.json parses with scripts.test running node --test — no package.json
at least one file under src/ — src/ is empty or missing
at least one test file under tests/ — tests/ has no .js file
node --test exits 0 with at least three ok lines — exit 0, 0 ok lines | TAP version 13 | 1..0 | # tests 0 | # suites 0 | # pass 0 | # fail 0 | # cancelled 0 | # skipped 0 | # todo 0 | # duration_ms 5.3067 |

Pass = every check ok. Checks assert filesystem, git and .agentic state, never model prose. Results are comparable only within one model.