agentic-workspace live run — 2026-08-26

Model: qwen3-14b-abliterated

Driver: in-process · Context length: 16384

Composite file: S9 was re-run on fix commit addeac1; S1, S3 and S5 are the original matrix rows, recorded before it.

Scenario	Result	Checks	Tool calls	Rounds	Wall (s)	Tokens (prompt/predicted)
S1	❌	1/6	1	2	8.5	8397/920
S3	✅	11/11	8	10	23.2	66803/2391
S5	✅	3/3	6	3	12.0	16076/1291
S9	❌	8/11	2	3	9.9	13897/1068

Failures

S1 — inspect → read → edit → verify
- bug fixed in src/math.js — module.exports = { add: (a, b) => a + b + 1 }; |
- node --test passes — 0) | at Test.runInAsyncScope (node:async_hooks:214:14) | at Test.run (node:internal/test_runner/test:1106:25) | at Test.start (node:internal/test_runner/test:1003:17) | at startSubtestAfterBootstrap (node:internal/test_runner/harness:358:17) { | generatedMessage: true, | code: 'ERR_ASSERTION', | actual: 5, | expected: 4, | operator: 'strictEqual', | diff: 'simple' | } |
- at least one applied transaction — 1 recorded calls
- inspected before editing — first inspect at -1, first edit at -1
- ran node --test through workspace_command — 0 run envelopes
S9 — full project from an empty directory
- at least one file under src/ — src/ is empty or missing
- at least one test file under tests/ — tests/ has no .js file
- node --test exits 0 with at least three ok lines — exit 0, 0 ok lines | TAP version 13 | 1..0 | # tests 0 | # suites 0 | # pass 0 | # fail 0 | # cancelled 0 | # skipped 0 | # todo 0 | # duration_ms 6.9814 |

Pass = every check ok. Checks assert filesystem, git and .agentic state, never model prose. Results are comparable only within one model.

Model: qwen3-14b-abliterated

Driver: in-process · Context length: 16384

Composite file: S9 was re-run on fix commit addeac1; S1, S3 and S5 are the original matrix rows, recorded before it.

Scenario	Result	Checks	Tool calls	Rounds	Wall (s)	Tokens (prompt/predicted)
S1	❌	1/6	1	2	8.5	8397/920
S3	✅	11/11	8	10	23.2	66803/2391
S5	✅	3/3	6	3	12.0	16076/1291
S9	❌	8/11	2	3	9.9	13897/1068

S1 — inspect → read → edit → verify
- bug fixed in src/math.js — module.exports = { add: (a, b) => a + b + 1 }; |
- node --test passes — 0) | at Test.runInAsyncScope (node:async_hooks:214:14) | at Test.run (node:internal/test_runner/test:1106:25) | at Test.start (node:internal/test_runner/test:1003:17) | at startSubtestAfterBootstrap (node:internal/test_runner/harness:358:17) { | generatedMessage: true, | code: 'ERR_ASSERTION', | actual: 5, | expected: 4, | operator: 'strictEqual', | diff: 'simple' | } |
- at least one applied transaction — 1 recorded calls
- inspected before editing — first inspect at -1, first edit at -1
- ran node --test through workspace_command — 0 run envelopes
S9 — full project from an empty directory
- at least one file under src/ — src/ is empty or missing
- at least one test file under tests/ — tests/ has no .js file
- node --test exits 0 with at least three ok lines — exit 0, 0 ok lines | TAP version 13 | 1..0 | # tests 0 | # suites 0 | # pass 0 | # fail 0 | # cancelled 0 | # skipped 0 | # todo 0 | # duration_ms 6.9814 |

Pass = every check ok. Checks assert filesystem, git and .agentic state, never model prose. Results are comparable only within one model.