agentic-workspace live run — 2026-08-26

Model: qwen/qwen3-1.7b

Driver: in-process · Context length: 16384

Composite file: S3, S5 and S9 were re-run on fix commit addeac1; S1 is the original matrix row, recorded before it.

Scenario	Result	Checks	Tool calls	Rounds	Wall (s)	Tokens (prompt/predicted)
S1	✅	6/6	6	7	6.7	40806/2074
S3	❌	9/11	26	28	34.6	312094/9758
S5	✅	3/3	8	3	4.4	16186/1369
S9	❌	10/11	28	22	20.9	246653/6126

Failures

S3 — plan mode: propose, /accept, execute
- bug fixed in src/math.js — module.exports = { add: (a, b) => a + b + 1 }; |
- node --test passes — 0) | at Test.runInAsyncScope (node:async_hooks:214:14) | at Test.run (node:internal/test_runner/test:1106:25) | at Test.start (node:internal/test_runner/test:1003:17) | at startSubtestAfterBootstrap (node:internal/test_runner/harness:358:17) { | generatedMessage: true, | code: 'ERR_ASSERTION', | actual: 5, | expected: 4, | operator: 'strictEqual', | diff: 'simple' | } |
S9 — full project from an empty directory (context overflow: prompt 16461 > 16384)
- node --test exits 0 with at least three ok lines — exit 1, 0 ok lines | not ok 3 - tests\test-punctuation.js | --- | duration_ms: 30.3679 | type: 'test' | location: 'C:\Users\rootm\AppData\Local\Temp\live-s9-akXQtO\tests\test-punctuation.js:1:1' | failureType: 'testCodeFailure' | exitCode: 1 | signal: ~ | error: 'test failed' | code: 'ERR_TEST_FAILURE' | ... | 1..3 | # tests 3 | # suites 0 | # pass 0 | # fail 3 | # cancelled 0 | # skipped 0 | # todo 0 | # duration_ms 49.8157 |

⚠️ = the run reached the context ceiling. Where the row still passes, the scored work had already completed and been written to disk before the model ran out of context; a longer task at the same context length would be cut off mid-work. Pair the plugin with a context compressor, or raise the context length.

Pass = every check ok. Checks assert filesystem, git and .agentic state, never model prose. Results are comparable only within one model.

agentic-workspace live run — 2026-08-26

Model: qwen/qwen3-1.7b

Driver: in-process · Context length: 16384

Composite file: S3, S5 and S9 were re-run on fix commit addeac1; S1 is the original matrix row, recorded before it.

Scenario

Result

Checks

Tool calls

Rounds

Invalid

Envelope

Wall (s)

Tokens (prompt/predicted)

✅

6/6

6.7

40806/2074

❌

9/11

34.6

312094/9758

✅

3/3

4.4

16186/1369

❌

10/11

20.9

246653/6126

Failures

S3 — plan mode: propose, /accept, execute

bug fixed in src/math.js — module.exports = { add: (a, b) => a + b + 1 }; |
node --test passes — 0) | at Test.runInAsyncScope (node:async_hooks:214:14) | at Test.run (node:internal/test_runner/test:1106:25) | at Test.start (node:internal/test_runner/test:1003:17) | at startSubtestAfterBootstrap (node:internal/test_runner/harness:358:17) { | generatedMessage: true, | code: 'ERR_ASSERTION', | actual: 5, | expected: 4, | operator: 'strictEqual', | diff: 'simple' | } |

S9 — full project from an empty directory (context overflow: prompt 16461 > 16384)

node --test exits 0 with at least three ok lines — exit 1, 0 ok lines | not ok 3 - tests\test-punctuation.js | --- | duration_ms: 30.3679 | type: 'test' | location: 'C:\Users\rootm\AppData\Local\Temp\live-s9-akXQtO\tests\test-punctuation.js:1:1' | failureType: 'testCodeFailure' | exitCode: 1 | signal: ~ | error: 'test failed' | code: 'ERR_TEST_FAILURE' | ... | 1..3 | # tests 3 | # suites 0 | # pass 0 | # fail 3 | # cancelled 0 | # skipped 0 | # todo 0 | # duration_ms 49.8157 |

Pass = every check ok. Checks assert filesystem, git and .agentic state, never model prose. Results are comparable only within one model.