agentic-workspace
Public
live / results / 2026-08-26-qwen-qwen3.8-27b.md
live / results / 2026-08-26-qwen-qwen3.8-27b.md
Model: qwen/qwen3.8-27b
Driver: in-process · Context length: 8192
| Scenario | Result | Checks | Tool calls | Rounds | Invalid | Envelope | Wall (s) | Tokens (prompt/predicted) |
|---|---|---|---|---|---|---|---|---|
| S1 | ✅ | 6/6 | 7 | 5 | 0 | 0 | 14.5 | 26628/1101 |
| S2 (ctx 16384) | ✅ | 10/10 | 12 | 9 | 0 | 0 | 29.8 | 65378/2352 |
| S3 (ctx 16384) | ✅ | 11/11 | 10 | 9 | 0 | 0 | 29.4 | 73102/2288 |
| S4 (ctx 16384) | ✅ | 11/11 | 14 | 12 | 0 | 0 | 33.2 | 108881/2671 |
| S5 | ✅ | 3/3 | 6 | 5 | 0 | 0 | 23.1 | 28953/1773 |
| S6 | ❌ | 2/5 | 4 | 3 | 0 | 0 | 44.6 | 11113/3197 |
| S6 (ctx 12288) | ✅ | 5/5 | 11 | 10 | 0 | 0 | 42.3 | 67527/3018 |
| S7 (ctx 16384) | ✅ | 7/7 | 10 | 11 | 0 | 0 | 53.7 | 106436/4141 |
| S8 (ctx 16384) | ✅ | 11/11 | 7 | 5 | 0 | 0 | 52.9 | 30831/1728 |
| S9 (ctx 16384) | ✅⚠️ | 11/11 | 12 | 10 | 0 | 0 | 49.3 | 101472/4046 |
⚠️ = the run reached the context ceiling. Where the row still passes, the scored work had already completed and been written to disk before the model ran out of context; a longer task at the same context length would be cut off mid-work. Pair the plugin with a context compressor, or raise the context length.
Pass = every check ok. Checks assert filesystem, git and .agentic state, never model prose. Results are comparable only within one model.
Model: qwen/qwen3.8-27b
Driver: in-process · Context length: 8192
| Scenario | Result | Checks | Tool calls | Rounds | Invalid | Envelope | Wall (s) | Tokens (prompt/predicted) |
|---|---|---|---|---|---|---|---|---|
| S1 | ✅ | 6/6 | 7 | 5 | 0 | 0 | 14.5 | 26628/1101 |
| S2 (ctx 16384) | ✅ | 10/10 | 12 | 9 | 0 | 0 | 29.8 | 65378/2352 |
| S3 (ctx 16384) | ✅ | 11/11 | 10 | 9 | 0 | 0 | 29.4 | 73102/2288 |
| S4 (ctx 16384) | ✅ | 11/11 | 14 | 12 | 0 | 0 | 33.2 | 108881/2671 |
| S5 | ✅ | 3/3 | 6 | 5 | 0 | 0 | 23.1 | 28953/1773 |
| S6 | ❌ | 2/5 | 4 | 3 | 0 | 0 | 44.6 | 11113/3197 |
| S6 (ctx 12288) | ✅ | 5/5 | 11 | 10 | 0 | 0 | 42.3 | 67527/3018 |
| S7 (ctx 16384) | ✅ | 7/7 | 10 | 11 | 0 | 0 | 53.7 | 106436/4141 |
| S8 (ctx 16384) | ✅ | 11/11 | 7 | 5 | 0 | 0 | 52.9 | 30831/1728 |
| S9 (ctx 16384) | ✅⚠️ | 11/11 | 12 | 10 | 0 | 0 | 49.3 | 101472/4046 |
⚠️ = the run reached the context ceiling. Where the row still passes, the scored work had already completed and been written to disk before the model ran out of context; a longer task at the same context length would be cut off mid-work. Pair the plugin with a context compressor, or raise the context length.
Pass = every check ok. Checks assert filesystem, git and .agentic state, never model prose. Results are comparable only within one model.