live / results / 2026-08-26-qwen-qwen3.8-27b-2.md
live / results / 2026-08-26-qwen-qwen3.8-27b-2.md
Model: qwen/qwen3.8-27b
Driver: in-process ยท Context length: 16384
| Scenario | Result | Checks | Tool calls | Rounds | Invalid | Envelope | Wall (s) | Tokens (prompt/predicted) |
|---|---|---|---|---|---|---|---|---|
| S1 | โ | 6/6 | 8 | 6 | 0 | 0 | 18.2 | 36997/1377 |
| S3 | โ | 11/11 | 9 | 8 | 0 | 0 | 23.1 | 61491/1810 |
| S5 | โ | 3/3 | 7 | 5 | 0 | 0 | 22.6 | 31204/1596 |
Pass = every check ok. Checks assert filesystem, git and .agentic state, never model prose. Results are comparable only within one model.
Model: qwen/qwen3.8-27b
Driver: in-process ยท Context length: 16384
| Scenario | Result | Checks | Tool calls | Rounds | Invalid | Envelope | Wall (s) | Tokens (prompt/predicted) |
|---|---|---|---|---|---|---|---|---|
| S1 | โ | 6/6 | 8 | 6 | 0 | 0 | 18.2 | 36997/1377 |
| S3 | โ | 11/11 | 9 | 8 | 0 | 0 | 23.1 | 61491/1810 |
| S5 | โ | 3/3 | 7 | 5 | 0 | 0 | 22.6 | 31204/1596 |
Pass = every check ok. Checks assert filesystem, git and .agentic state, never model prose. Results are comparable only within one model.