agentic-workspace
Public
live / results / 2026-08-26-qwen-qwen3.8-27b-installed.md
live / results / 2026-08-26-qwen-qwen3.8-27b-installed.md
Model: qwen/qwen3.8-27b
Driver: installed · Context length: 16384
Config verification: the installed session's tool list proves each scenario's pluginConfig reached the plugin —
8 tools with no workspace_plan in S1 and S9 (permissionMode 2, auto) and 9 tools including it in S3 (permissionMode 1, plan).
S9 ended on a hard prompt overflow: once the accumulated prompt exceeds the window LM Studio's engine returns 400,
which contextOverflowPolicy: "stopAtLimit" cannot prevent (it stops generation, it cannot shrink an already-oversized prompt).
All 11 checks still passed — wordcount was built and node --test-green on disk before the overflow.
Two in-process control runs at the same context length reproduced the same hard overflow (prompt 18337 and 16629 > 16384, 4/4 scored both times),
so this is S9 sitting at the edge of a 16384-token window for this model, not something the installed/SDK boundary introduces;
the committed in-process S9 row that passed with a warning was a sample that happened to land under the ceiling.
S1 and S3 match the in-process rows check for check, and S3 matches its tool-call profile exactly.
| Scenario | Result | Checks | Tool calls | Rounds | Invalid | Envelope | Wall (s) | Tokens (prompt/predicted) |
|---|---|---|---|---|---|---|---|---|
| S1 | ✅ | 6/6 | 12 | 9 | 0 | 0 | 26.6 | 65008/2074 |
| S3 | ✅ | 11/11 | 10 | 8 | 0 | 0 | 24.5 | 60406/1895 |
| S9 | ❌ | 11/11 | 12 | 9 | 1 | 0 | 44.0 | 73281/3812 |
⚠️ = the run reached the context ceiling. Where the row still passes, the scored work had already completed and been written to disk before the model ran out of context; a longer task at the same context length would be cut off mid-work. Pair the plugin with a context compressor, or raise the context length.
Pass = every check ok. Checks assert filesystem, git and .agentic state, never model prose. Results are comparable only within one model.
Model: qwen/qwen3.8-27b
Driver: installed · Context length: 16384
Config verification: the installed session's tool list proves each scenario's pluginConfig reached the plugin —
8 tools with no workspace_plan in S1 and S9 (permissionMode 2, auto) and 9 tools including it in S3 (permissionMode 1, plan).
S9 ended on a hard prompt overflow: once the accumulated prompt exceeds the window LM Studio's engine returns 400,
which contextOverflowPolicy: "stopAtLimit" cannot prevent (it stops generation, it cannot shrink an already-oversized prompt).
All 11 checks still passed — wordcount was built and node --test-green on disk before the overflow.
Two in-process control runs at the same context length reproduced the same hard overflow (prompt 18337 and 16629 > 16384, 4/4 scored both times),
so this is S9 sitting at the edge of a 16384-token window for this model, not something the installed/SDK boundary introduces;
the committed in-process S9 row that passed with a warning was a sample that happened to land under the ceiling.
S1 and S3 match the in-process rows check for check, and S3 matches its tool-call profile exactly.
| Scenario | Result | Checks | Tool calls | Rounds | Invalid | Envelope | Wall (s) | Tokens (prompt/predicted) |
|---|---|---|---|---|---|---|---|---|
| S1 | ✅ | 6/6 | 12 | 9 | 0 | 0 | 26.6 | 65008/2074 |
| S3 | ✅ | 11/11 | 10 | 8 | 0 | 0 | 24.5 | 60406/1895 |
| S9 | ❌ | 11/11 | 12 | 9 | 1 | 0 | 44.0 | 73281/3812 |
⚠️ = the run reached the context ceiling. Where the row still passes, the scored work had already completed and been written to disk before the model ran out of context; a longer task at the same context length would be cut off mid-work. Pair the plugin with a context compressor, or raise the context length.
Pass = every check ok. Checks assert filesystem, git and .agentic state, never model prose. Results are comparable only within one model.