agentic-workspace
Public
RELEASE_VALIDATION.md
RELEASE_VALIDATION.md
What was actually run before this release, on the machine it was built on. Anything not listed here was not verified.
Host: Windows 11, Node 24, LM Studio with SDK 1.5.0. Every run below is native Windows; no WSL, container or CI.
npm run typecheck — TypeScript against the installed real @lmstudio/sdk 1.5.0 declarations: passed.npm run typecheck:offline — the same sources with @lmstudio/sdk, zod and puppeteer-core any-typed through dev-stubs/external.d.ts: passed.npm run check and npm run check:offline — 291 tests, 287 passed, 0 failed, 4 skipped. The four skips are POSIX-only: file modes, O_NOFOLLOW, and two path-alias cases that name nothing on Windows./accept and /deny handling, every one of the ten tools, the sub-agent orchestrator, command execution (allowlists, NUL rejection, output caps, background cancellation, timeout state, Windows .cmd shim resolution), task boards, research persistence, the web URL policy, the result envelope contract, and the documentation assertions in tests/docs.test.ts.The live harness (npm run live) drives the real tools with a real local model, one model loaded at a time, and asserts on filesystem, git and .agentic/ state — never on model prose. Result files under live/results/ are never hand-edited — each .md is generated from its .json — and the two files that combine rows from before and after a fix commit say so in their header.
issue_create, pr_create, push to a remote).Two items from this list at 0.3.0 have since been verified and moved above: end-to-end compaction with the context compressor in one chat, and installing from the Hub (the 0.3.0 revision was cloned from the Hub with lms clone, byte-compared to the release tree, and installed).
What was actually run before this release, on the machine it was built on. Anything not listed here was not verified.
Host: Windows 11, Node 24, LM Studio with SDK 1.5.0. Every run below is native Windows; no WSL, container or CI.
npm run typecheck — TypeScript against the installed real @lmstudio/sdk 1.5.0 declarations: passed.npm run typecheck:offline — the same sources with @lmstudio/sdk, zod and puppeteer-core any-typed through dev-stubs/external.d.ts: passed.npm run check and npm run check:offline — 291 tests, 287 passed, 0 failed, 4 skipped. The four skips are POSIX-only: file modes, O_NOFOLLOW, and two path-alias cases that name nothing on Windows./accept and /deny handling, every one of the ten tools, the sub-agent orchestrator, command execution (allowlists, NUL rejection, output caps, background cancellation, timeout state, Windows .cmd shim resolution), task boards, research persistence, the web URL policy, the result envelope contract, and the documentation assertions in tests/docs.test.ts.The live harness (npm run live) drives the real tools with a real local model, one model loaded at a time, and asserts on filesystem, git and .agentic/ state — never on model prose. Result files under live/results/ are never hand-edited — each .md is generated from its .json — and the two files that combine rows from before and after a fix commit say so in their header.
issue_create, pr_create, push to a remote).Two items from this list at 0.3.0 have since been verified and moved above: end-to-end compaction with the context compressor in one chat, and installing from the Hub (the 0.3.0 revision was cloned from the Hub with lms clone, byte-compared to the release tree, and installed).
qwen/qwen3.8-27b, in-process driver. The file's base context is 8192; S2, S3, S4, S7, S8 and S9 were run at 16384 and S6 at 12288, and each such row is labelled with its own context length. Results: live/results/2026-08-26-qwen-qwen3.8-27b.md.openai/gpt-oss-20b, google/gemma-4-26b-a4b-qat, qwen3-14b-abliterated, qwen/qwen3-1.7b, meta/llama-3.2-3b and glm-4.7-flash-uncensored-heretic-neo-code-imatrix-max. One result file each, summarised in the README's Model compatibility table.qwen/qwen3.8-27b: lms dev --install -y, then the SDK's pluginTools() against the installed copy with an LM Studio API token. Results: live/results/2026-08-26-qwen-qwen3.8-27b-installed.md. All 56 src/ files were verified sha256-identical to the installed copy first, and the session tool lists were checked through the SDK boundary: Auto-mode sessions expose eight tools with workspace_plan absent, the Plan-mode session nine with it present.
contextOverflowPolicy: stopAtLimit cannot shrink a prompt that is already larger than the window, so the fix is a larger context or a context compressor in the same chat.qwen/qwen3.8-27b at 16384: all pass (S1 6/6, S3 11/11, S5 3/3). Results: live/results/2026-08-26-qwen-qwen3.8-27b-2.md — the first same-day rerun, written under the new non-overwriting -2 suffix./compress and /accept executed it exactly; a Manual-mode staged approval (approval_mtah40ge…, the commit of src/subtract.js + its test) survived a second /compress, the model recalled the id, the operation and its task item from the compacted context without re-reading, /accept re-issued the exact commit (+14/−0, one-shot consumption held — including against the model claiming approval mid-turn, which only re-staged the same id), and the follow-up npm test job exited 0. All verified from .agentic/ state, the approvals store and the saved conversation. The 0.3.1 workspace-path fix was exercised live by the same chat after a mid-stream plugin update. Model quirk recorded for the compatibility notes: after long, heavily compacted sessions qwen3.8-27b occasionally prints a tool call as prose in another template's syntax with an invented tool name; no plugin emits or executes those, and the turn recovers.npm install, then npm run check.lms dev --install -y.workspace_inspect capabilities and confirm the expected modules and permission mode./accept, confirm it then runs.pending_approval with an id, and that /accept <id> makes the model re-issue exactly the resume call..agentic/transactions/ before and after a commit.report.md.qwen/qwen3.8-27b, in-process driver. The file's base context is 8192; S2, S3, S4, S7, S8 and S9 were run at 16384 and S6 at 12288, and each such row is labelled with its own context length. Results: live/results/2026-08-26-qwen-qwen3.8-27b.md.openai/gpt-oss-20b, google/gemma-4-26b-a4b-qat, qwen3-14b-abliterated, qwen/qwen3-1.7b, meta/llama-3.2-3b and glm-4.7-flash-uncensored-heretic-neo-code-imatrix-max. One result file each, summarised in the README's Model compatibility table.qwen/qwen3.8-27b: lms dev --install -y, then the SDK's pluginTools() against the installed copy with an LM Studio API token. Results: live/results/2026-08-26-qwen-qwen3.8-27b-installed.md. All 56 src/ files were verified sha256-identical to the installed copy first, and the session tool lists were checked through the SDK boundary: Auto-mode sessions expose eight tools with workspace_plan absent, the Plan-mode session nine with it present.
contextOverflowPolicy: stopAtLimit cannot shrink a prompt that is already larger than the window, so the fix is a larger context or a context compressor in the same chat.qwen/qwen3.8-27b at 16384: all pass (S1 6/6, S3 11/11, S5 3/3). Results: live/results/2026-08-26-qwen-qwen3.8-27b-2.md — the first same-day rerun, written under the new non-overwriting -2 suffix./compress and /accept executed it exactly; a Manual-mode staged approval (approval_mtah40ge…, the commit of src/subtract.js + its test) survived a second /compress, the model recalled the id, the operation and its task item from the compacted context without re-reading, /accept re-issued the exact commit (+14/−0, one-shot consumption held — including against the model claiming approval mid-turn, which only re-staged the same id), and the follow-up npm test job exited 0. All verified from .agentic/ state, the approvals store and the saved conversation. The 0.3.1 workspace-path fix was exercised live by the same chat after a mid-stream plugin update. Model quirk recorded for the compatibility notes: after long, heavily compacted sessions qwen3.8-27b occasionally prints a tool call as prose in another template's syntax with an invented tool name; no plugin emits or executes those, and the turn recovers.npm install, then npm run check.lms dev --install -y.workspace_inspect capabilities and confirm the expected modules and permission mode./accept, confirm it then runs.pending_approval with an id, and that /accept <id> makes the model re-issue exactly the resume call..agentic/transactions/ before and after a commit.report.md.