Fast-trigger settings for a fresh chat: Absolute token trigger 3000, keep-recent
1500, chunk size 2000. Restore defaults (0 / 6000 / 16000) when done.
Seed prompt
Paste as the first message:
You are helping me test a context-compression plugin. Remember these facts
precisely — I will quiz you on them much later, after your context has been
compressed:
The project codename is VELVET-OTTER-19.
The database password hint is "quantum-pistachio-7788".
We decided to use PostgreSQL over MongoDB because of transactional integrity.
My favorite constant is e = 2.71828.
The deadline is March 14th and the budget is $47,250.
Now, to fill up your context quickly: write a detailed 800-word technical
explanation of how a compiler works, then list 40 programming languages with
one-sentence descriptions of each.
Sequence
Filler ("600 words on OS schedulers") until auto-compact fires → progress bar,
normal reply, meter line.
/usage → instant report, no reply; over-trigger note when applicable.
Quiz all five seeded facts → must survive compaction (core fidelity test).
"How much context do I have left?" → model calls get_context_usage.
/compact → stats block; /compact again → "Nothing to compress."
Abort mid-compaction → clean stop; next message resumes from cached chunks.
Tools: create/read a file through compaction; then a failing command → model
sees the error and reacts (no "Prediction failed").
Marathon tool loop (10 steps, no user turns) → no "No user query found".
Edit an old message, regenerate → clean recompaction.
Mid-task boundary tests (side-effect safety)
Mid-task compaction must preserve exactly-once semantics for tools:
Ask for a 10+ step tool task where step N writes a file right before a
pass boundary (rounds-per-pass 2 makes boundaries frequent) → after the
continuation, the model must NOT redo the write; the file's content
reflects exactly one write.
Cancel immediately after a tool completes near a boundary → resume; the
completed tool's result must still be visible to the model.
A failing tool on the last round of a pass → the error is fed back and
the continuation reacts to it (no repeat of a destructive attempt).
30+ sequential tool calls with aggregate output cap low (e.g. 2000) →
later results clamp with the aggregate marker; mid-task compaction
fires; the task still completes.
Restart LM Studio mid-marathon (after a mid-task compaction) → the next
message resumes with cached chunks and correct task state.
Consolidation tests (bounded summaries)
Set "Summary budget" low (e.g. 4000) in a chat with 4+ summary chunks →
next /compress shows "Consolidating N summaries into one…", the stats
note "incl. 1 consolidated from N earlier chunks", and /usage's chunk
count drops while coverage stays.
Fact recall after consolidation: quiz the TESTING seed facts → 5/5.
Repeated growth: keep chatting past the budget again → a second
consolidation folds [L1 + newer chunks] without touching the newest ones;
prompt returns to the floor each time.
Floor sanity: set "Compact down to" below what the budget + kept-verbatim
allow → one-time "floor is not reachable" warning appears and compaction
targets the clamped floor instead.
Attachment memory tests
Attach a .txt containing a distinctive fact (e.g. "the vault code is
OTTER-771"), chat past the compaction trigger so the attachment's region
compresses ("Preserving attachment…" status appears) → quiz the model on
the fact → recalled from the summary.
Same with an image on a vision model → the image gets described; quiz on
visible content. On a non-vision model → chunk summary shows
"content not retained", no crash.
Abort mid-"Preserving attachment…" → nothing persisted; next compaction
retries the extraction.
Restart LM Studio, compact a different chat referencing the same file →
extraction is NOT repeated (cached by identifier).
Agentic Workspace protocol tests
Model matrix (32-40)
Testing context-compressor
Fast-trigger settings for a fresh chat: Absolute token trigger 3000, keep-recent
1500, chunk size 2000. Restore defaults (0 / 6000 / 16000) when done.
Seed prompt
Paste as the first message:
You are helping me test a context-compression plugin. Remember these facts
precisely — I will quiz you on them much later, after your context has been
compressed:
The project codename is VELVET-OTTER-19.
The database password hint is "quantum-pistachio-7788".
We decided to use PostgreSQL over MongoDB because of transactional integrity.
My favorite constant is e = 2.71828.
The deadline is March 14th and the budget is $47,250.
Now, to fill up your context quickly: write a detailed 800-word technical
explanation of how a compiler works, then list 40 programming languages with
one-sentence descriptions of each.
Sequence
Filler ("600 words on OS schedulers") until auto-compact fires → progress bar,
normal reply, meter line.
/usage → instant report, no reply; over-trigger note when applicable.
Quiz all five seeded facts → must survive compaction (core fidelity test).
"How much context do I have left?" → model calls get_context_usage.
/compact → stats block; /compact again → "Nothing to compress."
Abort mid-compaction → clean stop; next message resumes from cached chunks.
Tools: create/read a file through compaction; then a failing command → model
sees the error and reacts (no "Prediction failed").
Marathon tool loop (10 steps, no user turns) → no "No user query found".
Edit an old message, regenerate → clean recompaction.
Mid-task boundary tests (side-effect safety)
Mid-task compaction must preserve exactly-once semantics for tools:
Ask for a 10+ step tool task where step N writes a file right before a
pass boundary (rounds-per-pass 2 makes boundaries frequent) → after the
continuation, the model must NOT redo the write; the file's content
reflects exactly one write.
Cancel immediately after a tool completes near a boundary → resume; the
completed tool's result must still be visible to the model.
A failing tool on the last round of a pass → the error is fed back and
the continuation reacts to it (no repeat of a destructive attempt).
30+ sequential tool calls with aggregate output cap low (e.g. 2000) →
later results clamp with the aggregate marker; mid-task compaction
fires; the task still completes.
Restart LM Studio mid-marathon (after a mid-task compaction) → the next
message resumes with cached chunks and correct task state.
Consolidation tests (bounded summaries)
Set "Summary budget" low (e.g. 4000) in a chat with 4+ summary chunks →
next /compress shows "Consolidating N summaries into one…", the stats
note "incl. 1 consolidated from N earlier chunks", and /usage's chunk
count drops while coverage stays.
Fact recall after consolidation: quiz the TESTING seed facts → 5/5.
Repeated growth: keep chatting past the budget again → a second
consolidation folds [L1 + newer chunks] without touching the newest ones;
prompt returns to the floor each time.
Floor sanity: set "Compact down to" below what the budget + kept-verbatim
allow → one-time "floor is not reachable" warning appears and compaction
targets the clamped floor instead.
Attachment memory tests
Attach a .txt containing a distinctive fact (e.g. "the vault code is
OTTER-771"), chat past the compaction trigger so the attachment's region
compresses ("Preserving attachment…" status appears) → quiz the model on
the fact → recalled from the summary.
Same with an image on a vision model → the image gets described; quiz on
visible content. On a non-vision model → chunk summary shows
"content not retained", no crash.
Abort mid-"Preserving attachment…" → nothing persisted; next compaction
retries the extraction.
Restart LM Studio, compact a different chat referencing the same file →
extraction is NOT repeated (cached by identifier).
Agentic Workspace protocol tests
Model matrix (32-40)
Enable context-compressor 0.7.0 and Agentic Workspace 0.2.0 in one disposable-workspace chat.
Create a todo_* board with multiple item_* entries, including a dependency and one critical next action.
Start a run_*, apply a tx_*, and execute a job_* that returns a known exit code.
Start a research_* project, run at least two queries, fetch two pages as source_* records, and produce a report.
Force /compress after those tool results are outside the verbatim tail.
Inspect the generated continuation: it must retain all IDs, latest statuses, next-action text/priority/dependencies, exit evidence, source URLs/hashes/content paths, report path, and diff/review artifacts.
Confirm large diff previews, page text, search arrays, and stdout/stderr were not copied into the summary.
Recover each durable object with Agentic Workspace show/status/changes calls and continue the task without reconstructing state from prose.
qwen3-8b (thinking) → chat past the compaction trigger; chunk summaries
contain no <think> tags or other reasoning leakage; confirm /no_think
is still sent to the summarizer.
qwen2.5-7b → compaction fires normally; confirm no stray /no_think
suffix reaches the summarizer.
DeepSeek-R1-Distill-7B → chunk summaries are clean; abort a chunk
summarization mid-reasoning → nothing gets cached for that chunk.
gemma-3-4b → the one-time system-role fold kicks in, compaction fires
with no template error, and seeded facts are still recalled after
compaction. (Expected-pass is required for this base scenario only —
see the mid-history-system variant below.)
Mid-history-system variant: after the base scenario passes, insert a
system-role message mid-chat (e.g. via a tool/plugin that emits one,
or by switching the system prompt mid-conversation if the UI allows
it) and send another turn. A strict-alternation rejection from
Gemma's chat template is a known residual here — verify the failure
mode is a VISIBLE error (a status/content-block error, e.g. "Chat
template render failed" or "Prediction failed"), never a silent
stall with no reply and no error shown.
Any model at 4096 context, run at DEFAULT settings (not the fast-trigger
settings above) → the small-context clamp notice fires once and
compaction still fires at the clamped defaults.
llama-3.2-3b or qwen3-1.7b → auto mode selects the compact summarizer
prompt; sanity-check summary quality and run npm run bench against it.
A non-tool-trained model with another tool plugin enabled → tools are
disabled with a visible notice; the reply still comes back normally.
One vision model → attachment memory smoke test (same as test 21,
unchanged).
Enable context-compressor 0.7.0 and Agentic Workspace 0.2.0 in one disposable-workspace chat.
Create a todo_* board with multiple item_* entries, including a dependency and one critical next action.
Start a run_*, apply a tx_*, and execute a job_* that returns a known exit code.
Start a research_* project, run at least two queries, fetch two pages as source_* records, and produce a report.
Force /compress after those tool results are outside the verbatim tail.
Inspect the generated continuation: it must retain all IDs, latest statuses, next-action text/priority/dependencies, exit evidence, source URLs/hashes/content paths, report path, and diff/review artifacts.
Confirm large diff previews, page text, search arrays, and stdout/stderr were not copied into the summary.
Recover each durable object with Agentic Workspace show/status/changes calls and continue the task without reconstructing state from prose.
qwen3-8b (thinking) → chat past the compaction trigger; chunk summaries
contain no <think> tags or other reasoning leakage; confirm /no_think
is still sent to the summarizer.
qwen2.5-7b → compaction fires normally; confirm no stray /no_think
suffix reaches the summarizer.
DeepSeek-R1-Distill-7B → chunk summaries are clean; abort a chunk
summarization mid-reasoning → nothing gets cached for that chunk.
gemma-3-4b → the one-time system-role fold kicks in, compaction fires
with no template error, and seeded facts are still recalled after
compaction. (Expected-pass is required for this base scenario only —
see the mid-history-system variant below.)
Mid-history-system variant: after the base scenario passes, insert a
system-role message mid-chat (e.g. via a tool/plugin that emits one,
or by switching the system prompt mid-conversation if the UI allows
it) and send another turn. A strict-alternation rejection from
Gemma's chat template is a known residual here — verify the failure
mode is a VISIBLE error (a status/content-block error, e.g. "Chat
template render failed" or "Prediction failed"), never a silent
stall with no reply and no error shown.
Any model at 4096 context, run at DEFAULT settings (not the fast-trigger
settings above) → the small-context clamp notice fires once and
compaction still fires at the clamped defaults.
llama-3.2-3b or qwen3-1.7b → auto mode selects the compact summarizer
prompt; sanity-check summary quality and run npm run bench against it.
A non-tool-trained model with another tool plugin enabled → tools are
disabled with a visible notice; the reply still comes back normally.
One vision model → attachment memory smoke test (same as test 21,
unchanged).