context-compactor
Public
Keeps a long LM Studio chat going past the context limit — in the same window, without starting a new one.
When the context fills up, the plugin summarizes the older part of the conversation into a consolidated state and sends the model that state plus the recent messages instead of the full history. Your chat is untouched: every message stays on screen and you keep scrolling through all of it. Only the prompt shrinks.
Because attention cost grows with context length, a shorter prompt is also a faster one. On a memory-limited machine, compacting is a speed setting as much as an overflow guard — measured at +28% tokens/s on a 12B model after a 17k → 850 token compaction.
It cannot delete messages from your chat. No plugin can: pullHistory() returns a copy and the SDK
exposes no truncate API. That turns out not to matter — the displayed history and the prompt sent to
the model are two different things, and only the second one needs to shrink.
Compaction rests on one assumption: that the past compresses. A working conversation does — decisions, paths, code, state, the reasons behind them. Twenty pages of that fold into one without losing anything you needed.
Bulk data does not. If your conversation is an analysis of large files — reverse engineering, log trawling, dumps of any kind — the past is the data, and a summary of a hex dump is not a smaller hex dump, it is nothing at all. The plugin then dutifully deletes exactly what the model needs, the model reads it again, the context fills again, and it compacts again. The symptom is a chat that compacts constantly and gets no further, and it is self-reinforcing: every compaction causes the re-reading that triggers the next.
No setting fixes this, because it is not a bug. It is what applying a conversation tool to a data
problem looks like. Those workloads want the opposite architecture: tools that answer a question
without ever dumping — objdump -d --start-address=… --stop-address=… rather than disassembling
3 MB, a headless decompiler that returns one function on request rather than a binary. Turn the
plugin off for them.
It is also of no use in short chats that never approach the context limit: it will simply never fire.
From the LM Studio Hub — open lmstudio.ai/alexandreaxell/context-compactor and install from
there. (The Hub has no plugin catalogue to browse, so the link is the only way in.)
From source — clone the repo and run install.bat (Windows), or:
LM Studio must be running for the install step.
Then enable context-compactor in the plugin menu of a chat. Plugins are enabled per conversation, so a chat you already had open will not have it yet.
Your model stays selected in the top bar as usual — the plugin reads it via tokenSource(), along
with your temperature, system prompt and the rest of your prediction config. Nothing about your setup
changes.
| Setting | Default | Notes |
|---|---|---|
| Compaction trigger | 0.6 | Fraction of the context window that triggers compaction. Lower keeps generation fast but compacts more often. |
| Recent context kept verbatim | 6000 | Tokens at the end of the conversation that are never summarized. Tokens, not messages — see below. |
| Summarize in chunks of | 16000 | Tokens of history per summary. See below — smaller is slower, not faster. |
| Maximum size of a tool result | 4000 | Tool results are truncated to this before the model sees them. |
| Archive folder | (empty) | Where transcripts and state notes are written. An Obsidian vault works well. Empty means nothing is written to disk. |
| Announce compaction in chat | on | Shows before/after token counts. |
Leave headroom on the trigger: the measurement counts the conversation, but tool definitions are injected downstream and are not included. With several MCP servers loaded, those definitions are a fixed cost that no amount of compaction can reduce.
Both the verbatim tail and the tool ceiling were once counted in something else — messages, and megabytes — and both failed the same way. A message is not a unit of size: a question is three tokens and a disassembly dump is forty thousand. Keeping "the last 8 messages" meant faithfully preserving 60k tokens of hex while diligently summarizing the 19k before it, and producing a "compacted" prompt that still overflowed. A megabyte of tool output is a quarter of a million tokens — four times a 64k window — so a cap that sounds small enough for a disk is no cap at all.
The context is measured in tokens. Anything guarding it has to be measured in tokens too.
Compaction runs once, before a reply. Tool results arrive during one, round after round, long after it has had its say. Servers impose no limits of their own — the official filesystem server will return a 6 MB binary if asked — so one careless call can end a reply that compaction had just made room for.
Every tool is therefore wrapped before it reaches the model: each result is truncated to the ceiling above, and all of them together are bounded by what the window actually has left. Structure is preserved — an MCP result stays an MCP result, only its text shrinks — and the tool still runs, so a call that writes a file still writes it. When the budget is spent, results carry a notice telling the model to answer with what it has rather than to keep calling.
Reading is far cheaper than writing. Measured on one local 12B: 3482 tok/s to read a prompt against 45 tok/s to generate — a 77:1 ratio. And a chunk costs one summary regardless of how much history it holds.
So halving the chunk size doubles the number of summaries, doubles the tokens generated, and roughly doubles how long a compaction takes — while the extra input it saves was nearly free anyway. Small chunks buy only finer cache granularity, at a price paid in the scarce resource.
The one real reason to lower it is fidelity: a small model summarizing 32k in one pass drops more than two summaries of 16k. Lower it if your state notes are losing detail, not to go faster.
Set an archive folder and every compaction writes two linked notes:
Logs/<timestamp>-<id>.transcript.md — the verbatim messages leaving the context. Nothing is lost;
it just stops being in the prompt.Projects/<id>.state.md — the consolidated state, overwritten on each compaction.This is deterministic. Telling a model in its system prompt to log its own work does not survive contact with a filling context — it logs when it remembers to, which is rarely, and less and less as the context fills.
Cuts never orphan a tool call. Splitting between an assistant's tool request and its result produces a conversation most inference backends reject. Cuts land on completed rounds only, and prefer a user turn — strict-alternation templates like Gemma's expect the window to open on one.
Summaries are always re-derived from the original messages, never from a previous summary.
Summarizing a summary is a game of telephone: errors set and compound with no source left to
arbitrate. pullHistory() always returns the full history, so re-deriving costs nothing in fidelity.
Per-chunk caching keeps re-deriving affordable. The history is chunked at a fixed token budget and each chunk's summary is cached by content hash. The budget is fixed rather than a fraction of the context window on purpose: a greedy walk from the start of the history yields the same boundaries every time, so every chunk but the last stays byte-identical between compactions and hits the cache. A budget scaled to the context window would produce one large chunk whose content changed every time, and nothing would ever be reused. Cost tracks new material, not conversation age.
The compacted prefix is stable between compactions, so the KV cache survives and prompts are not reprocessed on every message.
Failure is loud, not silent. If the (experimental) tool session API refuses, you get a visible notice rather than tools quietly disappearing. If compaction fails, the full history is sent — the behaviour you had without the plugin.
Read this before trusting the plugin with work you care about.
Registering a prediction loop handler means this plugin owns the loop. Everything LM Studio's own loop does, it now has to do — and it only reimplements what is documented here. Anything else is gone or degraded, whether or not this list names it.
Tool call confirmation still works — verified, with a tool set to "ask" and the plugin enabled. Tools reach the model through LM Studio's own proxy implementations, so confirmation happens on its side of that boundary and this plugin never gets the chance to bypass it. Your "ask" / "allow all" settings behave exactly as they did before.
Reasoning blocks are handled — also verified. Fragments are routed by their reasoningType:
reasoning text goes to a collapsible thinking block, and the start/end tags are consumed rather than
printed. Getting this wrong leaks LM Studio's internal separator token into the chat, which is what
this plugin did until someone hit it.
Citations and any other rendering the default loop performs remain untested. That list is written from what has been found so far, not from an inventory of what LM Studio's loop actually does — the gaps you should expect are the ones not named here.
startToolUseSession() is marked @experimental in the SDK; the generator handle path is marked
@experimental too. LM Studio can change or remove either without notice, and a routine app update
may break this plugin. When the tool session fails, the plugin says so in the chat rather than
silently dropping your tools — but the reply still goes out without them.
The summary is model-generated. It can drop or distort detail, and the model cannot consult the archived transcript — only what the summary preserved. A summary is only as good as the model writing it; a small model produces a small-model summary of your work.
Older attachments are lost outright: images and files in the compacted range are rendered as text and their content does not survive. Attachments inside the verbatim tail are unaffected.
Compaction blocks your reply. The first one on a long chat can take minutes on a local model, and there is no way to skip it once it starts. The per-chunk cache makes later compactions cheap, but it is empty the first time.
On a conversation that keeps pulling in large data, compaction fires again and again and each one is a pause. That is the workload described under When not to use it, and the answer there is to turn the plugin off, not to tune it.
Every compaction rewrites the start of the prompt, which invalidates the KV cache and forces a full prefill on the next message. This is a one-off cost per compaction, not per message — but a low trigger means more compactions, and therefore more prefills.
The token measurement counts the conversation only. Tool definitions are injected downstream and are not included, so real usage is higher than the plugin thinks — with several MCP servers, materially higher. Keep headroom on the trigger.
LM Studio with plugin support, and a model as token source (token accounting needs a real model handle, not a generator plugin).
Fourteen checks over the two rules that would break the plugin silently rather than loudly: the cut never orphaning a tool result from its request, and chunk boundaries staying stable as the conversation grows — the property the whole cache rests on.
MIT
Projects/<id>.state.md is overwritten each time.
Transcripts accumulate and are never cleaned up.npm install
lms dev --install -y
npx tsc && node test-split.js