Project Files
README.md
Text-only Documentation
Looking for the illustrated user documentation? https://github.com/ceveyne/user-docs-docs
LM Studio Plugin — Personal Knowledge Management supported by vision-capable Agents
Search your personal conversations, Markdown and PDF documentation from within an LM Studio chat. The plugin retrieves the most relevant text sections and, where available, delivers the images embedded in those sections directly into the conversation.
Probably, you have documents. Lots of them? Plugin guides, device manuals, personal notes, old conversations with useful information buried in them? And when you actually need something specific, good luck finding it again — especially if the answer sits on page 327 of a PDF or in a chat from three weeks ago.
LM Studio Plugin: user-docs — Personal Knowledge Management supported by vision-capable Agents.
pN candidatesTL;DR: You ask a question. The agent searches through all your docs, finds the answer, shows you the relevant screenshot if one exists, and gives you a short precise response. No more Ctrl+F across 47 files.
When you ask a question, user-docs searches your local documentation directories using hybrid retrieval (BM25 + embeddings). For each retrieved text chunk the plugin extracts image references — including embedded base64 images and Obsidian wikilinks — generates previews, and registers them as pN entries in chat_media_state.json. This means review_image, analyse_image, annotate_image can operate on retrieved screenshots without any extra configuration.
Known source files or conversations can be image-fetched directly via fetch_image. PDF pages can be rendered on demand via extract_image.
See USER_GUIDE.md for in-depth documentation and use cases.
In addition to the resources your main agent model requires, keep in mind:
extract_image. The plugin creates isolated virtual environments internally; nothing pollutes your system Python.lms runtime update --all.⚖️ Distributed setup: You can run the LM Studio Server on a powerful machine in your network while using the plugin from a lightweight client. Documents are always local to where the plugin runs; only model inference is offloaded via API calls. See Deployment scenarios below.
Customize your setup based on where LM Studio Server runs:
All components run on one machine. LM Studio acts as both client and server.
LM Studio Client and LM Studio backend share one machine. The LM Studio Server (agent inference) runs on a dedicated, more powerful machine.
baseUrlmust point to the reachable server. Configure via Plugin Settings → API server base-URL.
Install and activate ceveyne/user-docs in LM Studio.
💡 In case it's not activated right away after download, activate the user-docs plugin by choosing it from the model-loader
Your Generators.
The plugin requires a running local server to communicate with models and tools.
The agent model orchestrates all plugin commands and accompanies you through chats. Two recommended options:
| Option | Setting | Characteristics |
|---|---|---|
| Default (fast, compact) | qwen/qwen3.6-35b-a3b | Good for clearly instructed tasks |
| Option | Setting | Characteristics |
|---|---|---|
| More powerful (slower, flexible) | qwen/qwen3.6-27b | Handles complex and open-ended tasks better |
💡 Other OpenAI-compatible APIs work as well: other local backends (oLMX, Unsloth Studio), cloud providers (OpenAI, Claude). Configure via Plugin Settings → Agent Model, API server base-URL, (Optional) API Key.
Determine what you want from user-docs. This determines which directories to configure and whether the auto-read guide should be on or off:
What content do you want indexed?
Primary purpose:
Auto-read user-docs Guide ON (default) — agent starts fully operational every session. Sessions start slightly slower.Three directory types are supported:
| Type | Access | Purpose |
|---|---|---|
| Notes Directory | Read-write | Agent stores notes here (memorize_doc, export_doc). Contains USER-DOCS.md as the plugin's core guide. Recommended path: ~/Documents/User-docs |
| Content Directories | Read-only | Your documents — PDF, TXT, MD files fully searchable and indexable |
| LM Studio Conversations | Read-only | Add ~/.lmstudio/conversations to include chat history in the index |
With conversations added:
Configure via Plugin Settings → Notes Directory / Content Directories. Set
Auto-read user-docs Guidetoggle based on your Step 3 decision.
Two small models for search and image analysis:
| Setting | Recommended Value | Purpose |
|---|---|---|
| Embedding Model | ggml-org/bge-m3-Q8_0-GGUF | Semantic text search across documents (find_doc) |
| Vision Model (Qwen3-VL) | qwen/qwen3-vl-8b | Image analysis and object detection (analyse_image, annotate_image) |
Both models should be loaded via the LM Studio Server. Other models are not recommended unless they're variants (finetunes, quantizations) of these standards. Configure via Plugin Settings → Embedding Model / Qwen3-VL.
Two prompt fields for image analysis behavior:
| Setting | Default | Purpose |
|---|---|---|
Vision Prompt (analyse_image) | Empty | Sent to the vision model when no explicit description is requested. Leave empty → visual descriptions only on agent request (recommended default). analyse_image always returns generation metadata automatically. |
| Setting | Default | Purpose |
|---|---|---|
Object Detection Prompt (annotate_image) | Built-in hierarchical detection rules | Tells Qwen3-VL what to detect and mark with bounding boxes. The built-in default works well, but specifying a custom prompt per request gives precise control (e.g., "mark only the login button"). |
Configure via Plugin Settings → Vision Prompt / Object Detection Prompt.
After completing all steps, test with a live search:
find_doc results and screenshots if availableSetup complete. 🎉
| Issue | See also |
|---|---|
| Server not reachable | Step 1 — check green toggle in Developer panel |
| Wrong model loaded | Step 2 / Step 5 — verify settings match recommendations |
| Empty search results | Step 4 — ensure Content Directories are populated and indexed |
| Vision tools failing | Step 6 — confirm Qwen3-VL is vision-capable (supportsVision: true) |
| Python not found | See chapter: "Install Python, if needed" below |
find_doc, PDF parsing, and extract_image require Python 3.9+.
The plugin searches for Python automatically. It probes well-known install locations in this order:
/usr/local/bin/python3.X), Homebrew (/opt/homebrew/bin/python3.X), Xcode CLT, system /usr/bin/python3, then LM Studio's bundled Python as a last resort.%LOCALAPPDATA%\Programs\Python\Python3XX\python.exe), system installer (), , pyenv-win, then LM Studio's bundled Python as a last resort.If Python is not found:
All packages are installed into isolated venvs inside the plugin directory; nothing is written to your system Python.
The plugin creates and repairs isolated Python environments automatically:
| Environment | Used for | Minimum Python | Validation before reuse |
|---|---|---|---|
.rag-venv | RAG parsing, PDF handling | 3.9+ | python3, pip, docling, fitz |
Ready markers are only trusted when these checks pass. If an environment was deleted or is incomplete, the plugin rebuilds it instead of waiting until a later timeout.
| Setting | Default | Description |
|---|---|---|
model | qwen/qwen3.6-35b-a3b | Agent/orchestrator model |
baseUrl | http://127.0.0.1:1234/v1 | LM Studio API base URL |
apiKey | (empty) | API key (only if the server requires authentication) |
autoReadUserDocsGuide | true | Provide user-docs.md as non-persistent user-message context and seed it into notesDirectory if missing |
notesDirectory | ~/Documents/User-docs | Notes write target; always indexed |
contentDirectories | ~/Documents | Directories to index (TXT, MD, PDF) |
embeddingModel | ggml-org/bge-m3-Q8_0-GGUF | Embedding model for semantic retrieval (must be loaded in LM Studio) |
embeddingBaseUrl | http://127.0.0.1:1234/v1 | Embedding/Vision API Base URL |
embeddingApiKey |
| Tool | Description |
|---|---|
find_doc | Hybrid RAG search. Returns text context + registers embedded images as pN candidates. |
read_doc | Read a note file in full or partial form. Registers document images by default; base64 payloads are stripped. |
fetch_image | Register all images from one exact filename, Markdown document, image file, or LM Studio conversation as pN. |
extract_image | Render a PDF page to PNG via PyMuPDF and register it as iN. |
review_image | Present image candidates to the user for selection. |
review_sequence | Step through a sequence of images interactively. |
analyse_image | Describe the pixel content of a pN/iN/aN image via the configured LM Studio Vision API backend. |
annotate_image | Detect objects and draw bounding boxes via Qwen3-VL. |
show_image | Promote a pN/iN candidate to a visible chat image. |
skip_doc |
The interactive setup guide (SETUP.md) references screenshots stored in docs/initial-docs/images/. These images are not bundled inside the plugin — they are excluded from both the build output and the LM Studio Hub upload to stay within the Hub file-count limit.
Instead, the plugin fetches the images lazily from this GitHub repository the first time the setup guide is triggered after a fresh install or update. The fetch is intentional and targets a fixed path in this repository (docs/initial-docs/images/). LM Studio has no built-in update mechanism — to install a new version, users must manually delete the existing plugin directory first. That typically removes the entire directory, so the fetch runs again after each reinstall; within a single install it uses the locally cached copies in docs/initial-docs/images/ inside the plugin's install directory.
Only SETUP.md and docs/initial-docs/images/ are tracked in this repository; all other files under docs/initial-docs/ are git-ignored.
See USER_GUIDE.md for in-depth documentation, use cases, technical requirements, known issues, and more.
See CHANGELOG.md for version history and release notes.
MIT
%ProgramFiles%\Python3XX\python.exe%LOCALAPPDATA%\Python\bin\python.exe| API key (only if the server requires authentication) |
qwen3VlModelPath | qwen/qwen3-vl-8b | LM Studio model key for the Qwen3-VL Vision API model |
visionPrompt | (empty) | Default prompt for analyse_image. Leave empty to disable automatic description |
qwen3VlOdPrompt | _(built-in)_ | Instruction sent to Qwen3-VL for object detection (annotate_image) |
Remove a read_doc result from the API context (token saving). |
memorize_doc | Write a new note to notesDirectory with YAML frontmatter. |
rewrite_doc | Overwrite an existing note. |
forget_doc | Delete a note from notesDirectory. |
export_doc | Export the current chat as a Markdown note into notesDirectory. |
read_config | Return the current plugin settings as JSON so an agent can inspect live configuration values and assist with setup. |
┌──────────────────────────────────────────────────────────────────┐
│ MacBook Pro M5 │
│ │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ LM Studio App │ │
│ │ │ │
│ │ ┌──────────────────────────────────────────────────────┐ │ │
│ │ │ Plugin: user-docs │ │ │
│ │ │ │ │ │
│ │ │ vision-capability-primer: qwen/qwen3-vl-4b │ │ │
│ │ └──────────────────────────────────────────────────────┘ │ │
│ │ │ │ │
│ │ │ OpenAI-compat. API │ │
│ │ ▼ │ │
│ │ ┌──────────────────────────────────────────────────────┐ │ │
│ │ │ LM Studio Server (local) │ │ │
│ │ │ baseUrl: http://127.0.0.1:1234/v1 │ │ │
│ │ │ Agent Model: qwen/qwen3.6-27b │ │ │
│ │ │ Embedding Model: ggml-org/bge-m3-Q8_0-GGUF │ │ │
│ │ │ Vision Model: qwen/qwen3-vl-8b │ │ │
│ │ └──────────────────────────────────────────────────────┘ │ │
│ └────────────────────────────────────────────────────────────┘ │
│ │
└──────────────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────────┐
│ MacBook Neo │
│ │
│ ┌────────────────────────────────────────────────────────────┐ │ ┌─────────────────────────────┐
│ │ LM Studio App │ │ │ Mac Studio M3 Ultra │
│ │ │ │ │ │
│ │ ┌──────────────────────────────────────────────────────┐ │ │ │ │
│ │ │ Plugin: user-docs │ │ │ │ LM Studio Server │
│ │ │ │ │ │ │ │
│ │ │ vision-capability-primer: qwen/qwen3-vl-4b │ │ │ │ │
│ │ └──────────────────────────────────────────────────────┘ │ │ │ Agent Model: │
│ │ │ │ │ │ qwen/qwen3.6-27b │
│ │ │ OpenAI-compat. API ───────────────│──▶︎│ http://<studio-ip>:1234/v1 │
│ │ ▼ │ │ │ │
│ │ ┌──────────────────────────────────────────────────────┐ │ │ │ │
│ │ │ LM Studio Server (local) │ │ │ │ │
│ │ │ baseUrl: http://127.0.0.1:1234/v1 │ │ │ │ │
│ │ │ Agent Model: qwen/qwen3.6-27b │ │ │ │ │
│ │ │ Embedding Model: ggml-org/bge-m3-Q8_0-GGUF │ │ │ │ │
│ │ │ Vision Model: qwen/qwen3-vl-8b │ │ │ │ │
│ │ └──────────────────────────────────────────────────────┘ │ │ └─────────────────────────────┘
│ └────────────────────────────────────────────────────────────┘ │
│ │
└──────────────────────────────────────────────────────────────────┘