omnimind
Public
Project Files
README.md
OmniMind is a powerful local-first Retrieval-Augmented Generation (RAG) plugin built on top of LM Studio. It acts as a bridge between your local LLMs and your personal knowledge base, seamlessly ingesting your Obsidian Vault and Zotero Library into a local vector database for instant, private semantic search and chat.
While there are already several excellent MCP (Model Context Protocol) servers available for connecting to Obsidian, and robust integrations for Zotero, relying on them for local AI research presents a significant challenge: Tool Overload.
Providing too many disparate tools or multiple MCP servers to smaller, local agents (especially models under 15B parameters) often leads to unreliable behavior, confusion in tool selection, and severe hallucinations.
OmniMind was built to solve this by providing a single, highly-curated, and unified toolset specifically designed to help tiny models excel at academic and personal research.
Additionally, OmniMind brings industry-best retrieval algorithms directly to local agents:
To get the best performance out of OmniMind on a mid-range GPU (e.g., RTX 3060 with 10-12 GB VRAM), we recommend loading the following models simultaneously in LM Studio (ensure "Keep multiple models in memory" is enabled):
lmstudio-community/embeddinggemma-300m-qat-GGUF (Lightweight and highly accurate)ggml-org/DeepSeek-OCR-GGUF/DeepSeek-OCR-Q8_0.gguf (~4 GB, fantastic for extracting raw text from scanned PDFs)ibm/granite-4-h-tiny (~4 GB, fast reasoning for RAG synthesis)embeddinggemma-300m-qat-GGUF).Want to modify the plugin from source, or build it yourself? Please see our DEVELOPING.md guide for instructions on setting up your local environment and running tests!
You can easily configure OmniMind directly inside the LM Studio Plugin UI. There is no need to manually edit .env files!
Ensure the following paths are set correctly for your system:
C:\Path\To\Your\Obsidian\VaultC:\Path\To\Your\Zotero\zotero.sqliteC:\Path\To\Your\Zotero\storage4 (Recommended)lmstudio-community/embeddinggemma-300m-qat-GGUFOnce the plugin is running, navigate to:
The Control Panel provides a real-time dashboard to:
When using local models (especially those under 30B parameters like gpt-oss-20b or ibm/granite-4-h-tiny) for Retrieval-Augmented Generation (RAG), you may encounter specific quirks. Understanding these will help you steer the agent effectively.
Instruction-tuned models are heavily rewarded during training for being helpful. If you ask for a list of reading materials and the database only finds 1 or 2 matches, the model's neural weights will often "want" to give you a top-5 list. To satisfy this, it may invent (hallucinate) realistic-sounding reports or government agencies (e.g., inventing an "FTC AI Guidance" document).
The Fix: Use extremely strict, negative-constraint system prompts. Recommended System Prompt Template:
"You are an extraction assistant. Your job is to extract the names of books, papers, or authors based on the Zotero database or Obsidian notes that you have access via the tools. Do not mention any prior knowledge that does not explicitly appear from the search results of the tool."
If you ask about "EU AI Policy" in Turn 1, and then in Turn 2 ask "What is Floridi's take on this?", the LLM will strongly bind the word "this" to the exact topic of EU AI Policy. If it retrieves a broad paper by Floridi about general "Digital Governance," it may falsely claim the paper is irrelevant because it doesn't mention the EU explicitly.
The Fix: When you want the agent to synthesize broad concepts or make creative connections across texts, explicitly ask it to synthesize. Avoid ambiguous pronouns. Example: > "What is Floridi's take on the broader concept of AI governance, and how might his 2018 paper on 'Soft Ethics' apply to the EU policies we just discussed?"
Extremely small models (e.g., < 5B parameters) struggle with strict negative constraints. If you give them the strict prompt above, they may become overly rigid. For example, if you ask for "US reading materials", and a chunk is titled "AI Index Report 2026" but the text mentions "California" and "Montana", the tiny model might fail to realize that California is in the US, and output: "I could not find any US-specific materials."
The Fix: If you experience false negatives (the database retrieves the document but the agent ignores it), you either need to loosen the prompt slightly or upgrade your inference model to a slightly larger class (e.g., Llama-3-8B-Instruct or Qwen-2.5-7B) for the final RAG generation step.
MIT License. See LICENSE for more information.
ggml-org/DeepSeek-OCR-GGUF/DeepSeek-OCR-Q8_0.ggufibm/granite-4-h-tiny