Forked from crunch3r/ai-toolbox
Deep dive into the AI Toolbox plugin's system architecture, design patterns, and internal workflows.
┌─────────────────────────────────────────────────────────────────────┐ │ LM Studio Host │ │ │ │ ┌───────────────────────────────────────────────────────────────┐ │ │ │ Plugin Runner (Node.js) │ │ │ │ │ │ │ │ ┌─────────────────────────────────────────────────────────┐ │ │ │ │ │ AI Toolbox Plugin │ │ │ │ │ │ │ │ │ │ │ │ ┌──────────────┐ │ │ │ │ │ │ │ index.ts │◄─── Entry Point (main function) │ │ │ │ │ │ │ (entry) │ │ │ │ │ │ │ └──────┬───────┘ │ │ │ │ │ │ │ │ │ │ │ │ │ ▼ │ │ │ │ │ │ ┌─────────────────────────────────────────────────┐ │ │ │ │ │ │ │ Core Services │ │ │ │ │ │ │ │ ┌──────────┐ ┌──────────┐ ┌──────────────────┐ │ │ │ │ │ │ │ │ │ config.ts│ │security │ │stateManager.ts │ │ │ │ │ │ │ │ │ │(Zod+UI) │ │ .ts │ │(persistence) │ │ │ │ │ │ │ │ │ └──────────┘ │(validators)│ └──────────────────┘ │ │ │ │ │ │ │ │ └──────────┘ │ │ │ │ │ │ │ │ ┌──────────┐ ┌──────────┐ ┌──────────────────┐ │ │ │ │ │ │ │ │ │workingDir│ │performanc│ │promptPreprocessor │ │ │ │ │ │ │ │ │ │ .ts │ │eUtils.ts │ │ .ts │ │ │ │ │ │ │ │ │ │(path mgmt│ │(caching) │ │(Document RAG + │ │ │ │ │ │ │ │ │ └──────────┘ └──────────┘ │ ContextGuard) │ │ │ │ │ │ │ │ │ └──────────────────┘ │ │ │ │ │ │ │ └─────────────────────────────────────────────────┘ │ │ │ │ │ │ │ │ │ │ │ │ ┌──────────────────────────────────────────────────┐ │ │ │ │ │ │ │ Tool Registration Layer │ │ │ │ │ │ │ │ ┌─────────────────┐ ┌──────────────────────┐ │ │ │ │ │ │ │ │ │ toolsProvider.ts │ │ ToolRegistry │ │ │ │ │ │ │ │ │ │ (factory fn) │ │ (central map) │ │ │ │ │ │ │ │ │ └────────┬────────┘ └──────────┬───────────┘ │ │ │ │ │ │ │ └───────────┼──────────────────────┼─────────────┘ │ │ │ │ │ │ │ │ │ │ │ │ │ │ ┌───────────┴──────────────────────┴─────────────┐ │ │ │ │ │ │ │ Tool Modules (17 files) │ │ │ │ │ │ │ │ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ │ │ │ │ │ │ │ │ │fileSys │ │webRes │ │browser │ │ git │ │ │ │ │ │ │ │ │ │ (21) │ │ (4) │ │ (5) │ │ (13) │ │ │ │ │ │ │ │ │ └────────┘ └────────┘ └────────┘ └────────┘ │ │ │ │ │ │ │ │ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ │ │ │ │ │ │ │ │ │ datab │ │backgnd │ │exec │ │ utility│ │ │ │ │ │ │ │ │ │ (1) │ │ cmd(3) │ │ (5) │ │ (28) │ │ │ │ │ │ │ │ │ └────────┘ └────────┘ └────────┘ └────────┘ │ │ │ │ │ │ │ │ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ │ │ │ │ │ │ │ │ │ image │ │ http │ │ vector │ │ UI │ │ │ │ │ │ │ │ │ │ (4) │ │ (3) │ │ RAG(4) │ │ Gen(3) │ │ │ │ │ │ │ │ │ └────────┘ └────────┘ └────────┘ └────────┘ │ │ │ │ │ │ │ │ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ │ │ │ │ │ │ │ │ │ docParse│ │textProc│ │ Context │ │ Backup │ │ │ │ │ │ │ │ │ │ (1) │ │ (3) │ │ Mgmt(7) │ │ (4) │ │ │ │ │ │ │ │ │ └────────┘ └────────┘ └────────┘ └────────┘ │ │ │ │ │ │ │ │ ┌────────┐ │ │ │ │ │ │ │ │ │ LineOps │ │ │ │ │ │ │ │ │ │ (3) │ │ │ │ │ │ │ │ │ └────────┘ │ │ │ │ │ │ │ └─────────────────────────────────────────────┘ │ │ │ │ │ └───────────────────────────────────────────────────┘ │ │ │ └─────────────────────────────────────────────────────────┘ │ │ │ │ │ ┌─────────────────────────────────────────────────────────┐ │ │ │ │ External Dependencies │ │ │ │ │ Puppeteer │ isomorphic-git │ Tesseract.js │ pdf-parse │ │ │ │ │ duck-duck-scrape │ node:sqlite │ node-notifier │ │ │ │ └─────────────────────────────────────────────────────────┘ │ │ └───────────────────────────────────────────────────────────────┘
The project uses Tsup (esbuild-based bundler) for fast, zero-config compilation to both ESM and CJS formats.
tsconfig.json defines @/* → src/* for IDE and TypeScript support.tsup.config.ts maps @ → path.resolve(__dirname, 'src') to ensure the bundler resolves aliases correctly in the final output.src/toolsProvider.ts)Central registry managing all tool instances:
Key Design Decisions:
src/stateManager.ts)Persistent state management with dynamic path resolution:
Key Features:
Session Summary Tool Flow (v1.5.15+):
Storage Format:
zlib.gzipSync(level: 9) → base64 encoding → stored in StateManager (typically achieves ~30% size reduction)Working Directory Integration:
StateManager initializes via getMemoryFilePath() → resolves to {current_working_dir}/.ai_toolbox_memory.msgpacksrc/tools/contextManagementTools.ts)Persistent context storage for session tracking:
Key Features:
src/security.ts)Multi-layer security pipeline:
src/workingDir.ts)Mutable base path for all file operations:
| Cache | TTL | Max Entries | Purpose |
|---|---|---|---|
| Fuzzy Search | 60s | 100 | File name similarity results |
| Web Requests | 30s | 50 | HTTP responses |
Heavy dependencies loaded on first use:
Visual Indicator Example:
security.ts imports from workingDir.ts (not vice versa)stateManager.ts has minimal logger (no index.ts import)The Zod schema (src/config.ts) defines all plugin settings:
Each field maps to a UI element in LM Studio's settings panel via createConfigSchematics().
esm + cjs (dual-package compatibility)es2020 / node platform@lmstudio/sdk, puppeteer, sharp, tesseract.js, isomorphic-git, pdf-parse, mammoth, archiver, unzipper, node-notifier, pixelmatch, pngjs.d.ts files via dts: truesaveToFile()getMemoryFilePath()change_directorypersistenceEnabled === false, returns in-memory keys directly (test isolation). When enabled, reloads from disk before returning (handles working dir changes mid-session)._ready promise) to prevent race conditionsnpm run build # Compiles src/ → dist/ with sourcemaps
npm run typecheck # Validates types without emitting (tsc --noEmit)
npm run lint # ESLint static analysis
// index.ts
export function main(context: PluginContext) {
// 1. Register config schematics (UI toggles)
context.withConfigSchematics(configSchematics);
// 2. Register prompt preprocessor (Document RAG + ContextGuard)
context.withPromptPreprocessor(preprocess);
// 3. Register tools provider (all 108 tools)
context.withToolsProvider(toolsProvider);
// 4. Setup cleanup handlers
process.on('SIGTERM', cleanupBrowserSession);
process.on('SIGINT', cleanupBrowserSession);
}
toolsProvider() called by LM Studio SDK
│
▼
createToolsProvider(config)
│
▼
new ToolsProvider(config)
│
├── StateManager(config) ──────► Load state from disk
├── BackgroundCommandManager ──► Initialize process tracker
└── ToolRegistry.registerAll()
│
├── registerFileSystemTools() ──► 17 tools
├── registerWebResearchTools() ──► 4 tools
├── registerBrowserTools() ──► 5 tools
├── registerGitTools() ──► 15 tools
├── registerDatabaseTools() ──► 1 tool
├── registerDocumentTools() ──► 1 tool
├── registerBackgroundCommandTools() ─► 3 tools
├── registerExecutionTools() ──► 5 tools (filtered)
├── registerUtilityTools() ──► ~28 tools
├── registerImageProcessingTools() ─► 4 tools
├── registerHttpClientTools() ──► 3 tools
├── registerRagTools() ──► 4 tools
├── registerUiGenerationTools() ──► 3 tools
├── registerContextManagementTools() ─► 7 tools
└── registerBackupTools() ──► 4 tools
│
▼
ToolRegistry.toolMap (Map<string, TypedTool>)
│
▼
Return Tool[] to SDK ──► SDK registers with LLM
Session Activity Occurs
│
▼
auto_summarize_context() called
│
├── Analyze tool usage patterns
├── Detect configuration changes
├── Identify important decisions
└── Generate summary
▼
ContextStorageManager.addEntry(entry)
│
├── Load existing entries from .ai_toolbox_context.msgpack
├── Append new entry to beginning of array
├── Limit to 1000 entries (prevent unbounded growth)
└── Save atomically (temp file + rename)
▼
Persistent Storage (.ai_toolbox_context.msgpack)
│
├── get_context_memory() → Retrieve recent entries
├── search_context(query) → Text-based search
├── context_summary() → Statistics & counts
└── delete_context_entry(id) → Remove specific entry
class ToolRegistry {
private toolMap = new Map<string, TypedTool>();
registerAll(config, stateManager, bgCommandManager): void
getAll(): Tool[]
get(name: string): TypedTool | undefined
has(name: string): boolean
}
class StateManager {
private state: Map<string, StateEntry>;
private maxSize: number;
private persistenceEnabled: boolean;
private memoryFile!: string; // Resolved at runtime
set(key, value): void // In-memory + async disk write
get<T>(key): T | undefined // In-memory retrieval
delete(key): boolean // In-memory + async disk update
getAllKeys(): Promise<string[]> // Waits for initialization
clear(): void // Resets in-memory state
}
// save_session_summary writes:
await stateManager.set(`${summaryId}_data`, compressed); // Base64-encoded gzip stream < 10k chars
await stateManager.set(`${summaryId}_timestamp`, Date.now());
// get_session_summary reads with backward-compatible fallback:
const keys = await stateManager.getAllKeys(); // Waits for loadFromFile(), returns all keys
const compressedData = stateManager.get(summaryKey);
try {
const decompressed = zlib.gunzipSync(Buffer.from(compressedData, 'base64')).toString('utf-8');
sessionSummary = JSON.parse(decompressed); // New format (v1.5.15+)
} catch (parseErr) {
// Fallback for legacy uncompressed summaries (pre-v1.5.15)
if (typeof compressedData === 'string' && compressedData.startsWith('{')) {
try {
sessionSummary = JSON.parse(compressedData); // Legacy format
} catch (legacyErr) {
throw new Error(`Legacy summary parsing failed: ${String(legacyErr)}`);
}
} else {
throw parseErr; // Corrupted or unknown format
}
}
class ContextStorageManager {
private storagePath: string; // .ai_toolbox_context.msgpack
load(): Promise<ContextEntry[]>
save(entries: ContextEntry[]): Promise<void>
addEntry(entry: ContextEntry): Promise<void>
getRecentEntries(limit, type?): Promise<ContextEntry[]>
searchEntries(query, maxResults): Promise<ContextEntry[]>
deleteEntry(id): Promise<boolean>
clearAll(): Promise<void>
getSummary(): Promise<ContextSummary>
}
Input → Path Validation → Binary Detection → Command Sanitization → SQL Validation
(validatePath) (isBinaryFile) (sanitizeCommand) (validateSQLQuery)
let currentWorkingDir: string = BASE_DIR;
getWorkingDir(): string
setWorkingDir(newDir: string): boolean
resetWorkingDir(): void
resolvePath(userPath: string): string
getAllowedBases(): string[]
User Path Input
│
├── Empty check ────────────────► Reject
│
├── UNC path check (\\\) ──────► Reject
│
├── Relative path?
│ │
│ ├── Yes: Resolve against basePath
│ │ │
│ │ ├── Within base? ───► Allow
│ │ └── Outside base? ──► Reject
│ │
│ └── No (absolute):
│ │
│ ├── In allowed bases? ──► Allow
│ └── Outside allowed? ───► Reject
Command String
│
▼
Layer 1: Dangerous Pattern Blocking
│
├── Null byte injection ─────────► Reject
├── IFS tampering ───────────────► Reject
├── Dangerous patterns (rm -rf, sudo, etc.) ─► Reject
├── Too many pipes (>2) ─────────► Reject
├── Multiple semicolons (>1) ─────► Reject
├── Command substitution ($(), ``) ─► Reject
├── Environment modification ─────► Reject
│
▼
Layer 2: Tool-Category Enforcement
│
├── classifyCommand() → Set<string>
│ │
│ ├── git * / api.github.com → 'gitOperations'
│ ├── duckduckgo / google / bing → 'webSearch'
│ ├── puppeteer / playwright / chromium → 'browserAutomation'
│ ├── sqlite3 / mysql / psql → 'databaseQueries'
│ ├── curl / wget / http → 'httpClient'
│ └── nohup / disown / & → 'backgroundCommands'
│ │
│ ▼
│ Check against config toggles
│ │
│ ├── Category disabled + !godMode ─► Reject
│ └── Category enabled or godMode ──► Allow
│
▼
Allow Execution
JavaScript Code
│
├── require() detection ─────────► Reject
├── eval() detection ────────────► Reject
├── fs/child_process access ─────► Reject
├── Function constructor ────────► Reject
├── Dynamic import() ────────────► Reject
└── __proto__ access ────────────► Reject
// Stops calculating if minimum possible score drops below threshold
function levenshteinSimilarity(a: string, b: string, minScore: number): number | null {
// Quick rejection for very different lengths
if (lenDiff / maxLen > (1 - minScore)) return null;
// Two-row optimization (saves memory vs full matrix)
// Early exit when row minimum exceeds threshold
}
// Concurrency-controlled batch processing
async function findFilesAsync(dirPath, pattern, maxDepth, concurrencyLimit = 4) {
// Process directories in batches
for (const batch of batches) {
await Promise.all(batch.map(dir => searchDir(dir, depth + 1)));
}
}
User Message
│
▼
promptPreprocessor()
│
├── Check temporalAwareness config
│ │
│ └── Enabled?
│ │
│ ├── Yes: Get cached datetime (5min TTL)
│ │ │
│ │ ├── Format: Standard ([Zeit: ...]) or HEUTE IST Mode
│ │ │
│ │ └── Append timestamp to message end
│ │
│ └── No: Skip
│
▼
Final Prompt sent to LLM (with timestamp suffix)
User Message + Attached Files
│
▼
promptPreprocessor()
│
├── Detect directory paths ─────────► Inject confirmation prompt
│
└── Document RAG enabled?
│
├── Yes: Load embedding model
│ │
│ ├── Process files → chunks
│ │
│ ├── Semantic retrieval
│ │
│ ├── Filter by affinity threshold
│ │
│ └── Inject relevant chunks into prompt
│
└── No: Pass through unchanged
browser_open_page(url)
│
▼
BrowserSessionManager.getBrowser()
│
├── Browser exists & connected? ───► Reuse
│
└── No: Launch new Puppeteer instance
│
├── Retry with exponential backoff (max 2)
│
└── Reset inactivity timer (5 min)
│
▼
Navigate to URL
│
├── Wait for selector (optional)
│
├── Take screenshot (optional)
│
└── Extract text content
Session Activity Detected
│
▼
auto_summarize_context(sessionEvents, configChanges)
│
├── Analyze tool usage patterns (>3 uses = frequent pattern)
├── Track configuration changes
├── Identify important decisions
└── Generate session summary
▼
ContextStorageManager.addEntry(entry)
│
├── Load existing entries from .ai_toolbox_context.msgpack
├── Prepend new entry to array
├── Enforce 1000-entry limit
└── Atomic save (temp file + rename)
▼
Persistent Storage (.ai_toolbox_context.msgpack)
│
├── get_context_memory(limit, type?) → Retrieve entries
├── search_context(query, maxResults) → Text-based search
├── context_summary() → Statistics & counts
└── delete_context_entry(id) / clearContextMemory(confirm) → Management
User Message Arrives
│
▼
promptPreprocessor()
│
├── Check contextGuardEnabled config
│ │
│ └── Enabled?
│ │
│ ├── Yes: Count tokens in history
│ │ │
│ │ ├── Below 90% threshold? ──► Skip compression
│ │ │
│ │ └── Above 90% threshold?
│ │ │
│ │ ▼
│ │ compressHistory(messages)
│ │ │
│ │ ├── Identify messages to compress (all except last 10)
│ │ ├── Send to summary model
│ │ │ └── Use contextGuardSummaryModel or current chat model
│ │ │
│ │ ├── Generate summary with preserved file paths/names
│ │ │
│ │ ├── Calculate tokens saved
│ │ │
│ │ └── Inject visual indicator:
│ │ │
│ │ ├── 🧠 Emoji header
│ │ ├── Messages compressed count
│ │ ├── Tokens before → after (e.g., "~85k → ~42k")
│ │ ├── Percentage saved (e.g., "Saved ~43,000 tokens (~51%)")
│ │ ├── Timestamp
│ │ └── Visual separator lines
│ │
│ └── No: Skip ContextGuard processing
│
▼
Final Prompt sent to LLM (with or without compression indicator)
🧠 **ContextGuard Compression Active**
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
• Compressed 15 message(s) into summary
• Tokens before: ~85k → after: ~42k
• **Saved ~43,000 tokens (~51%)**
• Timestamp: 19:15:32
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
### CONTEXT SUMMARY (from 15 messages)
[Summary content here...]
index.ts
├── toolsProvider.ts
│ ├── config.ts
│ ├── stateManager.ts
│ ├── backgroundCommands.ts
│ └── tools/*.ts (16 modules)
│ ├── security.ts (shared)
│ ├── workingDir.ts (shared)
│ └── performanceUtils.ts (shared)
├── config.ts
├── promptPreprocessor.ts
│ └── config.ts
└── browserAutomationTools.ts (for cleanup)
ConfigSchema (Zod)
├── Tool Gating (13 booleans)
├── Execution Tools (4 booleans)
├── Search Settings (3 fields)
├── Browser Settings (2 fields)
├── Git Settings (2 fields)
├── Document RAG (3 fields)
├── Security Settings (4 fields)
├── State Management (2 fields)
├── i18n (1 field)
├── Notifications (1 field)
├── Temporal Awareness (2 fields: temporalAwareness, dateFormatStyle)
└── ContextGuard (6 fields): v1.4.2
├── contextGuardEnabled (boolean) — Master toggle
├── contextGuardTokenLimit (number 1K-200K) — Compression threshold
├── contextGuardSmartReading (boolean) — Keyword-based file reading
├── contextGuardSummaryModel (string) — Dedicated summary model name
├── contextGuardTerminalFilterEnabled (boolean) — Terminal output filtering
└── contextGuardTerminalFilterLength (number 100-20K) — Max terminal chars
src/
├── index.ts # Plugin entry point
├── toolsProvider.ts # Tool registration + ToolRegistry class
├── config.ts # Zod schema + UI schematics
├── security.ts # Path/SQL/command validators
├── stateManager.ts # Persistent state management
├── workingDir.ts # Working directory manager
├── performanceUtils.ts # Caching, async search, Levenshtein
├── promptPreprocessor.ts # Document RAG + ContextGuard integration
├── backgroundCommands.ts # Background process manager
├── fuzzySearch.ts # Fuzzy file search implementation
├── locales/ # i18n translation files
│ ├── en.ts
│ ├── de.ts
│ ├── zh-CN.ts
│ └── zh-TW.ts
├── tools/ # Tool category modules (17 files)
│ ├── fileSystemTools.ts # 17 file system tools
│ ├── webResearchTools.ts # 4 web research tools
│ ├── browserAutomationTools.ts # 5 browser tools
│ ├── gitGithubTools.ts # 15 Git/GitHub tools
│ ├── databaseTools.ts # 1 database tool
│ ├── documentTools.ts # 1 document parsing tool
│ ├── backgroundCommandTools.ts # 3 background command tools
│ ├── executionTools.ts # 5 execution tools (incl. run_tests)
│ ├── utilityTools.ts # ~28+ utility tools
│ ├── imageProcessingTools.ts # 4 image processing tools
│ ├── httpClientTools.ts # 3 HTTP client tools
│ ├── vectorRagTools.ts # 4 vector RAG tools
│ ├── textProcessingTools.ts # 3 text processing tools
│ ├── uiGenerationTools.ts # Interactive UI Generation (3 tools)
│ ├── contextManagementTools.ts # Auto-Context Management (7 tools)
│ ├── backupTools.ts # Backup & Restore (4 tools)
│ └── lineOperations.ts # Line-level text operations (3 tools)
└── types/ # Type definitions
└── types.d.ts
tests/ # Jest test suite
├── security.test.ts
├── security.edge-cases.test.ts
├── config.test.ts
├── stateManager.test.ts
├── fileSystemTools.test.ts
├── webResearchTools.test.ts
├── browserAutomationTools.test.ts
├── gitGithubTools.test.ts
├── databaseTools.test.ts
├── executionTools.test.ts
├── utilityTools.test.ts
├── backgroundCommands.test.ts
├── toolsProvider.test.ts
├── performanceUtils.test.ts
├── fuzzySearch.test.ts
├── workingDir.test.ts
├── findLMStudioHome.test.ts
└── i18n.test.ts