Forked from crunch3r/ai-toolbox
Forked from crunch3r/ai-toolbox
ARCHITECTURE.md
ARCHITECTURE.md
Deep dive into the AI Toolbox plugin's system architecture, design patterns, and internal workflows.
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β LM Studio Host β β β β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β β β Plugin Runner (Node.js) β β β β β β β β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β β β β β AI Toolbox Plugin β β β β β β β β β β β β ββββββββββββββββ β β β β β β β index.ts βββββ Entry Point (main function) β β β β β β β (entry) β β β β β β β ββββββββ¬ββββββββ β β β β β β β β β β β β β βΌ β β β β β β βββββββββββββββββββββββββββββββββββββββββββββββββββ β β β β β β β Core Services β β β β β β β β ββββββββββββ ββββββββββββ ββββββββββββββββββββ β β β β β β β β β config.tsβ βsecurity β βstateManager.ts β β β β β β β β β β(Zod+UI) β β .ts β β(persistence) β β β β β β β β β ββββββββββββ β(validators)β ββββββββββββββββββββ β β β β β β β β ββββββββββββ β β β β β β β β ββββββββββββ ββββββββββββ ββββββββββββββββββββ β β β β β β β β βworkingDirβ βperformancβ βpromptPreprocessor β β β β β β β β β β .ts β βeUtils.ts β β .ts β β β β β β β β β β(path mgmtβ β(caching) β β(Document RAG + β β β β β β β β β ββββββββββββ ββββββββββββ β ContextGuard) β β β β β β β β β ββββββββββββββββββββ β β β β β β β βββββββββββββββββββββββββββββββββββββββββββββββββββ β β β β β β β β β β β β ββββββββββββββββββββββββββββββββββββββββββββββββββββ β β β β β β β Tool Registration Layer β β β β β β β β βββββββββββββββββββ β β β β β β β β β toolsProvider.ts β β β β β β β β β β (factory fn) β β β β β β β β β ββββββββββ¬βββββββββ β β β β β β β βββββββββββββΌββββββββββββββββββββββββββββββββββββββββ β β β β β β β β β β β β β βββββββββββββ΄ββββββββββββββββββββββββββββββββββββββ β β β β β β β Tool Modules (22 registered files) β β β β β β β β ββββββββββ ββββββββββ ββββββββββ ββββββββββ β β β β β β β β βfileSys β βwebRes β βbrowser β β git β β β β β β β β β β (24) β β (3) β β (5) β β (15) β β β β β β β β β ββββββββββ ββββββββββ ββββββββββ ββββββββββ β β β β β β β β ββββββββββ ββββββββββ ββββββββββ ββββββββββ β β β β β β β β β datab β βbackgnd β βexec β β docParseβ β β β β β β β β β (1) β β cmd(3) β β (5) β β (1) β β β β β β β β β ββββββββββ ββββββββββ ββββββββββ ββββββββββ β β β β β β β β ββββββββββ ββββββββββ ββββββββββ ββββββββββ β β β β β β β β β image β β http β β vector β β UI β β β β β β β β β β (4) β β (3) β β RAG(7) β β Gen(3) β β β β β β β β β ββββββββββ ββββββββββ ββββββββββ ββββββββββ β β β β β β β β ββββββββββ ββββββββββ ββββββββββ β β β β β β β β β Context β βtextProcβ βAST Ref β βbgndCmdsβ β β β β β β β β β Mgmt(22)β β (3) β β factorβ β (3) β β β β β β β β β ββββββββββ ββββββββββ β (2) β β β β β β β β β βββββββββββ β β β β β β β βββββββββββββββββββββββββββββββββββββββββββββββ β β β β β βββββββββββββββββββββββββββββββββββββββββββββββββββββ β β β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β β β β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β β β External Dependencies β β β β Puppeteer β isomorphic-git β Tesseract.js β pdf-parse β β β β duck-duck-scrape β node:sqlite β node-notifier β β β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
The project uses Tsup (esbuild-based bundler) for fast, zero-config compilation to both ESM and CJS formats.
tsconfig.json defines @/* β src/* for IDE and TypeScript support.tsup.config.ts maps @ β path.resolve(__dirname, 'src') to ensure the bundler resolves aliases correctly in the final output.The former tool-discovery gateway pattern (explore_tools/execute_gateway_tool, src/tools/gatewayTools.ts; introduced v1.6.0) was abandoned in favor of direct SDK registration; the file was removed from the codebase (v1.9.10, 24.08). No gateway definitions remain under src/ β grammar-parser compatibility is handled by schema minification (toolsSchemaMinifier.ts), not tool-count gating.
Context Scoping (v1.9.1+): Entries tagged with global/project/session scope for future isolation filtering. Session entries expire after 24h and are automatically pruned during retrieval.
Heuristic Retrieval Ordering (v1.9.1+): All search/retrieval operations now apply deterministic composite scoring β recent + frequently accessed entries surface first, replacing raw insertion order.
src/toolsProvider.ts)Central registry managing all tool instances using a Declarative Registry Pattern (v1.8.2+):
Key Design Decisions:
config, , and at definition time via arrow functionssrc/stateManager.ts)Persistent state management with dynamic path resolution:
Key Features:
Session Summary Tool Flow (v1.5.15+):
Storage Format:
zlib.gzipSync(level: 9) β base64 encoding β stored in StateManager (typically achieves ~30% size reduction)Working Directory Integration:
StateManager initializes via getMemoryFilePath() β resolves to {current_working_dir}/.session_context/.ai_toolbox_memory.msgpacksrc/tools/contextManagementTools.ts)Persistent context storage for session tracking:
One-shot checkpoint lifecycle (v1.9.20, ARC-RESTORE 16.10): entries tagged verification_checkpoint are single-use reader instructions - mergeEntriesWithTiers defaults no-scope checkpoint-tagged writes to scope 'session' with a 24 h TTL and inline reap in addEntry; explicit writer scope is respected (no silent override). Restore-side rendering adds per-entry age labels, a top-N substantive cap, and the additive context_stale_fraction JSON field (fraction of entries >3 d) while preserving the boolean staleness contract.
Key Features:
Heuristic Scoring Formula (v1.9.1):
src/security.ts)Multi-layer security pipeline:
src/workingDir.ts)Mutable base path for all file operations:
β οΈ Status:
src/tools/gatewayTools.tshas been removed from the codebase (v1.9.10 session, 24.08) and no longer exists anywhere in the repository. The gateway pattern was abandoned in favor of direct SDK registration + schema minification (v1.8.0+). The following describes the design for historical reference only.
Purpose: Prevent LLM tool-bloat crashes by providing a single entry point for tool discovery and execution, reducing the initial grammar schema payload from ~132 tools to just 2.
Sending all 88+ tools directly to llama.cpp's grammar parser caused failed to parse grammar errors due to EBNF recursion limits. The AI also struggled with overwhelming options when deciding which tool to use.
The gateway pattern was abandoned because direct SDK registration with schema minification proved more effective. No integration is required β and since 24.08 (v1.9.10 session) gatewayTools.ts no longer exists in the repository at all; it was never imported or used, so its removal changes nothing functionally.
| Cache | TTL | Max Entries | Purpose |
|---|---|---|---|
| Fuzzy Search | 60s | 100 | File name similarity results |
| Web Requests | 30s | 50 | HTTP responses |
Heavy dependencies loaded on first use:
ripgrep tool ONLY (WASM build of rg; lazy dynamic import on first use, see Β§5 below). no longer consumes it β B' phase-1 prefilter removed FIX-34a 13.09src/utils/ripgrepEngine.ts) β current state (standalone ripgrep only)The engine is a self-contained, worker-isolated ripgrep runner with ONE live consumer (its former second consumer β pattern_scan's B' prefilter β was removed in FIX-34a 13.09):
Design points (full contracts in the module header):
pattern_scan β pure full-JS walk; does not call this engine (B' prefilter removed FIX-34a 13.09); its scan guard + abort contract live in the watchdog note above.Priority Order: Working Dir β Plugin Root β In-Memory RAM
Impact: Eliminates cross-project memory bleed β session summaries and memory entries are always project-specific when available locally. Existing SDK global memory calls preserved as fallbacks for backward compatibility.
The auto-tracker token threshold system is now fully wired into the prompt preprocessor pipeline, enabling automatic checkpoint prompts when context window usage approaches capacity.
FSM States: IDLE β THRESHOLD_REACHED (prompt generated) β CONFIRMED (saved) or DECLINED β IDLE (reset).
Pending checkpoint warnings persist across unrelated state transitions β cleared only on consume/reply/compress paths, and injected into every preprocessor return path while pending (Fix C).
Impact: Users now receive actionable warnings when token usage approaches context window capacity, with confirmation flow to save session memory before potential overflow. Auto-tracked decisions, completions, and error fixes are flushed to persistent storage during checkpoint saves.
chat used Log Field (FIX #20, v1.9.9+)Per-turn tool loops accumulate token deltas without waiting for the next full history count:
+delta β turn total β chat used.| chat used β N tok field is emitted only when the turn-start baseline > 0; it is omitted if the ContextGuard recount fails.Two-tier context-save contract: TIER 1 @75% (autoTrackTokenThreshold) stays an interactive YES/NO prompt on the user's next message (AutoTracker FSM, unchanged above); TIER 2 @contextGuardCompressionPercent (default 90 β the same value PART B compresses at on the NEXT user message) now also fires a FORCED structured session-memory save inside the tool loop, because nobody is listening mid-turn to answer tier-1's prompt and usage can run past 100% before the next preprocess() runs. Owner spec: save ONLY (no mid-turn history compression β that stays at PART B), once per turn (one-shot latch), non-fatal (a save failure never breaks or delays a tool call). No new config key β tier 2 rides the existing contextGuardEnabled + contextGuardCompressionPercent.
Wiring: promptPreprocessor.ts publishes a reference to the current message list at the SAME finally point as TokenStatsManager.setTurnEvaluation() (post-compression when compression ran; cleared per turn by resetMidLoopDelta()) and exposes the guard instance via getContextGetter() β a one-way read next to the existing setter-only setContextGuard() (sole production setter: index.ts). The wrapper is the first external reader; callers handle null AND keep their own method gate for stubs. Pinned by tests/tier2MidLoopSave.test.ts: inclusive boundary, once-per-turn idempotency + re-arm on a new provider run, strict gate matrix, save-failure non-fatality, span-guard skip with latch hold, footer-OFF independence (the pre-fix defect class), and the suppression-warn regression pin.
FIX A closes a gap where threshold crossings armed NO state across turns (usage dropping below threshold after compression/context-increase vanished silently); FIX B fills the mid-loop band between tier-2's @~90% forced SAVE and the model's 100% hard stop β a save never shrinks context:
Full ladder after this arc: TIER 1 @75% interactive prompt + Fix-A one-shot notice on successful crossings Β· TIER 2 @contextGuardCompressionPercent (~90) forced SAVE mid-loop Β· FIX B @95 pre-emptive COMPRESSION mid-loop Β· PART B at the next user message β the boundary path stays untouched for turns that never cross mid-loop.
Deterministic context-limit protection on the READ path + a budget-relative spill cap in the wrapper. Decision core: src/budgetGuard.ts (PURE β no fs writes, no store mutation; READ-ONLY + FAIL-OPEN by invariant). All three mechanisms ride the SAME shared projection source as the wrapper's other consumers (TokenStatsManager.getTurnBaseline() + mid-loop delta), so the guard fires conservatively BEFORE the hard limit.
Full protection ladder after this arc (rung order deliberate, stays intact): TIER 1 @75% interactive prompt β M1 advisory @80% (read path only) β M3 auto-persist @85% (insurance checkpoint) β TIER 2 @~90% forced save (user-configured) β M2 budget spill edge @92% (= limitΓ(1βmargin)) β FIX B @95% pre-emptive compression mid-loop β PART B at the next user message.
Invariants: default is enabled=true; explicit enabled=false β every entry point short-circuits to legacy behavior byte-identically β rollback without a code revert still available by config alone. No published baseline/limit (maxContextTokens = 0 before the first preprocess()) β all gates silent, exactly like the wrapper mid-loop gates. M1 fail-open on stat errors / unknown sizes; unexpected exceptions never break an otherwise valid read.
Hermetic gate: tests/budgetGuard.test.ts (pinned densities 1.307β1.670 c/tok band, advisory thresholds, pointer completeness, M3 one-shotness, enabled=false byte-equality) + the existing wrapper suites.
Visual Indicator Example:
contextGuardForceSummaryOnCompress)When compression fires, an arc-C step runs after PART B-2's checkpoint-warning settlement and BEFORE compressHistory() (src/promptPreprocessor.ts), so a durable structured summary exists before the history it describes is destroyed:
Non-fatal by design: PART B's telemetry snapshot + F1 checkpoint notice remain byte-identical on the arc-C path; ordering and all failure sides are pinned by tests/arcCCompressionWiring.test.ts.
In v1.8.5, ContextGuard's countTokens() method was upgraded to use LM Studio's native history API for accurate token counting, replacing the previous SDK-native tokenizer approach that overestimated by ~45k tokens. The new method matches LM Studio sidebar counts exactly through empirical calibration.
Engineering Notes:
getLength(), at(i), getText() properly extract all message content including tool calls β no more silent zeros from broken casting.Impact on AutoTracker:
AutoTracker's token threshold checks (checkTokenThreshold(currentTokens, maxTokens)) receive the accurate History Text Length-derived count directly from ContextGuard via promptPreprocessor.ts. No additional changes were required in autoTracker.ts β the end-to-end threshold pipeline now fires precisely at configured percentages (e.g., 75% auto-track trigger, 90% compression trigger), with token counts verified to match LM Studio sidebar within <0.5% deviation.
The LM Studio local REST endpoint (src/lmStudioApi.ts) was evaluated early and rejected: server availability is fragile (port detection, model load/switch/context-clear windows), failures throw into production logs, and it adds an external network dependency where the native history API always succeeds while a model is loaded. The native history API + empirical ratio approach replaced it for deterministic accuracy.
The native history API approach was chosen because it provides deterministic accuracy without introducing fragile network dependencies. The Γ 0.24 ratio is not a hack β it's an empirically calibrated bridge between the raw character count and LM Studio's internal token counting logic, verified across thousands of real-world interactions with <0.5% deviation from sidebar display.
Used only when history data is unavailable (rare edge case); primary method remains History Text Length Γ 0.24.
security.ts imports from workingDir.ts (not vice versa)stateManager.ts has minimal logger (no index.ts import)The Zod schema (src/config.ts) defines all plugin settings:
Each field maps to a UI element in LM Studio's settings panel via createConfigSchematics().
src/tools/recodeTool/)The modular "Recode" architecture was introduced in v1.5.34 to support AST-based code transformations:
Engine Features:
runRecodeEngine() function.bak file creation before modificationsplugins: ['typescript'])Integration: The existing refactor_code tool delegates unused_import_cleanup operation to the new engine via lazy-load import in toolsProvider.ts.
The following rule files are defined in the proposal but NOT yet created:
rules/securityHardener.ts β Security pattern hardeningrules/duplicateCodeExtraction.ts β Duplicate code detection & extractionThe following rule files were implemented in the v1.9.8 window but never wired into any tool operation β removed as Tier-1 dead code on 01.09.2026 (rules/unusedImports.ts is the only live rule; see CHANGELOG_v2.md entry [01.09.2026 ~18:30]):
rules/deadCodeDetection.ts, rules/modulePathNormalization.ts, rules/typeInference.ts, rules/asyncModernizer.tsAll tool categories are now fully registered in toolsProvider.ts using the declarative registry pattern:
| Category | File(s) | Tool Count | Registered? | Default State |
|---|---|---|---|---|
| File System | fileSystemTools.ts (+ patternScan.ts engine; + line_operations folded in from textProcessingTools.ts, 23.09 Q6) | 24 | β Yes | Enabled |
| Web Research | webResearchTools.ts | 3 | β Yes | Enabled (rag_web_content under Vector RAG since v1.9.10) |
| Browser Automation | browserAutomationTools.ts | 5 | β Yes | Disabled |
| Git & GitHub | gitGithubTools.ts | 15 | β Yes | Disabled |
| Database | databaseTools.ts | 1 | β Yes | Disabled |
| Document Parsing | documentTools.ts | 1 | β Yes | Enabled |
| Background Commands | backgroundCommandTools.ts | 3 | β Yes | Disabled |
| Image Processing | imageProcessingTools.ts | 4 | β Yes | Enabled |
| HTTP Client | httpClientTools.ts | 3 | β Yes | Disabled |
| Vector RAG | vectorRagTools.ts | 7 | β Yes | Enabled |
| UI Generation | uiGenerationTools.ts | 3 | β Yes | Enabled (flipped from disabled, owner decision 10.10) |
| Context Management | contextManagementTools.ts | 22 (re-audited 28.09 vs live registry: 12 core memory/session + session-index & project-registry 9 + composite restore_session_context; was "12" β "20" at the 05.09 audit) | β Yes | Enabled |
| Text Processing | textProcessingTools.ts (β line_operations folded to fileSystemTools.ts, 23.09 Q6) | 3 | β Yes | Enabled |
| AST Refactoring | refactorCodeTools.ts | 1 (was "2" β unusedImports is a recode-engine rule object, not a registered tool; audited 05.09) |
Locale β code parity: the locale sets (all five) carry 115 entries / 113 distinct names β full parity with live code (
read_document+rag_web_contentare served by two categories each; four ghost entries purged, four utility tools added in the 29.09 closure). Locale entries are data, not behavior. 07.11 delta (CLOSED, re-audit done):apply_patchentries added to all five sets β each set now carries 120 entry lines / 118 distinct names; every toolName documented in TOOLS_REFERENCE has an entry in all five sets (diff-verified).
Note: all previously "unregistered" utility tool categories (backup, data visualization, line operations, markdown preview) are registered under the
utilityconfig key. The former gateway file (gatewayTools.ts) no longer exists β direct SDK registration + schema minification handles grammar-parser compatibility.
Five major architectural improvements inspired by graphify repository analysis β confidence-tagged results, hub-exclusion clustering, project auto-detection, context tier provenance, and cluster-aware tool priority ranking.
src/types/confidenceTypes.ts)Typed confidence metadata for all tool execution outputs following graphify's confidence-tagging pattern.
Root Cause Addressed: Prior to this fix, LLMs had no way to distinguish between deterministic results and inferred outputs, leading to over-trusting of low-confidence semantic similarity scores or fallback-path results. This pattern follows graphify_integration_analysis.md Section 1 (Confidence-Tagged Results).
src/utils/hubExclusionClustering.ts)Louvain community detection with hub-exclusion for architectural transparency and refactoring guidance.
Root Cause Addressed: Prior to this feature, there was no systematic way to analyze module dependency structure. Hub-exclusion clustering enables architectural visibility with modularity scoring, cluster density metrics, and hub identification β all running synchronously under 10ms for typical plugin dependency graphs.
src/projectAutoDetect.ts)Automatically detects and registers projects in the cross-project registry when searches return empty results.
searchWithAutoRegister() & initializeProjectDetection() (v1.9.8+)Both functions are deprecated as of v1.9.8+:
searchWithAutoRegister(): No longer called from any code path. Registration now requires explicit user confirmation via the register_project tool with confirmed path.initializeProjectDetection(): Removed from startup flow in index.ts. Added explanatory comment: "NO AUTO-REGISTRATION ON STARTUP". Projects must be registered explicitly.promptPreprocessor.ts) + Registry Sync (_syncFromSessionMemory())This two-layer approach eliminates the clarification loop:
_syncFromSessionMemory(): Lazy auto-sync from session memory when search_projects or get_project_info is called β ensures registered projects are always up-to-date without explicit user confirmationRoot Cause Addressed: Prior to this fix, the "ai-toolbox not found" issue occurred when cross-project registry searches returned empty results β no auto-discovery mechanism existed. User-mentioned project names were not used as registration signals, causing failed lookups even when the project was clearly present in CWD.
src/contextTiers.ts)Typed provenance markers for tier-scoped context replacement following graphify's build_merge pattern.
Root Cause Addressed: Prior to this feature, context updates replaced entire node sets without tracking data origin. This caused silent overwrites of unchanged tiers (e.g., AST nodes replaced even when only semantic insights changed). The tier-provenance system follows graphify_integration_analysis.md Section 2 (Context Tier Provenance) to enable incremental, lossless context updates.
src/tools/toolPriority.ts)Five-tier priority ranking with hub-exclusion clustering integration for intelligent tool filtering.
Root Cause Addressed: Prior to this feature, all enabled tools were sent to the LLM without priority ordering. When tool count exceeded grammar parser limits (llama.cpp EBNF recursion), there was no intelligent way to decide which tools to prune β alphabetical sorting was arbitrary and could exclude critical file system tools while keeping low-usage backup tools. The cluster-aware priority system ensures architecturally important modules (high centrality) are retained first.
The tool priority system integrates with existing schema minification pipeline (toolsSchemaMinifier.ts) for intelligent filtering when needed:
toolsProvider() (declarative registry pattern, v1.8.2+)src/tools/projectAutoDetect.ts + registryManager) β NEW (v1.9.8+)Automatic synchronization of project registry entries from session memory decisions.
| Tool | Sync Trigger | Purpose |
|---|---|---|
search_projects | _syncFromSessionMemory() before query | Ensures registry includes projects from past decisions |
get_project_info | _syncFromSessionMemory() before lookup | Same β prevents stale registry entries |
register_project tool). If the user's session memory contained references to a project but the registry was empty, the AI would still fail to find it. The _syncFromSessionMemory() lazy sync ensures that any project mentioned in past decisions is automatically added to the registry when needed β eliminating false negatives from stale registries.src/)projectAutoDetect.ts β Project Auto-Detection & Registration module (automatic CWD detection, name normalization, fuzzy matching)contextTiers.ts β Context Tier Provenance System (typed _origin: 'ast' | 'semantic' markers for tier-scoped replacement)src/tools/)toolPriority.ts β Cluster-Aware Tool Priority System (5-tier ranking with hub-exclusion clustering integration, centrality scoring)src/types/)confidenceTypes.ts β Confidence-Tagged Results types (EXTRACTED | INFERRED | AMBIGUOUS + provenance tracking)src/utils/)tests/confidenceTypes.test.ts β Confidence-tagged results validation (determineConfidence, createToolResult, createErrorResult)tests/hubExclusionClustering.test.ts β Hub-exclusion clustering algorithm verification (83 tests covering graph construction, hub identification, Louvain convergence, majority-vote reattachment, density/modularity calculations)tests/projectAutoDetect.test.ts β Project auto-detection and registration workflow (name normalization, confidence scoring, fuzzy matching, auto-registration flow)src/utils/)One pipeline enforcing the outcome taxonomy and the finalizeContent invariant across tool execution (replaces ad-hoc per-tool error handling).
success | error | deny | abstain (ToolOutcome = ToolSuccess | ToolError | ToolDeny | ToolAbstain).success that finalizes to empty content raises a FAIL-LOUD console.warn (non-blocking by design).ToolExecutionPipeline.ts (PURE β no fs/SDK/globals)describeError() (25.09 lint close): safe rendering of unknown error values β string passthrough / Error β "name: message" / JSON.stringify (circular-safe fallback) / 'Unknown error'. Never relies on Object default stringification ([object Object] would mask real tool errors; the prior String(err ?? β¦) form also tripped @typescript-eslint/no-base-to-string).withPipeline.ts (process-level wiring)All file-modifying tools now use the shared atomicWrite utility (src/utils/atomicWrite.ts) for crash-resilient, async file operations.
atomicWrite)atomicWriteBinaryFile)For source code safety, refactorCodeTools and recodeEngine implement rollback-on-failure:
All previously synchronous file-write tools converted to async with shared atomicWrite:
| Module | Tools Affected | Write Pattern | Rollback? |
|---|---|---|---|
lineOperations.ts β REMOVED 23.09 NEXT-REV | delete_lines [deleted]; line_operations already lived in textProcessingTools.ts per v1.8.9 changelog (prior attribution here was incorrect) | async β atomicWrite | No |
refactorCodeTools.ts | rename_identifier, move_function, extract_function, unused_import_cleanup | async β atomicWrite + .bak backup | β Yes |
utilityTools.ts | ~25 tools (backup, chart, etc.) | All async β atomicWrite | No |
dataVisualizationTools.ts | generate_chart | async β atomicWriteBinaryFile | No |
imageProcessingTools.ts | attachment temp-file materialization (resolveAttachmentFile) | async β atomicWriteBinaryFile | No |
markdownPreviewTools.ts | markdown_preview HTML save | async β atomicWrite | No |
imageProcessingTools.ts | screenshot_desktop PNG/JPEG save | written directly by the external platform process (PowerShell/screencapture/gnome-screenshot) β no Node-side write | n/a |
uiGenerationTools.ts | UI component saves | async β atomicWrite | No |
recodeEngine.ts (recodeTool/) | AST transformation output | async β atomicWrite + .bak backup | β Yes |
writeFileSync remaining in src/tools/ directoryrenameSync remaining in src/tools/ directoryThe plugin carries a localization subsystem under src/locales/ providing tool descriptions in five languages: English (en, reference), German (de), Spanish (es β added 04.09 with the i18n Tier-2 epic), Simplified Chinese (zh-CN) and Traditional Chinese (zh-TW).
| File | Role |
|---|---|
src/locales/types.ts | Shared types: LanguageCode = 'en' | 'de' | 'es' | 'zh-CN' | 'zh-TW'; Translation { toolName, description, parameters[], example? }; ToolCategoryTranslations { categoryTitle, tools[] }; FullTranslationSet β 19 per-category blocks (audited 05.09 β was "20"; lineOperations & markdownPreview have no category of their own) + a general block (pluginName, enabledTools, disabledTools, errorPrefix, successPrefix). Since the Tier-2 completion (04.09) all 19 categories are REQUIRED in every locale set. |
src/locales/en.ts / de.ts / es.ts / zh-CN.ts / zh-TW.ts | One complete translation set per language (enTranslations, β¦). Coverage: locale data re-audited 28.09 against live code; ghosts purged + unlisted tools added 29.09 β full parity: each locale set carries exactly toolName entries ( distinct names β and are each served by two categories); order + coverage remain parity-verified against via the guards; localeβcode drift fully closed 29.09 (see drift note in Β§ Tool Registration Summary). |
src/locales/i18n.ts)src/config.ts)language: z.enum(['en', 'de', 'es', 'zh-CN', 'zh-TW']).default('en') in the // i18n Settings block; .tests/i18n.test.ts)Translation-file loading + formatting suite including the four parity guard tests added with the Tier-2 epic β all five locale sets must stay at full live-tool coverage and in exact en order, so adding or removing a tool breaks CI until every locale is updated.
Deep dive into the AI Toolbox plugin's system architecture, design patterns, and internal workflows.
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β LM Studio Host β β β β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β β β Plugin Runner (Node.js) β β β β β β β β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β β β β β AI Toolbox Plugin β β β β β β β β β β β β ββββββββββββββββ β β β β β β β index.ts βββββ Entry Point (main function) β β β β β β β (entry) β β β β β β β ββββββββ¬ββββββββ β β β β β β β β β β β β β βΌ β β β β β β βββββββββββββββββββββββββββββββββββββββββββββββββββ β β β β β β β Core Services β β β β β β β β ββββββββββββ ββββββββββββ ββββββββββββββββββββ β β β β β β β β β config.tsβ βsecurity β βstateManager.ts β β β β β β β β β β(Zod+UI) β β .ts β β(persistence) β β β β β β β β β ββββββββββββ β(validators)β ββββββββββββββββββββ β β β β β β β β ββββββββββββ β β β β β β β β ββββββββββββ ββββββββββββ ββββββββββββββββββββ β β β β β β β β βworkingDirβ βperformancβ βpromptPreprocessor β β β β β β β β β β .ts β βeUtils.ts β β .ts β β β β β β β β β β(path mgmtβ β(caching) β β(Document RAG + β β β β β β β β β ββββββββββββ ββββββββββββ β ContextGuard) β β β β β β β β β ββββββββββββββββββββ β β β β β β β βββββββββββββββββββββββββββββββββββββββββββββββββββ β β β β β β β β β β β β ββββββββββββββββββββββββββββββββββββββββββββββββββββ β β β β β β β Tool Registration Layer β β β β β β β β βββββββββββββββββββ β β β β β β β β β toolsProvider.ts β β β β β β β β β β (factory fn) β β β β β β β β β ββββββββββ¬βββββββββ β β β β β β β βββββββββββββΌββββββββββββββββββββββββββββββββββββββββ β β β β β β β β β β β β β βββββββββββββ΄ββββββββββββββββββββββββββββββββββββββ β β β β β β β Tool Modules (22 registered files) β β β β β β β β ββββββββββ ββββββββββ ββββββββββ ββββββββββ β β β β β β β β βfileSys β βwebRes β βbrowser β β git β β β β β β β β β β (24) β β (3) β β (5) β β (15) β β β β β β β β β ββββββββββ ββββββββββ ββββββββββ ββββββββββ β β β β β β β β ββββββββββ ββββββββββ ββββββββββ ββββββββββ β β β β β β β β β datab β βbackgnd β βexec β β docParseβ β β β β β β β β β (1) β β cmd(3) β β (5) β β (1) β β β β β β β β β ββββββββββ ββββββββββ ββββββββββ ββββββββββ β β β β β β β β ββββββββββ ββββββββββ ββββββββββ ββββββββββ β β β β β β β β β image β β http β β vector β β UI β β β β β β β β β β (4) β β (3) β β RAG(7) β β Gen(3) β β β β β β β β β ββββββββββ ββββββββββ ββββββββββ ββββββββββ β β β β β β β β ββββββββββ ββββββββββ ββββββββββ β β β β β β β β β Context β βtextProcβ βAST Ref β βbgndCmdsβ β β β β β β β β β Mgmt(22)β β (3) β β factorβ β (3) β β β β β β β β β ββββββββββ ββββββββββ β (2) β β β β β β β β β βββββββββββ β β β β β β β βββββββββββββββββββββββββββββββββββββββββββββββ β β β β β βββββββββββββββββββββββββββββββββββββββββββββββββββββ β β β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β β β β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β β β External Dependencies β β β β Puppeteer β isomorphic-git β Tesseract.js β pdf-parse β β β β duck-duck-scrape β node:sqlite β node-notifier β β β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
The project uses Tsup (esbuild-based bundler) for fast, zero-config compilation to both ESM and CJS formats.
tsconfig.json defines @/* β src/* for IDE and TypeScript support.tsup.config.ts maps @ β path.resolve(__dirname, 'src') to ensure the bundler resolves aliases correctly in the final output.The former tool-discovery gateway pattern (explore_tools/execute_gateway_tool, src/tools/gatewayTools.ts; introduced v1.6.0) was abandoned in favor of direct SDK registration; the file was removed from the codebase (v1.9.10, 24.08). No gateway definitions remain under src/ β grammar-parser compatibility is handled by schema minification (toolsSchemaMinifier.ts), not tool-count gating.
Context Scoping (v1.9.1+): Entries tagged with global/project/session scope for future isolation filtering. Session entries expire after 24h and are automatically pruned during retrieval.
Heuristic Retrieval Ordering (v1.9.1+): All search/retrieval operations now apply deterministic composite scoring β recent + frequently accessed entries surface first, replacing raw insertion order.
src/toolsProvider.ts)Central registry managing all tool instances using a Declarative Registry Pattern (v1.8.2+):
Key Design Decisions:
config, , and at definition time via arrow functionssrc/stateManager.ts)Persistent state management with dynamic path resolution:
Key Features:
Session Summary Tool Flow (v1.5.15+):
Storage Format:
zlib.gzipSync(level: 9) β base64 encoding β stored in StateManager (typically achieves ~30% size reduction)Working Directory Integration:
StateManager initializes via getMemoryFilePath() β resolves to {current_working_dir}/.session_context/.ai_toolbox_memory.msgpacksrc/tools/contextManagementTools.ts)Persistent context storage for session tracking:
One-shot checkpoint lifecycle (v1.9.20, ARC-RESTORE 16.10): entries tagged verification_checkpoint are single-use reader instructions - mergeEntriesWithTiers defaults no-scope checkpoint-tagged writes to scope 'session' with a 24 h TTL and inline reap in addEntry; explicit writer scope is respected (no silent override). Restore-side rendering adds per-entry age labels, a top-N substantive cap, and the additive context_stale_fraction JSON field (fraction of entries >3 d) while preserving the boolean staleness contract.
Key Features:
Heuristic Scoring Formula (v1.9.1):
src/security.ts)Multi-layer security pipeline:
src/workingDir.ts)Mutable base path for all file operations:
β οΈ Status:
src/tools/gatewayTools.tshas been removed from the codebase (v1.9.10 session, 24.08) and no longer exists anywhere in the repository. The gateway pattern was abandoned in favor of direct SDK registration + schema minification (v1.8.0+). The following describes the design for historical reference only.
Purpose: Prevent LLM tool-bloat crashes by providing a single entry point for tool discovery and execution, reducing the initial grammar schema payload from ~132 tools to just 2.
Sending all 88+ tools directly to llama.cpp's grammar parser caused failed to parse grammar errors due to EBNF recursion limits. The AI also struggled with overwhelming options when deciding which tool to use.
The gateway pattern was abandoned because direct SDK registration with schema minification proved more effective. No integration is required β and since 24.08 (v1.9.10 session) gatewayTools.ts no longer exists in the repository at all; it was never imported or used, so its removal changes nothing functionally.
| Cache | TTL | Max Entries | Purpose |
|---|---|---|---|
| Fuzzy Search | 60s | 100 | File name similarity results |
| Web Requests | 30s | 50 | HTTP responses |
Heavy dependencies loaded on first use:
ripgrep tool ONLY (WASM build of rg; lazy dynamic import on first use, see Β§5 below). no longer consumes it β B' phase-1 prefilter removed FIX-34a 13.09src/utils/ripgrepEngine.ts) β current state (standalone ripgrep only)The engine is a self-contained, worker-isolated ripgrep runner with ONE live consumer (its former second consumer β pattern_scan's B' prefilter β was removed in FIX-34a 13.09):
Design points (full contracts in the module header):
pattern_scan β pure full-JS walk; does not call this engine (B' prefilter removed FIX-34a 13.09); its scan guard + abort contract live in the watchdog note above.Priority Order: Working Dir β Plugin Root β In-Memory RAM
Impact: Eliminates cross-project memory bleed β session summaries and memory entries are always project-specific when available locally. Existing SDK global memory calls preserved as fallbacks for backward compatibility.
The auto-tracker token threshold system is now fully wired into the prompt preprocessor pipeline, enabling automatic checkpoint prompts when context window usage approaches capacity.
FSM States: IDLE β THRESHOLD_REACHED (prompt generated) β CONFIRMED (saved) or DECLINED β IDLE (reset).
Pending checkpoint warnings persist across unrelated state transitions β cleared only on consume/reply/compress paths, and injected into every preprocessor return path while pending (Fix C).
Impact: Users now receive actionable warnings when token usage approaches context window capacity, with confirmation flow to save session memory before potential overflow. Auto-tracked decisions, completions, and error fixes are flushed to persistent storage during checkpoint saves.
chat used Log Field (FIX #20, v1.9.9+)Per-turn tool loops accumulate token deltas without waiting for the next full history count:
+delta β turn total β chat used.| chat used β N tok field is emitted only when the turn-start baseline > 0; it is omitted if the ContextGuard recount fails.Two-tier context-save contract: TIER 1 @75% (autoTrackTokenThreshold) stays an interactive YES/NO prompt on the user's next message (AutoTracker FSM, unchanged above); TIER 2 @contextGuardCompressionPercent (default 90 β the same value PART B compresses at on the NEXT user message) now also fires a FORCED structured session-memory save inside the tool loop, because nobody is listening mid-turn to answer tier-1's prompt and usage can run past 100% before the next preprocess() runs. Owner spec: save ONLY (no mid-turn history compression β that stays at PART B), once per turn (one-shot latch), non-fatal (a save failure never breaks or delays a tool call). No new config key β tier 2 rides the existing contextGuardEnabled + contextGuardCompressionPercent.
Wiring: promptPreprocessor.ts publishes a reference to the current message list at the SAME finally point as TokenStatsManager.setTurnEvaluation() (post-compression when compression ran; cleared per turn by resetMidLoopDelta()) and exposes the guard instance via getContextGetter() β a one-way read next to the existing setter-only setContextGuard() (sole production setter: index.ts). The wrapper is the first external reader; callers handle null AND keep their own method gate for stubs. Pinned by tests/tier2MidLoopSave.test.ts: inclusive boundary, once-per-turn idempotency + re-arm on a new provider run, strict gate matrix, save-failure non-fatality, span-guard skip with latch hold, footer-OFF independence (the pre-fix defect class), and the suppression-warn regression pin.
FIX A closes a gap where threshold crossings armed NO state across turns (usage dropping below threshold after compression/context-increase vanished silently); FIX B fills the mid-loop band between tier-2's @~90% forced SAVE and the model's 100% hard stop β a save never shrinks context:
Full ladder after this arc: TIER 1 @75% interactive prompt + Fix-A one-shot notice on successful crossings Β· TIER 2 @contextGuardCompressionPercent (~90) forced SAVE mid-loop Β· FIX B @95 pre-emptive COMPRESSION mid-loop Β· PART B at the next user message β the boundary path stays untouched for turns that never cross mid-loop.
Deterministic context-limit protection on the READ path + a budget-relative spill cap in the wrapper. Decision core: src/budgetGuard.ts (PURE β no fs writes, no store mutation; READ-ONLY + FAIL-OPEN by invariant). All three mechanisms ride the SAME shared projection source as the wrapper's other consumers (TokenStatsManager.getTurnBaseline() + mid-loop delta), so the guard fires conservatively BEFORE the hard limit.
Full protection ladder after this arc (rung order deliberate, stays intact): TIER 1 @75% interactive prompt β M1 advisory @80% (read path only) β M3 auto-persist @85% (insurance checkpoint) β TIER 2 @~90% forced save (user-configured) β M2 budget spill edge @92% (= limitΓ(1βmargin)) β FIX B @95% pre-emptive compression mid-loop β PART B at the next user message.
Invariants: default is enabled=true; explicit enabled=false β every entry point short-circuits to legacy behavior byte-identically β rollback without a code revert still available by config alone. No published baseline/limit (maxContextTokens = 0 before the first preprocess()) β all gates silent, exactly like the wrapper mid-loop gates. M1 fail-open on stat errors / unknown sizes; unexpected exceptions never break an otherwise valid read.
Hermetic gate: tests/budgetGuard.test.ts (pinned densities 1.307β1.670 c/tok band, advisory thresholds, pointer completeness, M3 one-shotness, enabled=false byte-equality) + the existing wrapper suites.
Visual Indicator Example:
contextGuardForceSummaryOnCompress)When compression fires, an arc-C step runs after PART B-2's checkpoint-warning settlement and BEFORE compressHistory() (src/promptPreprocessor.ts), so a durable structured summary exists before the history it describes is destroyed:
Non-fatal by design: PART B's telemetry snapshot + F1 checkpoint notice remain byte-identical on the arc-C path; ordering and all failure sides are pinned by tests/arcCCompressionWiring.test.ts.
In v1.8.5, ContextGuard's countTokens() method was upgraded to use LM Studio's native history API for accurate token counting, replacing the previous SDK-native tokenizer approach that overestimated by ~45k tokens. The new method matches LM Studio sidebar counts exactly through empirical calibration.
Engineering Notes:
getLength(), at(i), getText() properly extract all message content including tool calls β no more silent zeros from broken casting.Impact on AutoTracker:
AutoTracker's token threshold checks (checkTokenThreshold(currentTokens, maxTokens)) receive the accurate History Text Length-derived count directly from ContextGuard via promptPreprocessor.ts. No additional changes were required in autoTracker.ts β the end-to-end threshold pipeline now fires precisely at configured percentages (e.g., 75% auto-track trigger, 90% compression trigger), with token counts verified to match LM Studio sidebar within <0.5% deviation.
The LM Studio local REST endpoint (src/lmStudioApi.ts) was evaluated early and rejected: server availability is fragile (port detection, model load/switch/context-clear windows), failures throw into production logs, and it adds an external network dependency where the native history API always succeeds while a model is loaded. The native history API + empirical ratio approach replaced it for deterministic accuracy.
The native history API approach was chosen because it provides deterministic accuracy without introducing fragile network dependencies. The Γ 0.24 ratio is not a hack β it's an empirically calibrated bridge between the raw character count and LM Studio's internal token counting logic, verified across thousands of real-world interactions with <0.5% deviation from sidebar display.
Used only when history data is unavailable (rare edge case); primary method remains History Text Length Γ 0.24.
security.ts imports from workingDir.ts (not vice versa)stateManager.ts has minimal logger (no index.ts import)The Zod schema (src/config.ts) defines all plugin settings:
Each field maps to a UI element in LM Studio's settings panel via createConfigSchematics().
src/tools/recodeTool/)The modular "Recode" architecture was introduced in v1.5.34 to support AST-based code transformations:
Engine Features:
runRecodeEngine() function.bak file creation before modificationsplugins: ['typescript'])Integration: The existing refactor_code tool delegates unused_import_cleanup operation to the new engine via lazy-load import in toolsProvider.ts.
The following rule files are defined in the proposal but NOT yet created:
rules/securityHardener.ts β Security pattern hardeningrules/duplicateCodeExtraction.ts β Duplicate code detection & extractionThe following rule files were implemented in the v1.9.8 window but never wired into any tool operation β removed as Tier-1 dead code on 01.09.2026 (rules/unusedImports.ts is the only live rule; see CHANGELOG_v2.md entry [01.09.2026 ~18:30]):
rules/deadCodeDetection.ts, rules/modulePathNormalization.ts, rules/typeInference.ts, rules/asyncModernizer.tsAll tool categories are now fully registered in toolsProvider.ts using the declarative registry pattern:
| Category | File(s) | Tool Count | Registered? | Default State |
|---|---|---|---|---|
| File System | fileSystemTools.ts (+ patternScan.ts engine; + line_operations folded in from textProcessingTools.ts, 23.09 Q6) | 24 | β Yes | Enabled |
| Web Research | webResearchTools.ts | 3 | β Yes | Enabled (rag_web_content under Vector RAG since v1.9.10) |
| Browser Automation | browserAutomationTools.ts | 5 | β Yes | Disabled |
| Git & GitHub | gitGithubTools.ts | 15 | β Yes | Disabled |
| Database | databaseTools.ts | 1 | β Yes | Disabled |
| Document Parsing | documentTools.ts | 1 | β Yes | Enabled |
| Background Commands | backgroundCommandTools.ts | 3 | β Yes | Disabled |
| Image Processing | imageProcessingTools.ts | 4 | β Yes | Enabled |
| HTTP Client | httpClientTools.ts | 3 | β Yes | Disabled |
| Vector RAG | vectorRagTools.ts | 7 | β Yes | Enabled |
| UI Generation | uiGenerationTools.ts | 3 | β Yes | Enabled (flipped from disabled, owner decision 10.10) |
| Context Management | contextManagementTools.ts | 22 (re-audited 28.09 vs live registry: 12 core memory/session + session-index & project-registry 9 + composite restore_session_context; was "12" β "20" at the 05.09 audit) | β Yes | Enabled |
| Text Processing | textProcessingTools.ts (β line_operations folded to fileSystemTools.ts, 23.09 Q6) | 3 | β Yes | Enabled |
| AST Refactoring | refactorCodeTools.ts | 1 (was "2" β unusedImports is a recode-engine rule object, not a registered tool; audited 05.09) |
Locale β code parity: the locale sets (all five) carry 115 entries / 113 distinct names β full parity with live code (
read_document+rag_web_contentare served by two categories each; four ghost entries purged, four utility tools added in the 29.09 closure). Locale entries are data, not behavior. 07.11 delta (CLOSED, re-audit done):apply_patchentries added to all five sets β each set now carries 120 entry lines / 118 distinct names; every toolName documented in TOOLS_REFERENCE has an entry in all five sets (diff-verified).
Note: all previously "unregistered" utility tool categories (backup, data visualization, line operations, markdown preview) are registered under the
utilityconfig key. The former gateway file (gatewayTools.ts) no longer exists β direct SDK registration + schema minification handles grammar-parser compatibility.
Five major architectural improvements inspired by graphify repository analysis β confidence-tagged results, hub-exclusion clustering, project auto-detection, context tier provenance, and cluster-aware tool priority ranking.
src/types/confidenceTypes.ts)Typed confidence metadata for all tool execution outputs following graphify's confidence-tagging pattern.
Root Cause Addressed: Prior to this fix, LLMs had no way to distinguish between deterministic results and inferred outputs, leading to over-trusting of low-confidence semantic similarity scores or fallback-path results. This pattern follows graphify_integration_analysis.md Section 1 (Confidence-Tagged Results).
src/utils/hubExclusionClustering.ts)Louvain community detection with hub-exclusion for architectural transparency and refactoring guidance.
Root Cause Addressed: Prior to this feature, there was no systematic way to analyze module dependency structure. Hub-exclusion clustering enables architectural visibility with modularity scoring, cluster density metrics, and hub identification β all running synchronously under 10ms for typical plugin dependency graphs.
src/projectAutoDetect.ts)Automatically detects and registers projects in the cross-project registry when searches return empty results.
searchWithAutoRegister() & initializeProjectDetection() (v1.9.8+)Both functions are deprecated as of v1.9.8+:
searchWithAutoRegister(): No longer called from any code path. Registration now requires explicit user confirmation via the register_project tool with confirmed path.initializeProjectDetection(): Removed from startup flow in index.ts. Added explanatory comment: "NO AUTO-REGISTRATION ON STARTUP". Projects must be registered explicitly.promptPreprocessor.ts) + Registry Sync (_syncFromSessionMemory())This two-layer approach eliminates the clarification loop:
_syncFromSessionMemory(): Lazy auto-sync from session memory when search_projects or get_project_info is called β ensures registered projects are always up-to-date without explicit user confirmationRoot Cause Addressed: Prior to this fix, the "ai-toolbox not found" issue occurred when cross-project registry searches returned empty results β no auto-discovery mechanism existed. User-mentioned project names were not used as registration signals, causing failed lookups even when the project was clearly present in CWD.
src/contextTiers.ts)Typed provenance markers for tier-scoped context replacement following graphify's build_merge pattern.
Root Cause Addressed: Prior to this feature, context updates replaced entire node sets without tracking data origin. This caused silent overwrites of unchanged tiers (e.g., AST nodes replaced even when only semantic insights changed). The tier-provenance system follows graphify_integration_analysis.md Section 2 (Context Tier Provenance) to enable incremental, lossless context updates.
src/tools/toolPriority.ts)Five-tier priority ranking with hub-exclusion clustering integration for intelligent tool filtering.
Root Cause Addressed: Prior to this feature, all enabled tools were sent to the LLM without priority ordering. When tool count exceeded grammar parser limits (llama.cpp EBNF recursion), there was no intelligent way to decide which tools to prune β alphabetical sorting was arbitrary and could exclude critical file system tools while keeping low-usage backup tools. The cluster-aware priority system ensures architecturally important modules (high centrality) are retained first.
The tool priority system integrates with existing schema minification pipeline (toolsSchemaMinifier.ts) for intelligent filtering when needed:
toolsProvider() (declarative registry pattern, v1.8.2+)src/tools/projectAutoDetect.ts + registryManager) β NEW (v1.9.8+)Automatic synchronization of project registry entries from session memory decisions.
| Tool | Sync Trigger | Purpose |
|---|---|---|
search_projects | _syncFromSessionMemory() before query | Ensures registry includes projects from past decisions |
get_project_info | _syncFromSessionMemory() before lookup | Same β prevents stale registry entries |
register_project tool). If the user's session memory contained references to a project but the registry was empty, the AI would still fail to find it. The _syncFromSessionMemory() lazy sync ensures that any project mentioned in past decisions is automatically added to the registry when needed β eliminating false negatives from stale registries.src/)projectAutoDetect.ts β Project Auto-Detection & Registration module (automatic CWD detection, name normalization, fuzzy matching)contextTiers.ts β Context Tier Provenance System (typed _origin: 'ast' | 'semantic' markers for tier-scoped replacement)src/tools/)toolPriority.ts β Cluster-Aware Tool Priority System (5-tier ranking with hub-exclusion clustering integration, centrality scoring)src/types/)confidenceTypes.ts β Confidence-Tagged Results types (EXTRACTED | INFERRED | AMBIGUOUS + provenance tracking)src/utils/)tests/confidenceTypes.test.ts β Confidence-tagged results validation (determineConfidence, createToolResult, createErrorResult)tests/hubExclusionClustering.test.ts β Hub-exclusion clustering algorithm verification (83 tests covering graph construction, hub identification, Louvain convergence, majority-vote reattachment, density/modularity calculations)tests/projectAutoDetect.test.ts β Project auto-detection and registration workflow (name normalization, confidence scoring, fuzzy matching, auto-registration flow)src/utils/)One pipeline enforcing the outcome taxonomy and the finalizeContent invariant across tool execution (replaces ad-hoc per-tool error handling).
success | error | deny | abstain (ToolOutcome = ToolSuccess | ToolError | ToolDeny | ToolAbstain).success that finalizes to empty content raises a FAIL-LOUD console.warn (non-blocking by design).ToolExecutionPipeline.ts (PURE β no fs/SDK/globals)describeError() (25.09 lint close): safe rendering of unknown error values β string passthrough / Error β "name: message" / JSON.stringify (circular-safe fallback) / 'Unknown error'. Never relies on Object default stringification ([object Object] would mask real tool errors; the prior String(err ?? β¦) form also tripped @typescript-eslint/no-base-to-string).withPipeline.ts (process-level wiring)All file-modifying tools now use the shared atomicWrite utility (src/utils/atomicWrite.ts) for crash-resilient, async file operations.
atomicWrite)atomicWriteBinaryFile)For source code safety, refactorCodeTools and recodeEngine implement rollback-on-failure:
All previously synchronous file-write tools converted to async with shared atomicWrite:
| Module | Tools Affected | Write Pattern | Rollback? |
|---|---|---|---|
lineOperations.ts β REMOVED 23.09 NEXT-REV | delete_lines [deleted]; line_operations already lived in textProcessingTools.ts per v1.8.9 changelog (prior attribution here was incorrect) | async β atomicWrite | No |
refactorCodeTools.ts | rename_identifier, move_function, extract_function, unused_import_cleanup | async β atomicWrite + .bak backup | β Yes |
utilityTools.ts | ~25 tools (backup, chart, etc.) | All async β atomicWrite | No |
dataVisualizationTools.ts | generate_chart | async β atomicWriteBinaryFile | No |
imageProcessingTools.ts | attachment temp-file materialization (resolveAttachmentFile) | async β atomicWriteBinaryFile | No |
markdownPreviewTools.ts | markdown_preview HTML save | async β atomicWrite | No |
imageProcessingTools.ts | screenshot_desktop PNG/JPEG save | written directly by the external platform process (PowerShell/screencapture/gnome-screenshot) β no Node-side write | n/a |
uiGenerationTools.ts | UI component saves | async β atomicWrite | No |
recodeEngine.ts (recodeTool/) | AST transformation output | async β atomicWrite + .bak backup | β Yes |
writeFileSync remaining in src/tools/ directoryrenameSync remaining in src/tools/ directoryThe plugin carries a localization subsystem under src/locales/ providing tool descriptions in five languages: English (en, reference), German (de), Spanish (es β added 04.09 with the i18n Tier-2 epic), Simplified Chinese (zh-CN) and Traditional Chinese (zh-TW).
| File | Role |
|---|---|
src/locales/types.ts | Shared types: LanguageCode = 'en' | 'de' | 'es' | 'zh-CN' | 'zh-TW'; Translation { toolName, description, parameters[], example? }; ToolCategoryTranslations { categoryTitle, tools[] }; FullTranslationSet β 19 per-category blocks (audited 05.09 β was "20"; lineOperations & markdownPreview have no category of their own) + a general block (pluginName, enabledTools, disabledTools, errorPrefix, successPrefix). Since the Tier-2 completion (04.09) all 19 categories are REQUIRED in every locale set. |
src/locales/en.ts / de.ts / es.ts / zh-CN.ts / zh-TW.ts | One complete translation set per language (enTranslations, β¦). Coverage: locale data re-audited 28.09 against live code; ghosts purged + unlisted tools added 29.09 β full parity: each locale set carries exactly toolName entries ( distinct names β and are each served by two categories); order + coverage remain parity-verified against via the guards; localeβcode drift fully closed 29.09 (see drift note in Β§ Tool Registration Summary). |
src/locales/i18n.ts)src/config.ts)language: z.enum(['en', 'de', 'es', 'zh-CN', 'zh-TW']).default('en') in the // i18n Settings block; .tests/i18n.test.ts)Translation-file loading + formatting suite including the four parity guard tests added with the Tier-2 epic β all five locale sets must stay at full live-tool coverage and in exact en order, so adding or removing a tool breaks CI until every locale is updated.
esm + cjs (dual-package compatibility)es2020 / node platform@lmstudio/sdk, puppeteer, sharp, tesseract.js, isomorphic-git, pdf-parse, mammoth, archiver, unzipper, node-notifier, pixelmatch, pngjs.d.ts files via dts: truestateManagerbackgroundCommandManagersaveToFile()getMemoryFilePath()change_directorypersistenceEnabled === false, returns in-memory keys directly (test isolation). When enabled, reloads from disk before returning (handles working dir changes mid-session)._ready promise) to prevent race conditionsglobalprojectsessionpattern_scanripgrep tool (new 14.09 TOOL SWAP β owner directive: full replacement of the removed grep_files, AST mode included). The ENTIRE walk + match runs inside ONE worker-isolated rg process off the host thread; there is no JS fallback for this tool.pattern_scan β pure full-JS walk (no engine call; its B' prefilter was removed in FIX-34a 13.09); scan guard + WALK-ABORT contract: shared grepGuard wall deadline, PARTIAL results on abortREMOVED (FIX-34a 13.09)** β its rg-WASM search ran synchronously on the host thread (the 13.09 wedge class); pattern_scan is now a pure full-JS walk whose per-line regex eval runs INLINE on this thread, gated by isSafeRegex BEFORE any disk I/O (DE-STRAngle 16.09) with deterministic SIZE bounds (maxFileSizeBytes, maxFileLines, maxEvalLineLength); the former wall cap was removed with it (host-signal-only) and RE-ARMED 24.09 per owner order ("abort after 3 seconds") β the scan now runs under ONE shared grep guard: PATTERN_SCAN_MAX_RUN_MS = 3_000 wall deadline + host signal converging on one cooperative abort β PARTIAL results + aborted: true (WALK-ABORT 28.09: the BFS walk phase itself now checks this same guard at every directory boundary β before, only file boundaries were checked, so a slow-FS / deep-tree walk ran unbounded past the deadline; an aborted walk returns a PARTIAL target set); see Β§5 watchdog note + CHANGELOG entries [24.09] RE-ARM and [28.09] WALK-ABORT. No engine call remains in that path.ripgrep (pithings/ripgrep-node 0.3.1, ESM-only WASM build of rg) is dynamically imported on first use only; a missing or broken package degrades gracefully (the standalone tool reports the failure as a clean result) and never breaks plugin boot.GREP_FILES_MAX_RUN_MS = 3000, shared constant in src/utils/grepGuard.ts) β the engine's budget terminates a wedged worker off-thread β clean timeout result; host aborts are forwarded straight into the engine (pre-aborted settles immediately).pattern_mode: 'fixed-strings' + hint in the tool result.include_glob to scan them).--max-depth offset +1 inside the engine (rg WASM boundary semantics); omitted = unbounded.src/autoTracker.ts + src/promptPreprocessor.ts): every SUCCESSFUL mid-loop save arms a pending {usagePercent, sessionId?}; preprocess() renders it F1-style onto userPrompt via consume-once takePendingMidLoopNotice() β "π MID-LOOP CONTEXT THRESHOLD NOTICE β¦ ~X% at crossing (checkpoint id) β¦ briefly acknowledge, no YES/NO reply". The FSM is deliberately untouched (documented guard contract); the notice is independent of and co-exists with checkpointSavedNotice; a failed save arms nothing. Pinned by tests/fixA_midLoopNotice.test.ts.src/contextGuard.ts + src/toolsProvider.ts): when the wrapper's shared projection crosses PREEMPTIVE_COMPRESSION_PERCENT = 95 (fixed constant by design β no config key; it must stay below the wall even if contextGuardCompressionPercent is user-set to its schema max of 100), run the FULL PART-B pipeline mid-loop via NEW ContextGuard.compressMidLoop() β store-before-prune β PART-B telemetry snapshot β Arc-C continuity save (strict ===true, keepLast=10) β compressHistory() with the return GATED on getLastCompressionInfo()?.compressed (FAIL-LOUD log if its internal recount is below its own trigger; the wrapper's char-heuristic projection can overestimate). On success: host-history pop-to-zero/append swap via duck-typed ctx.client.llm.history(), post-compression list republished, and a countTokens recount re-published as baseline. INDEPENDENT one-shot latch (midLoopCompactionFiredThisTurn) β tier-2's @~90% save must not suppress the @95% compression and vice versa; every failure non-fatal (tool result byte-unchanged). SIBLING of tier-2 AND the footer gate inside if (baseline > 0) β fires for every payload shape regardless of the contextUsageFooter toggle. Pinned by tests/tierB_midLoopCompaction.test.ts..contenthistoryTextLength parameter is passed from promptPreprocessor.ts after native API iteration in Step 0.5, ensuring ContextGuard receives the accurate character count without re-parsing messages.src/contextGuard.ts carries a live TOKEN_SCALING_FACTOR = 1 calibration constant applied to the fallback count β at 1 it is identity; non-1 values scale per Plugin Count Γ factor β Sidebar Display.typeof m.content === 'string' β array blocks: join each block's .text with newlines β ChatMessage objects: .getText() first, then .text, JSON.stringify() only as absolute last resort. Keeps the promptString passed to model.countTokens() semantically complete on the fallback path.RecodeConfig.ruleConfigs| β Yes |
| Enabled |
| Execution | executionTools.ts | 5 | β Yes | Mixed (JS/Python: enabled, Terminal/Shell: disabled) |
| Backup Operations | backupTools.ts + cleanupBackupsTool.ts (typo ".js" fixed 05.09) | 5 | β Yes | Utility toggle |
| Data Visualization | dataVisualizationTools.ts | 1 | β Yes | Utility toggle |
| Markdown Preview | markdownPreviewTools.ts | 1 | β Yes | Utility toggle |
| Task Planning | taskPlanningTools.ts | 3 | β Yes | Enabled (default) |
| Total Registered | Code-defined total (full re-audit 28.09): 113 live tool definitions across the 22 registered module files; exposed set is toggle-dependent β GOD MODE max β 113 (read_document, rag_web_content are each served by two categories) Β· locale data: 115 entries / 113 distinct names in all five locales (29.09 ghost purge + unlisted-tool entries β as of that audit) Β· 07.11 delta (CLOSED, re-audit done): apply_patch shipped (+1 definition β GOD MODE max β 116); locale sets re-measured and updated to 120 entry lines / 118 distinct names per set (all five), apply_patch entries added β every toolName documented in TOOLS_REFERENCE now has an entry in all five sets |
hubExclusionClustering.ts β Hub-Exclusion Clustering algorithm (Louvain community detection, hub identification, majority-vote reattachment)simulation.ts REMOVED (self-executing dev script, Tier-1 dead code β clustering assertions live on in tests/hubExclusionClustering.test.ts)toolPayloadCompaction.ts β C compaction family policy (PURE): pre-summarization prune of oversized role-tool payloads (>16384 UTF-8 bytes strict) into head-2048 + tail-512 preview under the [ai_toolbox compaction] marker, opaque locator compaction://<sha256hex>; digest-pairing fail-loud skip (never mints fake digests); in-place prune with byte-savings accounting. Contract: docs/tool-consolidation-draft.md Β§9c.toolPayloadStorage.ts β C compaction family storage backend: verbatim <cwd>/.ai_toolbox/compaction/<sha256>.payload, 0o600 'wx' exclusive create + symlink refusal on every path segment, EEXIST β byte-compare idempotent reuse (mismatched bytes throw); store-before-prune default ON in promptPreprocessor.ts β any store failure aborts that turn's prune.ToolExecutionPipeline.ts β tool-execution pipeline (PURE): outcome taxonomy success/error/deny/abstain; full contract in the Pipeline Hygiene section below.withPipeline.ts β process-level pipeline wiring; detail in the Pipeline Hygiene section below.[ToolExecutionPipeline]defaultFinalizer is not a function.getPipeline()); exports withPipeline(), registerToolFinalizer(toolName, fn), resetToolGuard().toolsProvider.ts calls resetToolGuard() on EVERY provider invocation β the monotonic seen-set is per-turn by construction (turn = one toolsProvider call; test suites get isolation for free).withPipeline() yet (behavior-neutral until wired).toolsProvider.ts requires a per-file moduleNameMapper entry in jest.config.cjs β same convention as every other .js-suffixed relative import; generic rules remain banned.read_documentrag_web_contententests/i18n.test.tssrc/locales/i18n.ts | The I18nManager class (API below). |
DEFAULT_CONFIG.language = 'en'.field('language', 'select', { displayName: 'π Language', options: [English, Deutsch (German), EspaΓ±ol, Simplified Chinese, Traditional Chinese] }, DEFAULT_CONFIG.language) is the first field of createConfigSchematics() β and because schematics declaration order defines LM Studio's settings-menu order, π Language sits at the top of the menu by design (no HTML involved; the panel is generated from these declarations).I18nManager, Zod config field, settings-panel entry, parity test guards β all committed (90f7a66 zh Tier-1, 6ddb7a8 es + Tier-2 full coverage).src/locales/ is consumed from production code at two points β (a) per-turn: promptPreprocessor.ts resolves the active language (resolveTurnLanguage(): configured β host locale β OS first tag β en) and renders S-banners + answer-language line; (b) boot: uiLabelBake.ts bakes settings-panel labels from the same chain via a state file. The config enum includes 'auto'; parity is regression-locked by byte-pin tests (tests/ulv2BytePin.test.ts).npm run build # Compiles src/ β dist/ with sourcemaps
npm run typecheck # Validates types without emitting (tsc --noEmit)
npm run lint # ESLint static analysis
// index.ts
export function main(context: PluginContext) {
// 1. Register config schematics (UI toggles) β GLOBAL scope since Option A (02.11): settings persist across chats/app restarts
context.withGlobalConfigSchematics(configSchematics);
// 2. Register prompt preprocessor (Document RAG + ContextGuard)
context.withPromptPreprocessor(preprocess);
// 3. Register tools provider (all registered categories based on config)
context.withToolsProvider(toolsProvider);
// 4. Setup cleanup handlers
process.on('SIGTERM', cleanupBrowserSession);
process.on('SIGINT', cleanupBrowserSession);
}
toolsProvider() called by LM Studio SDK
β
βΌ
createToolsProvider(config, stateManager, bgCommandManager)
β
βββ StateManager(config) βββββββΊ Load state from disk
βββ BackgroundCommandManager βββΊ Initialize process tracker
βββ Declarative Registry Pattern (v1.8.2+):
β
βββ TOOL_REGISTRIES array (21 in-array `{ key: }` entries; execution tools still registered separately AFTER the loop)
β βββ Each entry captures dependencies at definition time
β βββ Single for...of loop iterates all entries
β βββ Config key gating + GOD MODE bypass
β
βββ registerFileSystemTools() βββΊ 24 tools (enabled by default)
βββ registerWebResearchTools() βββΊ 3 tools (enabled by default)
βββ registerGitTools() βββΊ 15 tools (disabled by default)
βββ registerBrowserTools() βββΊ 5 tools (disabled by default)
βββ registerDatabaseTools() βββΊ 1 tool (disabled by default)
βββ registerDocumentTools() βββΊ 1 tool (enabled by default)
βββ registerBackgroundCommandTools() ββΊ 3 tools (disabled by default)
βββ registerImageProcessingTools() ββΊ 4 tools (enabled by default)
βββ registerHttpClientTools() βββΊ 3 tools (disabled by default)
βββ registerRagTools() βββΊ 7 tools (enabled by default: rag_index_files/pdf/docx/xlsx, rag_query_vector, rag_clear_index, rag_web_content β since v1.9.2/v1.9.10)
βββ registerUiGenerationTools() βββΊ 3 tools (enabled by default)
βββ registerContextManagementTools() ββΊ **22 tools** (12 core memory/session + 9 session-index & project-registry + composite `restore_session_context`; enabled by default)
βββ registerTextProcessingTools() βββΊ 3 tools (enabled by default)
βββ registerRefactorCodeTools() βββΊ **1 tool** (`refactor_code`; `unused_import_cleanup` (a recode-engine rule object) is not a registered tool β audited 05.09, confirmed in the 28.09 re-audit; enabled by default)
βββ registerExecutionTools() βββΊ 5 tools (mixed defaults)
β
βΌ
Return Tool[] to SDK βββΊ **Code-registered total: 113 live tool definitions across the 22 registering module files** (full re-audit 28.09 β `src/toolsProvider.ts` registry + per-module `name:` scan; exposed set is toggle-dependent, GOD MODE max β 113) Β· **locale data: 115 entries / 113 distinct names in all five locales** (full parity; `read_document` + `rag_web_content` served by two categories each) Β· **07.11 delta (CLOSED, re-audit done):** + `apply_patch` shipped in `fileSystemTools.ts` β GOD MODE max β 116 (canon); all five locale sets now carry 120 entry lines / 118 distinct names each β every toolName documented in TOOLS_REFERENCE has an entry in all five sets (diff-verified, gap closed)
Session Activity Occurs
β
βΌ
auto_summarize_context() called
β
βββ Analyze tool usage patterns
βββ Detect configuration changes
βββ Identify important decisions
βββ Generate summary
βΌ
ContextStorageManager.addEntry(entry)
β
βββ Load existing entries from .ai_toolbox_memory.msgpack (Working Dir β Plugin Root fallback)
βββ Apply default scope: 'global' (v1.9.1+) unless explicitly specified
βββ Set TTL for session-scoped entries: 24h (v1.9.1+)
βββ Increment frequency counter on existing entries with matching ID
βββ Append new entry to beginning of array
βββ Limit to 1000 entries (prevent unbounded growth)
βββ Save atomically (temp file + rename)
βΌ
Persistent Storage (.ai_toolbox_memory.msgpack)
β
βββ get_context_memory(limit, type?) β Retrieve recent entries
β βββ Prune expired session entries (TTL check: 24h threshold, v1.9.1+)
β βββ Apply heuristic scoring: Recency(70%) + Frequency(30%) sort
β βββ Return top N scored entries
β
βββ search_context(query, maxResults) β Text-based search
β βββ Prune expired session entries (v1.9.1+)
β βββ Filter by title/content/tags match
β βββ Apply heuristic scoring to results
β βββ Return top N scored matches
β
βββ context_summary() β Statistics & counts
βββ delete_context_entry(id) β Remove specific entry
Session Activity Occurs
β
βΌ
auto_summarize_context() called
β
βββ Analyze tool usage patterns
βββ Detect configuration changes
βββ Identify important decisions
βββ Generate summary
βΌ
ContextStorageManager.addEntry(entry)
β
βββ Load existing entries from .ai_toolbox_memory.msgpack β .session_context/.ai_toolbox_memory.msgpack
βββ Append new entry to beginning of array
βββ Limit to 1000 entries (prevent unbounded growth)
βββ Save atomically (temp file + rename)
βΌ
Persistent Storage (.ai_toolbox_memory.msgpack)
β
βββ get_context_memory() β Retrieve recent entries
βββ search_context(query) β Text-based search
βββ context_summary() β Statistics & counts
βββ delete_context_entry(id) β Remove specific entry
// Simplified registration pattern (actual implementation uses closure-based registry)
export async function toolsProvider(ctl: ToolsProviderController): Promise<Tool[]> {
const pluginConfig = ctl.getGlobalPluginConfig(configSchematics); // Option A (02.11): global scope β settings persist across chats/app restarts
// Construct typed PluginConfig from .get() calls
const config: PluginConfig = { /* ... */ };
// Initialize managers (singleton pattern)
if (!stateManager) stateManager = new StateManager(config);
if (!backgroundCommandManager) backgroundCommandManager = new BackgroundCommandManager(config);
const tools: Tool[] = [];
// --- Declarative Registry Definition (v1.8.2+) ---
const TOOL_REGISTRIES: ToolRegistryEntry[] = [
{ key: 'fileSystem', register: () => registerFileSystemTools(config, stateManager) },
{ key: 'webSearch', register: () => registerWebResearchTools(config) },
// ... 19 more entries (21 total β audited 05.09; execution tools handled separately, not via the array)
];
// --- Registry Loop (replaces ~80 lines of if/else blocks) ---
for (const entry of TOOL_REGISTRIES) {
if (config[entry.key] || isGodMode) {
tools.push(...entry.register());
}
}
return tools;
}
class StateManager {
private state: Map<string, StateEntry>;
private maxSize: number;
private persistenceEnabled: boolean;
private memoryFile!: string; // Resolved at runtime
set(key, value): void // In-memory + async disk write
get<T>(key): T | undefined // In-memory retrieval
delete(key): boolean // In-memory + async disk update
getAllKeys(): Promise<string[]> // Waits for initialization
clear(): void // Resets in-memory state
}
// save_session_summary writes:
await stateManager.set(`${summaryId}_data`, compressed); // Base64-encoded gzip stream < 10k chars
await stateManager.set(`${summaryId}_timestamp`, Date.now());
// get_session_summary reads with backward-compatible fallback:
const keys = await stateManager.getAllKeys(); // Waits for loadFromFile(), returns all keys
const compressedData = stateManager.get(summaryKey);
try {
const decompressed = zlib.gunzipSync(Buffer.from(compressedData, 'base64')).toString('utf-8');
sessionSummary = JSON.parse(decompressed); // New format (v1.5.15+)
} catch (parseErr) {
// Fallback for legacy uncompressed summaries (pre-v1.5.15)
if (typeof compressedData === 'string' && compressedData.startsWith('{')) {
try {
sessionSummary = JSON.parse(compressedData); // Legacy format
} catch (legacyErr) {
throw new Error(`Legacy summary parsing failed: ${String(legacyErr)}`);
}
} else {
throw parseErr; // Corrupted or unknown format
}
}
export type MemoryScope = 'global' | 'project' | 'session';
class ContextStorageManager {
private storagePath: string; // .ai_toolbox_memory.msgpack
load(): Promise<ContextEntry[]>
save(entries: ContextEntry[]): Promise<void>
addEntry(entry: ContextEntry): Promise<void> // Applies default scope + TTL
getRecentEntries(limit, type?): Promise<ContextEntry[]> // Heuristic scoring applied
searchEntries(query, maxResults): Promise<ContextEntry[]> // Heuristic scoring applied
deleteEntry(id): Promise<boolean>
clearAll(): Promise<void>
getSummary(): Promise<ContextSummary>
pruneExpiredSessionEntries(): Promise<number> // TTL pruning (v1.9.1+)
}
interface ContextEntry {
id: string;
timestamp: number;
date: string;
type: 'decision' | 'pattern' | 'configuration' | 'file_change' | 'error' | 'summary';
title: string;
content: string;
tags?: string[];
scope?: MemoryScope; // NEW (v1.9.1): Context isolation
frequency?: number; // NEW (v1.9.1): Access count for scoring
ttl_ms?: number; // NEW (v1.9.1): Expiration threshold
}
// Recency Decay: Exponential decay based on age (lambda = 1 day)
const recencyFactor = Math.exp(-ageMs / (24 * 60 * 60 * 1000));
// Frequency Saturation: Prevents infinite bias toward frequently accessed entries
const frequencyFactor = freq / (freq + 5);
// Weighted composite score
return (recencyFactor * 0.7) + (frequencyFactor * 0.3);
Input β Path Validation β Binary Detection β Command Sanitization β SQL Validation
(validatePath) (isBinaryFile) (sanitizeCommand) (validateSQLQuery)
let currentWorkingDir: string = BASE_DIR;
getWorkingDir(): string
setWorkingDir(newDir: string): boolean
resetWorkingDir(): void
resolvePath(userPath: string): string
getAllowedBases(): string[]
// former src/tools/gatewayTools.ts (REMOVED from codebase 24.08; shown for historical reference)
export async function getGatewayTools(
provider: ToolsProvider,
config: PluginConfig
): Promise<Tool[]> {
const exploreTools = tool({
name: 'explore_tools',
description: 'Discover available tools and their categories...',
parameters: { category: z.string().optional() },
implementation: async (params) => {
await provider.getAvailableTools(); // Ensure registry loaded
return { success: true, categories: [...] }; // Returns category names only
}
});
const executeGatewayTool = tool({
name: 'execute_gateway_tool',
description: 'Executes a specific tool by its name...',
parameters: {
toolName: z.string(),
arguments: z.record(z.unknown())
},
implementation: async (params) => {
return await provider.executeTool(params.toolName, params.arguments); // Delegates to registry
}
});
return [exploreTools, executeGatewayTool];
}
User Message β AI calls explore_tools(category="fileSystem")
β Returns: { success: true, categories: ["read_file", "write_file", ...] }
β AI decides to use read_file
β AI calls execute_gateway_tool(toolName="read_file", arguments={file_name: "example.txt"})
β Gateway delegates to provider.executeTool("read_file", args)
β Tool executes with full validation, security checks, error handling
User Path Input
β
βββ Empty check βββββββββββββββββΊ Reject
β
βββ UNC path check (\\\) βββββββΊ Reject
β
βββ Relative path?
β β
β βββ Yes: Resolve against basePath
β β β
β β βββ Within base? ββββΊ Allow
β β βββ Outside base? βββΊ Reject
β β
β βββ No (absolute):
β β
β βββ In allowed bases? βββΊ Allow
β βββ Outside allowed? ββββΊ Reject
Command String
β
βΌ
Layer 1: Dangerous Pattern Blocking
β
βββ Null byte injection ββββββββββΊ Reject
βββ IFS tampering ββββββββββββββββΊ Reject
βββ Dangerous patterns (rm -rf, sudo, etc.) ββΊ Reject
βββ Too many pipes (>2) ββββββββββΊ Reject
βββ Multiple semicolons (>1) ββββββΊ Reject
βββ Command substitution ($(), ``) ββΊ Reject
βββ Environment modification ββββββΊ Reject
β
βΌ
Layer 2: Tool-Category Enforcement
β
βββ classifyCommand() β Set<string>
β β
β βββ git * / api.github.com β 'gitOperations'
β βββ duckduckgo / google / bing β 'webSearch'
β βββ puppeteer / playwright / chromium β 'browserAutomation'
β βββ sqlite3 / mysql / psql β 'databaseQueries'
β βββ curl / wget / http β 'httpClient'
β βββ nohup / disown / & β 'backgroundCommands'
β β
β βΌ
β Check against config toggles
β β
β βββ Category disabled + !godMode ββΊ Reject
β βββ Category enabled or godMode βββΊ Allow
β
βΌ
Allow Execution
JavaScript Code
β
βββ require() detection ββββββββββΊ Reject
βββ eval() detection βββββββββββββΊ Reject
βββ fs/child_process access ββββββΊ Reject
βββ Function constructor βββββββββΊ Reject
βββ Dynamic import() βββββββββββββΊ Reject
βββ __proto__ access βββββββββββββΊ Reject
// Stops calculating if minimum possible score drops below threshold
function levenshteinSimilarity(a: string, b: string, minScore: number): number | null {
// Quick rejection for very different lengths
if (lenDiff / maxLen > (1 - minScore)) return null;
// Two-row optimization (saves memory vs full matrix)
// Early exit when row minimum exceeds threshold
}
// Concurrency-controlled batch processing
async function findFilesAsync(dirPath, pattern, maxDepth, concurrencyLimit = 4) {
// Process directories in batches
for (const batch of batches) {
await Promise.all(batch.map(dir => searchDir(dir, depth + 1)));
}
}
User Message
β
βΌ
promptPreprocessor()
β
βββ Check temporalAwareness config
β β
β βββ Enabled?
β β
β βββ Yes: Get cached datetime (5min TTL)
β β β
β β βββ Format: Standard ([Zeit: ...]) or HEUTE IST Mode
β β β
β β βββ Append timestamp to message end
β β
β βββ No: Skip
β
βΌ
Final Prompt sent to LLM (with timestamp suffix)
User Message Arrives β Step 0: Attachments + Checkpoint Suffix
β
βΌ
Step 0.5: ContextGuard Token Counting & Auto-Tracker Threshold Check
β
βββ Count tokens via History Text Length Γ 0.25 (+10% buffer)
βββ autoTracker.checkAndGeneratePrompt() β threshold warning if β₯75% used
βββ Inject checkpointSuffix into all return paths (unified injection)
β
βΌ
Step 0.6: Auto-Tracker Checkpoint Reply Handling + Message Analysis
β
βββ Detect "YES"/"NO" (or German "JA"/"NEIN") reply to pending checkpoint prompt β flush/save memory
βββ Analyze message for tracking triggers (decisions, completions, errors)
βββ Buffer actions for later flush at next threshold checkpoint
β
βΌ
Step 0.7: Project Keyword Detection β NEW (v1.9.8+)
β
βββ Read project_registry.json from disk
βββ Extract candidate words from user message (filter stop-words)
βββ Match against registered projects with fuzzy normalization (hyphenβunderscore)
βββ If matched β Inject "β οΈ REGISTERED PROJECT DETECTED" confirm-first banner
β (Does NOT change CWD this turn; one-shot switch only after an explicit YES/JA reply in a later message)
β
βΌ
Step 1: Directory Path Detection β Unchanged
β
βββ Detect Windows/Unix/Relative paths in message
βββ If found β Inject "WORKING DIRECTORY DETECTED" confirmation prompt
β
βΌ
Step 2: Document RAG (if enabled)
β
βββ Yes: Load embedding model
β β
β βββ Process files β chunks
β β
β βββ Semantic retrieval
β β
β βββ Filter by affinity threshold
β β
β βββ Inject relevant chunks into prompt
β
βββ No: Pass through unchanged
browser_open_page(url)
β
βΌ
BrowserSessionManager.getBrowser()
β
βββ Browser exists & connected? ββββΊ Reuse
β
βββ No: Launch new Puppeteer instance
β
βββ Retry with exponential backoff (max 2)
β
βββ Reset inactivity timer (5 min)
β
βΌ
Navigate to URL
β
βββ Wait for selector (optional)
β
βββ Take screenshot (optional)
β
βββ Extract text content
Session Activity Detected
β
βΌ
auto_summarize_context(sessionEvents, configChanges)
β
βββ Analyze tool usage patterns (>3 uses = frequent pattern)
βββ Track configuration changes
βββ Identify important decisions
βββ Generate session summary
βΌ
ContextStorageManager.addEntry(entry)
β
βββ Load existing entries from .ai_toolbox_memory.msgpack
βββ Prepend new entry to array
βββ Enforce 1000-entry limit
βββ Atomic save (temp file + rename)
βΌ
Persistent Storage (.ai_toolbox_memory.msgpack)
β
βββ get_context_memory(limit, type?) β Retrieve entries
βββ search_context(query, maxResults) β Text-based search
βββ context_summary() β Statistics & counts
βββ delete_context_entry(id) / clearContextMemory(confirm) β Management
User calls get_session_summary() or get_memory()
β
βΌ
1οΈβ£ Check Working Directory File:
{current_working_dir}/.session_context/.ai_toolbox_memory.msgpack
β
βββ Found? Decode msgpack β Return β
(Local-first hit)
β
βββ Missing/Empty/Corrupt? Continue...
β
βΌ
2οΈβ£ Check Plugin Root File:
{plugin_root}/.session_context/.ai_toolbox_memory.msgpack
β
βββ Found? Decode msgpack β Return β οΈ (Fallback hit)
β
βββ Missing/Empty/Corrupt? Continue...
β
βΌ
3οΈβ£ Check In-Memory State (RAM):
stateManager.get('session_summary_latest') or memory_* keys
β
βββ Found? Return β οΈ (Last resort)
βββ Not found? Return β error
User Message Arrives β Step 0.5: ContextGuard Token Counting
β
βββ autoTracker.checkAndGeneratePrompt(tokenCount, maxTokens):
β ββ Calculate usagePercentage = (effectiveTokens / maxTokens) * 100
β ββ Compare against threshold (default: 75%)
β ββ If >= threshold β Generate warning + FSM: IDLE β THRESHOLD_REACHED
β
βΌ
Warning injected into user message prompt:
β οΈ SESSION WARNING: You have reached {usage}% of your token limit...
User replies "YES" (German "JA" normalized) β Step 0.6: Checkpoint Reply Detection
β
βββ autoTracker.hasPendingWarning()? YES β
βββ replyMatch === 'YES'? YES β
βββ autoTracker.processUserReply('YES') β FSM: THRESHOLD_REACHED β CONFIRMED
βββ autoTracker.checkAndSaveTokenThreshold():
ββ flushActionsToMemory() β Save buffered decisions/completions/errors
ββ autoSaveSessionMemory() β Create checkpoint entry with token stats
User replies "NO" (German "NEIN" normalized) β Step 0.6: Checkpoint Reply Detection
β
βββ autoTracker.hasPendingWarning()? YES β
βββ replyMatch === 'NO'? YES β
βββ autoTracker.processUserReply('NO') β FSM: THRESHOLD_REACHED β DECLINED β IDLE (reset)
tool result β tokenStatsManager.recordToolResult()
βββ measures payload size β running mid-loop delta for this turn
βββ logs [AutoTracker] [DELTA] β¦ | chat used β N tok
where N = turnBaselineTokens (TokenCheck baseline at turn start) + midLoopEstTokens
next preprocess()/threshold evaluation
βββ compares against historyCount + running deltas (FIX #20),
so 75% / 90% triggers fire inside long multi-tool turns, not only between messages
tool result β toolsProvider wrapper pass (src/toolsProvider.ts, per registered tool)
β
βββ shared projection: baseline + midLoopDeltaTokens + this payload's estimate + footer self-size
β (computed ONCE per pass and consumed by BOTH the usage-footer gate and tier 2 β one number, two independent gates)
β
βββ TIER-2 block (sibling of the footer gate inside `baseline > 0`, NOT nested under the contextUsageFooter toggle):
Gate: !forcedSaveFiredThisTurn β one-shot per turn; re-armed at provider entry (= new turn, resetToolGuard convention)
AND config.contextGuardEnabled === true β strict (undefined/false/0 skip β mirrors PART B)
AND limit > 0 AND percent is a finite number > 0 β read LIVE from pluginConfig each pass (UI toggle change takes effect this run)
AND projected >= limit * percent / 100 β INCLUSIVE boundary (== threshold fires; pinned by test)
β
βββ Gate/guard fails βββΊ Skip silently (guard instance null pre-startup, or a stub without the method β PART B covers it)
β
βββ Gate passes β LATCH forcedSaveFiredThisTurn = true BEFORE the first await (a failure must not re-arm mid-turn):
βΌ
turnMessages = TokenStatsManager.getTurnMessages() ?? [] β post-compression list published at PART B's single setTurnEvaluation finally point (Step 2 hook); an absent publish coalesces to []
β
βββ Span guard: toSummarize = slice(0, len β keepLast=10) on the CALLER side (same exclusion as compressHistory), computed BEFORE its gate `toSummarize.length >= 10` (INCLUSIVE β a 20-message list fires exactly)
β βββ <10 summarizable messages (short list or absent publish) β deterministic [TIER-2] skip-log ("nothing summarizable yet"), latch still holds
β
βββ flushActionsToMemory() β buffered auto-tracked actions durable before the summary spans them (PART B parity)
βββ guard.generateSessionContinuity(toSummarize) β uncast call; same Arc-C machinery PART B uses at compression time
β βββ no structured summary (no usable model) β [TIER-2] warn, save skipped β compression still pending at next user message
β
βββ persistGeneratedSessionSummary(summary) β canonical save_session_summary writer path (.session_context/)
βββ saved=true β β
telemetry line Β· saved=false β β οΈ persist-failed telemetry line (never throws)
every failure inside the block is caught β [TIER-2] console.warn, tool result flows through byte-unchanged (non-fatal by spec)
Config (src/config.ts): nested budgetGuard block {default enabled=true since 06.11 owner order; explicit false β byte-identical legacy behavior}
+ five flat UI keys (toolsProvider.ts assembles the nested block from them β house pattern)
π‘οΈ Budget Guard master switch, advisory/spill/autopause thresholds (1β99 %), safety margin (0β50 %)
M1 β pre-call cost gate on the READ path (src/tools/fileSystemTools.ts read_file + read_file_chunked; src/tools/documentTools.ts):
projected = baseline + midLoopDelta + estimateTokensFromCharsOfClass(expectedChars, 'highEntropy')
fires when limit > 0 AND projected > limit Γ advisory_threshold_pct/100 β default 80 %
β self-describing ADVISORY returned INSTEAD of the file dump (what was withheld + numbers
+ controlled re-read hints: read_file_chunked small chunks / ripgrep / pattern_scan)
M2 β budget-aware spill cap in the toolsProvider wrapper 'A spill pattern' block (4th consumer):
edge = limit Γ (1 β safety_margin_pct/100) β default margin 8 β 92 % edge
fires when projected (incl. CTX-FOOTER self-size reserve) > edge
β inline preview capped at budgetChars = floor(remainingBudget / applied high-entropy density);
STRICTER bound wins: effectiveMax = min(absolute compaction cap, budgetChars); full payload still
stored verbatim via the existing spillTextIfNeeded() (disk store + SHA-256 locator pointer);
self-describing 'SPILLED by budget guard' marker line attached ONLY on budget spills β
absolute-cap spills stay byte-identical
M3 β insurance auto-persist at the autopause edge (latch + Arc-C pipeline in toolsProvider.ts next to its
tier-2/FIX-B siblings; pure threshold resolution + marker live in src/budgetGuard.ts):
fires when limit > 0 AND projected β₯ limit Γ autopause_threshold_pct/100 β default 85 %
AND !autoPersistFiredThisSession (session-scoped one-shot latch, deliberately never re-armed per provider run)
β Arc-C pipeline parity with tier-2: flushActionsToMemory() first, generateSessionContinuity(),
persistGeneratedSessionSummary(); marker PREPENDED to task_description ('[Budget guard] auto_saved=true β¦')
so the NEXT restore_session_context dossier shows it as its own note. M3 checkpoints only β
it caps nothing and never compresses; every failure non-fatal ([M3-AUTOSAVE] telemetry)
User Message Arrives
β
βΌ
promptPreprocessor()
β
βββ Check contextGuardEnabled config
β β
β βββ Enabled?
β β
β βββ Yes: Count tokens in history
β β β
β β βββ Below 90% threshold? βββΊ Skip compression
β β β
β β βββ Above 90% threshold?
β β β
β β βΌ
β β compressHistory(messages)
β β β
β β βββ Identify messages to compress (all except last 10)
β β βββ Send to summary model
β β β βββ Use contextGuardSummaryModel or current chat model
β β β
β β βββ Generate summary with preserved file paths/names
β β β
β β βββ Calculate tokens saved
β β β
β β βββ Inject visual indicator:
β β β
β β βββ π§ Emoji header
β β βββ Messages compressed count
β β βββ Tokens before β after (e.g., "~85k β ~42k")
β β βββ Percentage saved (e.g., "Saved ~43,000 tokens (~51%)")
β β βββ Timestamp
β β βββ Visual separator lines
β β
β βββ No: Skip ContextGuard processing
β
βΌ
Final Prompt sent to LLM (with or without compression indicator)
π§ **ContextGuard Compression Active**
βββββββββββββββββββββββββββββββββββββββββββββββ
β’ Compressed 15 message(s) into summary
β’ Tokens before: ~85k β after: ~42k
β’ **Saved ~43,000 tokens (~51%)**
β’ Timestamp: 19:15:32
βββββββββββββββββββββββββββββββββββββββββββββββ
### CONTEXT SUMMARY (from 15 messages)
[Summary content here...]
Above threshold β PART B-2 settlement (pending checkpoint warning consumed)
β
βΌ
Gate: contextGuardForceSummaryOnCompress === true β strict; undefined/false skips the step entirely
AND typeof guard.generateSessionContinuity === 'function' β legacy/stub guards skip byte-identically to pre-Arc C
AND safeMessages.length > 10 β keepLast=10 exclusion guarantees a non-empty span
β
βββ Gate fails βββΊ Skip (legacy path, unchanged)
β
βββ Gate passes (all failures below are caught β compression still runs):
βΌ
flushActionsToMemory() β buffered auto-tracked actions durable first
βΌ
generateSessionContinuity(safeMessages.slice(0, len-10), { fallbackModelId, abortSignal })
β (summary model = contextGuardSummaryModel or current chat model)
βββ Summary βββΊ persistGeneratedSessionSummary() via the canonical save_session_summary writer
β βββ saved β log β
β βββ not durable β telemetry warn (fallback)
βββ null (unparsable / no usable model) βββΊ telemetry warn (fallback)
βββ throw βββΊ caught, telemetry warn (non-fatal)
βΌ
compressHistory(safeMessages, ctl.abortSignal) β history destruction strictly last β runs in every case
User Message Arrives (ContextGuard Enabled)
β
βΌ
promptPreprocessor() β Native History API Iteration
β
βββ history.getLength() β Get message count
βββ For each message i from 0 to length-1:
β βββ msg = history.at(i) β Retrieve message by index
β βββ msg.getText() β Extract text content via getter method
β βββ msg.getToolCallRequests() β Serialize tool calls if present
β βββ msg.getToolCallResults() β Serialize tool results if present
β
βΌ
contextGuard.countTokens(messages, imageCount, modelId, systemPrompt, historyTextLength)
β
βββ PRIMARY METHOD: History Text Length Γ 0.25 ratio (v1.8.8+) β effective ~0.275 with +10% buffer
β βββ If historyTextLength provided from native API iteration:
β β β
β β βββ primaryTokenCount = Math.ceil(historyTextLength * 0.25)
β β βββ Add image tokens if applicable (+500 per image)
β β βββ Return totalTokens β β
Matches LM Studio sidebar exactly
β β
β βββ Verified at ~130K tokens for 544,578 chars β <0.5% deviation from sidebar
β
βββ VERIFIED at ~130K tokens for 544,578 chars β <0.3% deviation from sidebar (improved from <0.5%)
βββ FALLBACK: SDK-native countTokens() Γ calibration (if history unavailable)
β βββ Format messages into prompt string
β βββ Call model.countTokens(promptString) via LM Studio SDK
β βββ Apply TOKEN_SCALING_FACTOR = 1 for overhead calibration β β οΈ Legacy fallback
β
βΌ
Threshold Check: totalTokens >= tokenLimit * 0.9?
β
βββ Yes: compressHistory(messages) β Uses History Text Length Γ 0.24 for compressedPreview too
βββ No: Skip compression
index.ts
βββ toolsProvider.ts
β βββ config.ts
β βββ stateManager.ts
β βββ backgroundCommands.ts
β βββ tools/*.ts (22 live registration modules + helpers β audited 05.09)
β βββ security.ts (shared)
β βββ workingDir.ts (shared)
β βββ performanceUtils.ts (shared)
βββ config.ts
βββ promptPreprocessor.ts
βββ config.ts
ConfigSchema (Zod)
βββ Tool Gating (13 booleans)
βββ Execution Tools (4 booleans)
βββ Search Settings (3 fields)
βββ Browser Settings (2 fields)
βββ Git Settings (2 fields)
βββ Document RAG (3 fields)
βββ Security Settings (4 fields)
βββ State Management (2 fields)
βββ i18n (1 field)
βββ Notifications (1 field)
βββ Temporal Awareness (2 fields: temporalAwareness, dateFormatStyle)
βββ ContextGuard (7 fields): v1.4.2
βββ contextGuardEnabled (boolean) β Master toggle
βββ contextGuardCompressionPercent (int 50β100, default 90) β Compression trigger as % of the effective context window (07.10: replaced absolute contextGuardTokenLimit; built-in 30k fallback when no model window reported)
βββ contextGuardForceSummaryOnCompress (boolean, default true) β forced structured session-memory save at the compression trigger β flushes auto-tracked actions, then generates + persists a structured SessionSummaryData via the canonical save_session_summary writer BEFORE history destruction; strict `=== true` gate + typeof check on the guard method; non-fatal telemetry fallback on any failure
βββ contextGuardSmartReading (boolean) β Keyword-based file reading
βββ contextGuardSummaryModel (string) β Dedicated summary model name
βββ contextGuardTerminalFilterEnabled (boolean) β Terminal output filtering
βββ contextGuardTerminalFilterLength (number 100-20K) β Max terminal chars
βββ BudgetGuard (5 flat UI keys): budgetGuardEnabled (boolean, ON by default (explicit `false` β byte-identical legacy behavior); all five fields localized), budgetGuardAdvisory/Spill/AutopauseThresholdPct (int 1β99; defaults 80 / 92 / 85) + budgetGuardSafetyMarginPct (int 0β50, default 8 β M2 edge = limitΓ(1βmargin)); the nested consumer-facing `budgetGuard` block is assembled from these five in toolsProvider.ts. Full design: Β§ Budget-Guard Payload Protection above
src/
βββ index.ts # Plugin entry point
βββ toolsProvider.ts # Tool registration (conditional config gating)
βββ config.ts # Zod schema + UI schematics
βββ security.ts # Path/SQL/command validators
βββ stateManager.ts # Persistent state management
βββ workingDir.ts # Working directory manager
βββ performanceUtils.ts # Caching, async search, Levenshtein
βββ promptPreprocessor.ts # Document RAG + ContextGuard integration
βββ backgroundCommands.ts # Background process manager
βββ fuzzySearch.ts # Fuzzy file search implementation
βββ locales/ # i18n subsystem (see "Localization (i18n)" section)
β βββ types.ts # LanguageCode, Translation, ToolCategoryTranslations, FullTranslationSet
β βββ en.ts # English β reference locale (order + coverage baseline)
β βββ de.ts # German
β βββ es.ts # Spanish (added 04.09 with the i18n Tier-2 epic)
β βββ zh-CN.ts # Simplified Chinese
β βββ zh-TW.ts # Traditional Chinese
β βββ i18n.ts # I18nManager (language switching + accessors)
βββ tools/ # Tool category modules (26 source files β audited 05.09 vs v1.9.15 rev 27: 22 live register fns; utilityTools.ts exports 2 more but neither is ever called [dead]; patternScan/fileModTracker/toolPriority = helper-only)
β βββ fileSystemTools.ts # File system operations (24 tools β REGISTERED, incl. pattern_scan; + line_operations folded in 23.09, Q6)
β βββ patternScan.ts # pattern_scan search engine (clean-room module, ReDoS-gated; tool registered in fileSystemTools.ts)
β βββ webResearchTools.ts # Web research & search (3 tools β REGISTERED; rag_web_content served by vectorRagTools.ts since v1.9.10)
β βββ browserAutomationTools.ts # Browser automation (5 tools β REGISTERED)
β βββ gitGithubTools.ts # Git local ops + GitHub API (15 tools β REGISTERED)
β βββ databaseTools.ts # Database queries (1 tool β REGISTERED)
β βββ documentTools.ts # Document parsing (PDF/DOCX) (1 tool β REGISTERED)
β βββ backgroundCommandTools.ts # Background process management (3 tools β REGISTERED)
β βββ executionTools.ts # Code execution JS/Python/Terminal (5 tools β REGISTERED)
β βββ # utilityTools.ts β REMOVED (flagged dead code in tool-consolidation-draft Β§7: unregistered registerUtilityTools aggregator + duplicate memory/session tools; its live functions now sit in the dedicated modules listed below, registered under the 'utility' toggle)
β βββ imageProcessingTools.ts # Image processing & OCR (4 tools β REGISTERED)
β βββ httpClientTools.ts # HTTP client operations (3 tools β REGISTERED)
β βββ vectorRagTools.ts # Vector RAG semantic search (7 tools β REGISTERED: rag_index_files, rag_index_pdf, rag_index_docx, rag_index_xlsx, rag_query_vector, rag_clear_index, rag_web_content)
β βββ textProcessingTools.ts # Text transformation (3 tools β REGISTERED; β line_operations folded to fileSystemTools.ts 23.09, Q6)
β βββ uiGenerationTools.ts # UI component generation (3 tools β REGISTERED)
β βββ contextManagementTools.ts # Context management & tracking (12 tools β REGISTERED)
β βββ refactorCodeTools.ts # AST-based code refactoring (2 tools β REGISTERED)
β βββ dataVisualizationTools.ts # Chart generation (1 tool β REGISTERED under 'utility' toggle)
β βββ backupTools.ts # Backup & restore operations (4 tools β REGISTERED under 'utility' toggle)
β βββ cleanupBackupsTool.ts # Cleanup backups utility (1 tool β REGISTERED under 'utility' toggle)
β βββ # lineOperations.ts β REMOVED 23.09 (NEXT-REV: deprecated `delete_lines` alias stub, registered under the `utility` key since Q5=A; module + registration deleted, see docs/tool-consolidation-draft.md Β§9a)
β βββ taskPlanningTools.ts # Task planning & execution tracking (3 tools β REGISTERED)
β βββ markdownPreviewTools.ts # Markdown preview generation (1 tool β REGISTERED under 'utility' toggle)
β βββ fileModTracker.ts # File modification tracker (REGISTERED)
β βββ # networkToolsRegistry.ts β REMOVED 24.08 (was an orphan file with zero imports; deletion was tracked backlog since v1.9.3 and completed in the rag_web_content fix suite)
β βββ toolPriority.ts # Cluster-aware tool priority ranking (REGISTERED)
β βββ # backupUtils.ts / executionRegistry.ts / toolProtocolWarnings.ts / utilityRegistry.ts β REMOVED 01.09.2026 (Tier-1 dead code: zero referencers; see CHANGELOG_v2.md entry ~18:30)
β βββ restoreFromBak.ts # Backup restoration utility (REGISTERED)
β βββ attachmentManager.ts # β οΈ audited 05.09: lives in src/, NOT src/tools/ β attachment handling
β βββ browserActions.ts # β οΈ lives in src/ β browser action execution & validation
β βββ findLMStudioHome.ts # β οΈ lives in src/ β LM Studio home directory detection
β βββ lmStudioApi.ts # β οΈ lives in src/ β LM Studio REST API integration layer
β βββ budgetGuard.ts # Budget-Guard-Arc decision core (M1/M2/M3 pure gates; lives in src/ β see Β§ Data Flow β Budget-Guard Payload Protection)
βββ tokenStatsManager.ts # β οΈ lives in src/ β token statistics tracking (manager, not a tool); C family 24.09: active-turn tracking + bounded drain (`drainActiveToolTurns`, poll 250 ms / cap 15 s)
βββ types/ # Type definitions
βββ dom-augment.d.ts # DOM type augmentations for browser automation
βββ node-notifier.d.ts # Node.js notifier type declarations
βββ types.d.ts # Core shared type definitions
tests/ # Jest test suite β 84 suites / 1262 tests (v1.9.24)
βββ security.test.ts # Core security validation tests
βββ security.edge-cases.test.ts # Security boundary & edge case testing
βββ config.test.ts # Zod schema + UI schematics validation
βββ stateManager.test.ts # Persistence, path resolution, atomic writes
βββ fileSystemTools.test.ts # File system operation tests (module now 24 tools after line_operations fold 23.09 Q6; incl. pattern_scan β the folded tool has no dedicated suite yet)
βββ webResearchTools.test.ts # Multi-engine search & fetch tests
βββ browserAutomationTools.test.ts # Puppeteer session management tests
βββ gitGithubTools.test.ts # Git local ops + GitHub API tests
βββ databaseTools.test.ts # SQLite query validation tests
βββ executionTools.test.ts # JS/Python/Terminal sandboxed execution tests
βββ utilityTools.test.ts # Utility tools (backup, chart, line ops) tests
βββ backgroundCommands.test.ts # Background process management tests
βββ toolsProvider.test.ts # Declarative registry pattern integration tests
βββ performanceUtils.test.ts # Caching, async search, Levenshtein tests
βββ fuzzySearch.test.ts # Fuzzy file search similarity scoring tests
βββ workingDir.test.ts # Working directory manager path resolution tests
βββ findLMStudioHome.test.ts # LM Studio home detection & fallback tests
βββ i18n.test.ts # Translation file loading & formatting tests
βββ autoTracker.test.ts # Token threshold checkpointing & session memory tests (v1.6.6+)
βββ browserActions.test.ts # Browser action execution & validation tests
βββ fileSearch.test.ts # Recursive file search with exclusion patterns tests
βββ ripgrepEngine.test.ts # Standalone ripgrep tool + engine contract β REAL-timer suite incl. 3 s watchdog cap-fire (new 14.09)
βββ ripgrepTools.test.ts # ripgrep response-shape pins: mode echo, auto-demotion hint gating, no-match pattern_mode (new 14.09; former grep_files suites deleted in the same window)
βββ refactorCodeTools.test.ts # AST-based refactoring & dry-run diff tests (v1.5.30+)
βββ hubExclusionClustering.test.ts # Hub-exclusion clustering algorithm verification (83 tests) β NEW v1.9.8
βββ projectAutoDetect.test.ts # Project auto-detection & registration workflow tests β NEW v1.9.8
src/tools/recodeTool/
βββ rules/
β βββ unusedImports.ts β Tier 1: Implemented β
(extracted from refactorCodeTools.ts) β the only live rule as of 01.09.2026
β βββ # deadCodeDetection / modulePathNormalization / typeInference / asyncModernizer β REMOVED 01.09.2026 (Tier-1 dead code: never wired into any tool operation; see CHANGELOG_v2.md entry ~18:30)
βββ recodeEngine.ts β AST transformation orchestrator with dry-run diff support (LCS-based)
βββ recodeTypes.ts β Shared interfaces & schemas (RuleContext, RuleResult, RecodeRule)
export type Confidence = 'EXTRACTED' | 'INFERRED' | 'AMBIGUOUS';
interface ToolResultMetadata {
confidence: Confidence; // EXTRACTED (deterministic), INFERRED (semantic), AMBIGUOUS (uncertain)
provenance?: string; // e.g., "file:src/utils.ts L42", "rag_query_vector"
note?: string; // Additional context for confidence assessment
}
// Helper functions
function determineConfidence(operationType, success, fallbackUsed): Confidence;
function createToolResult<T>(data: T, confidence: Confidence, options?): { success: true; data: T & ToolResultMetadata };
function createErrorResult(message: string, provenance?): { success: false; error: string; data: ToolResultMetadata };
Build Dependency Graph β Calculate Degrees β Identify Hubs (80th percentile) β Create Non-Hub Subgraph β Louvain Community Detection β Majority-Vote Hub Reattachment β Cluster Density & Modularity Calculation
function buildDependencyGraph(sourceDirs: string[]): Map<string, Set<string>>;
function addEdge(adjacency, source, target): void;
function calculateDegrees(adjacency): Map<string, number>;
function identifyHubs(degrees, hubThresholdPercentile = 80): Set<string>;
function louvainCommunityDetection(adjacency): Map<string, number>;
function reattachHubsByMajorityVote(hubs, adjacency, nonHubCommunities): Record<string, number>;
function calculateClusterDensity(members, adjacency): number;
function calculateModularity(edges, nodeDegrees, clusterAssignments): number;
function performHubExclusionClustering(adjacency, hubThresholdPercentile = 80): HubExclusionResult;
function analyzeAiToolboxDependencies(): HubExclusionResult;
interface HubExclusionResult {
nodes: ModuleNode[]; // All modules with degrees (sorted by degree descending)
edges: Edge[]; // All connections in the graph
hubs: string[]; // Identified hub module IDs
nonHubs: string[]; // Non-hub modules for clustering
clusters: ClusterInfo[]; // Community clusters with density metrics
hubAssignments: Record<string, number>; // Hub β cluster ID mapping via majority-vote
hubThresholdPercentile: number; // Threshold used (default: 80th percentile)
modularity?: number; // Overall clustering quality [0-1]
}
interface ClusterInfo {
clusterId: number; // Sequential cluster identifier (0-indexed)
members: string[]; // Module IDs in this cluster
size: number; // Number of members
density?: number; // Internal edge density [0-1]
}
interface ProjectDetectionResult {
path: string; // Absolute path to detected project
isValid: boolean; // Whether this looks like a valid project (β₯0.3 confidence)
name?: string; // Detected project name from package.json or fallback
sourceDirs?: string[]; // Source directories within the project
confidence: number; // Detection confidence score [0-1]
}
// Confidence signals:
// - package.json exists: +0.4 (strongest signal)
// - src/ or lib/ directory exists: +0.3
// - .git directory exists: +0.1
// - tsconfig.json or jest.config.* exists: +0.2
function normalizeProjectName(name: string): string; // "ai-toolbox" β "ai_toolbox", "@lmstudio/ai-toolbox" β "lmstudio_ai_toolbox"
function generateNameVariants(name: string): string[]; // "aitoolbox" β ["aitoolbox", "ai-tool-box"]
async function searchWithAutoRegister(query, cwd, maxResults = 10): Promise<Array<{ name: string; path: string }>> {
let results = await enhancedSearchProjects(query, maxResults);
if (results.length === 0) {
const autoDetected = autoDetectAndRegister(cwd, query, true /* explicitConfirmation */);
if (autoDetected.registered) {
results = await enhancedSearchProjects(query, maxResults);
}
}
return results;
}
function initializeProjectDetection(cwd: string): void; // β οΈ DEPRECATED (v1.9.8+): No longer called from index.ts at startup. Registration requires explicitConfirmation=true via register_project tool. See src/index.ts comment: "NO AUTO-REGISTRATION ON STARTUP"
// Step 0.7 in promptPreprocessor.ts (v1.9.8+) β NEW
async function detectProjectKeywords(message: string): Promise<string | null> {
// 1. Read project_registry.json from disk
const registry = await readProjectRegistry();
// 2. Extract candidate words from user message
const words = extractCandidateWords(message); // Filter stop-words, lowercase
// 3. Fuzzy-match against registered projects (hyphenβunderscore normalization)
for (const word of words) {
for (const project of registry.projects) {
if (normalizeProjectName(word) === normalizeProjectName(project.name)) {
return `REGISTERED PROJECT DETECTED: ${project.name} at ${project.path}`;
}
}
}
return null; // No match β fall through to directory detection (Step 1)
}
// _syncFromSessionMemory() in registry manager β NEW (v1.9.8+)
async function _syncFromSessionMemory(): Promise<void> {
// Read .ai_toolbox_memory.msgpack from working dir and plugin root
const entries = await loadContextEntries();
for (const entry of entries) {
if ('decision' in entry.data && typeof entry.data.decision === 'string') {
const decision = entry.data.decision as string;
// Match project names from past decisions (e.g., "switched to ai-toolbox")
const match = extractProjectNameFromDecision(decision);
if (match) {
await registerProject(match.name, match.path);
}
}
}
}
export type ContextOrigin = 'ast' | 'semantic';
interface ContextNode {
id: string;
_origin: ContextOrigin; // "ast" (raw file/AST) or "semantic" (derived insight)
label?: string; // Human-readable label
source_file?: string; // Original file path (for ast origin)
data?: unknown; // Payload/data
timestamp?: number; // Optional timestamp for ordering
}
function replaceTier(oldNodes: ContextNode[], newNodes: ContextNode[]): ContextNode[] {
const oldAst = oldNodes.filter(n => n._origin === 'ast');
const oldSem = oldNodes.filter(n => n._origin === 'semantic');
const newAst = newNodes.filter(n => n._origin === 'ast');
const newSem = newNodes.filter(n => n._origin === 'semantic');
return [
...oldAst.filter(a => !newAst.some(n => n.id === a.id)), // Old AST not replaced
...newAst, // New AST
...oldSem.filter(s => !newSem.some(n => n.id === s.id)), // Old Sem not replaced
...newSem // New Sem
];
}
function createAstNode(id: string, data: unknown, sourceFile?: string): ContextNode;
function createSemanticNode(id: string, data: unknown, label?: string): ContextNode;
export type PriorityTier = 'critical' | 'high' | 'standard' | 'optional' | 'background';
const PRIORITY_TIER_VALUES: Record<PriorityTier, number> = {
critical: 1, // File system tools (module now 24 after 23.09 Q6 fold; line_operations deliberately kept 'standard' β owner flag)
high: 2, // Web research, execution, git operations (30+ tools β essential workflows)
standard: 3, // Browser automation, image processing, RAG, HTTP client (25+ tools β useful but not essential)
optional: 4, // Context management tools (12 tools β specialized or low-usage)
background: 5 // Backup, cleanup, chart generation, markdown preview (8+ tools β utility/maintenance)
};
interface ClusterAwarePriority extends ToolPriority {
moduleDegree?: number; // Number of dependencies/connections
isHub?: boolean; // Whether this tool's module is identified as a hub
clusterId?: number; // Assigned cluster ID from Hub-Exclusion clustering
centralityScore?: number; // Centrality score [0-1] β higher = more architecturally important
}
function computeCentralityScores(tools: ToolPriority[], clusteringResult: HubExclusionResult): Map<string, number>;
function sortToolsByClusterAwarePriority(tools: { name: string }[], clusteringResult?: HubExclusionResult): typeof tools;
function generateClusterAwareFilterReport(tools: { name: string }[], limit: number, clusteringResult?: HubExclusionResult): string;
const CATEGORY_TO_MODULE: Record<string, string | readonly string[]> = {
fileSystem: ['fileSystemTools.ts', 'tools/fileSystemTools.ts'],
webResearch: ['webResearchTools.ts', 'tools/webResearchTools.ts'],
// ... 20 categories mapped to source files with dual-name support (bare + path-prefixed)
};
// _syncFromSessionMemory() β Called lazily when search_projects or get_project_info is invoked
async function _syncFromSessionMemory(): Promise<void> {
// Load context entries from working dir + plugin root
const entries = await loadContextEntries();
for (const entry of entries) {
if ('decision' in entry.data && typeof entry.data.decision === 'string') {
const decisionText = entry.data.decision as string;
// Extract project names from past decisions
// e.g., "switched to ai-toolbox at C:\Source Code\..."
const match = extractProjectNameFromDecision(decisionText);
if (match) {
await registerProject(match.name, match.path);
}
}
}
}
// Called from search_projects tool β ensures registry is up-to-date before querying
async function searchProjects(query: string): Promise<ProjectInfo[]> {
await _syncFromSessionMemory(); // Lazy sync β NEW
return enhancedSearchProjects(query, 10);
}
promptPreprocessor() β detectProjectKeywords(message)
β
βββ If match found β Inject confirmation prompt
β ("REGISTERED PROJECT DETECTED: ai-toolbox at C:\...")
β
βββ User replies "YES" β AI calls register_project(workingDir, confirmed=true)
β
βΌ
registryManager._syncFromSessionMemory() β Ensures future searches find it
// New cross-module relationships:
toolPriority.ts β hubExclusionClustering.js (centrality scoring integration)
contextTiers.ts β (standalone β used by ContextStorageManager for tier-provenance)
projectAutoDetect.ts β DEPRECATED: No longer called from index.ts at startup. Registration requires explicitConfirmation=true via register_project tool. See src/index.ts comment: "NO AUTO-REGISTRATION ON STARTUP"
confidenceTypes.ts β all tool modules (via createToolResult<T>() helper functions)
hubExclusionClustering.ts β analysis utility (analyzeAiToolboxDependencies() pre-populated graph)
execute(toolName, args, impl, finalizer?) β { success, data?, error?, kind }
βββ monotonic guard: canonical key = tool name + deep-stable sorted-key JSON of args;
β identical call already seen this turn β kind "abstain" (no re-execution)
βββ legacy { success, data/error } return shapes normalized into the taxonomy
β (error text rendered by describeError() β see below)
βββ unhandled impl exceptions caught β kind "error" + code "UNHANDLED_EXCEPTION"
registerFinalizer(toolName, fn) // per-tool finalizeContent hook
resetGuard() // clears the monotonic seen-set (per-turn reset seam)
// module-level helpers: deny(reason), abstain(reason)
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Atomic Write Utility β
β (src/utils/atomicWrite.ts) β
β β
β ββββββββββββββββββββ ββββββββββββββββββββ β
β β atomicWrite() β β atomicWriteBinaryβ β
β β (text files) β β File() β β
β β β β (binary files) β β
β ββββββββββ¬ββββββββββ ββββββββββ¬ββββββββββ β
β β β β
β βΌ βΌ β
β βββββββββββββββββββββββββββββββββββββββββββββββ β
β β 1. Generate random temp filename β β
β β crypto.randomBytes(9) β 72-bit entropy β β
β β Format: {original}.{hex}.tmp β β
β ββββββββββββββββββββββ¬βββββββββββββββββββββββββ β
β β β
β βΌ β
β βββββββββββββββββββββββββββββββββββββββββββββββ β
β β 2. Write content to temp file β β
β β fs.promises.writeFile(tempPath, data) β β
β β (text: UTF-8 | binary: raw buffer) β β
β ββββββββββββββββββββββ¬βββββββββββββββββββββββββ β
β β β
β βΌ β
β βββββββββββββββββββββββββββββββββββββββββββββββ β
β β 3. Atomic rename (survives crashes) β β
β β fs.promises.rename(tempPath, original) β β
β β OS-level atomic operation β β
β ββββββββββββββββββββββ¬βββββββββββββββββββββββββ β
β β β
β βΌ β
β βββββββββββββββββββββββββββββββββββββββββββββββ β
β β 4. Cleanup on failure (if rename fails) β β
β β fs.promises.unlink(tempPath) β β
β βββββββββββββββββββββββββββββββββββββββββββββββ β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
Original file remains intact even if process crashes
between steps 2 and 3 β temp file orphaned but safe.
import * as crypto from 'crypto';
import { writeFile, rename, unlink } from 'fs/promises';
export async function atomicWrite(filePath: string, content: string): Promise<void> {
// Generate randomized temp filename (72-bit entropy)
const tempFile = `${filePath}.${crypto.randomBytes(9).toString('hex')}.tmp`;
try {
await writeFile(tempFile, content, 'utf-8'); // Step 1: Write to temp
await rename(tempFile, filePath); // Step 2: Atomic rename
} catch (err) {
await unlink(tempFile).catch(() => {}); // Step 3: Cleanup on failure
throw err; // Re-throw original error
}
}
export async function atomicWriteBinaryFile(
filePath: string,
buffer: Buffer
): Promise<void> {
const tempFile = `${filePath}.${crypto.randomBytes(9).toString('hex')}.tmp`;
try {
await writeFile(tempFile, buffer); // Raw buffer write (no encoding)
await rename(tempFile, filePath); // Atomic rename
} catch (err) {
await unlink(tempFile).catch(() => {}); // Cleanup on failure
throw err; // Re-throw original error
}
}
// BEFORE atomic write attempt β create .bak backup
await fs.copyFile(originalPath, `${originalPath}.bak`);
try {
await atomicWrite(originalPath, newContent); // Attempt async atomic write
} catch (err) {
// Rollback: restore from .bak backup
await fs.copyFile(`${originalPath}.bak`, originalPath);
throw new Error(`Atomic write failed β restored from backup: ${err.message}`);
}
class I18nManager {
// constructor: registers all five locale sets in a Map<LanguageCode, FullTranslationSet>; default 'en'
setLanguage(language: LanguageCode): void // throws Error on unsupported language code
getTranslations(): FullTranslationSet // the active translation set
get(category: keyof FullTranslationSet, key?: string): unknown // whole category block, or one tool entry by name
getAvailableLanguages(): LanguageCode[] // ['en', 'de', 'es', 'zh-CN', 'zh-TW']
getCurrentLanguage(): LanguageCode
}
esm + cjs (dual-package compatibility)es2020 / node platform@lmstudio/sdk, puppeteer, sharp, tesseract.js, isomorphic-git, pdf-parse, mammoth, archiver, unzipper, node-notifier, pixelmatch, pngjs.d.ts files via dts: truestateManagerbackgroundCommandManagersaveToFile()getMemoryFilePath()change_directorypersistenceEnabled === false, returns in-memory keys directly (test isolation). When enabled, reloads from disk before returning (handles working dir changes mid-session)._ready promise) to prevent race conditionsglobalprojectsessionpattern_scanripgrep tool (new 14.09 TOOL SWAP β owner directive: full replacement of the removed grep_files, AST mode included). The ENTIRE walk + match runs inside ONE worker-isolated rg process off the host thread; there is no JS fallback for this tool.pattern_scan β pure full-JS walk (no engine call; its B' prefilter was removed in FIX-34a 13.09); scan guard + WALK-ABORT contract: shared grepGuard wall deadline, PARTIAL results on abortREMOVED (FIX-34a 13.09)** β its rg-WASM search ran synchronously on the host thread (the 13.09 wedge class); pattern_scan is now a pure full-JS walk whose per-line regex eval runs INLINE on this thread, gated by isSafeRegex BEFORE any disk I/O (DE-STRAngle 16.09) with deterministic SIZE bounds (maxFileSizeBytes, maxFileLines, maxEvalLineLength); the former wall cap was removed with it (host-signal-only) and RE-ARMED 24.09 per owner order ("abort after 3 seconds") β the scan now runs under ONE shared grep guard: PATTERN_SCAN_MAX_RUN_MS = 3_000 wall deadline + host signal converging on one cooperative abort β PARTIAL results + aborted: true (WALK-ABORT 28.09: the BFS walk phase itself now checks this same guard at every directory boundary β before, only file boundaries were checked, so a slow-FS / deep-tree walk ran unbounded past the deadline; an aborted walk returns a PARTIAL target set); see Β§5 watchdog note + CHANGELOG entries [24.09] RE-ARM and [28.09] WALK-ABORT. No engine call remains in that path.ripgrep (pithings/ripgrep-node 0.3.1, ESM-only WASM build of rg) is dynamically imported on first use only; a missing or broken package degrades gracefully (the standalone tool reports the failure as a clean result) and never breaks plugin boot.GREP_FILES_MAX_RUN_MS = 3000, shared constant in src/utils/grepGuard.ts) β the engine's budget terminates a wedged worker off-thread β clean timeout result; host aborts are forwarded straight into the engine (pre-aborted settles immediately).pattern_mode: 'fixed-strings' + hint in the tool result.include_glob to scan them).--max-depth offset +1 inside the engine (rg WASM boundary semantics); omitted = unbounded.src/autoTracker.ts + src/promptPreprocessor.ts): every SUCCESSFUL mid-loop save arms a pending {usagePercent, sessionId?}; preprocess() renders it F1-style onto userPrompt via consume-once takePendingMidLoopNotice() β "π MID-LOOP CONTEXT THRESHOLD NOTICE β¦ ~X% at crossing (checkpoint id) β¦ briefly acknowledge, no YES/NO reply". The FSM is deliberately untouched (documented guard contract); the notice is independent of and co-exists with checkpointSavedNotice; a failed save arms nothing. Pinned by tests/fixA_midLoopNotice.test.ts.src/contextGuard.ts + src/toolsProvider.ts): when the wrapper's shared projection crosses PREEMPTIVE_COMPRESSION_PERCENT = 95 (fixed constant by design β no config key; it must stay below the wall even if contextGuardCompressionPercent is user-set to its schema max of 100), run the FULL PART-B pipeline mid-loop via NEW ContextGuard.compressMidLoop() β store-before-prune β PART-B telemetry snapshot β Arc-C continuity save (strict ===true, keepLast=10) β compressHistory() with the return GATED on getLastCompressionInfo()?.compressed (FAIL-LOUD log if its internal recount is below its own trigger; the wrapper's char-heuristic projection can overestimate). On success: host-history pop-to-zero/append swap via duck-typed ctx.client.llm.history(), post-compression list republished, and a countTokens recount re-published as baseline. INDEPENDENT one-shot latch (midLoopCompactionFiredThisTurn) β tier-2's @~90% save must not suppress the @95% compression and vice versa; every failure non-fatal (tool result byte-unchanged). SIBLING of tier-2 AND the footer gate inside if (baseline > 0) β fires for every payload shape regardless of the contextUsageFooter toggle. Pinned by tests/tierB_midLoopCompaction.test.ts..contenthistoryTextLength parameter is passed from promptPreprocessor.ts after native API iteration in Step 0.5, ensuring ContextGuard receives the accurate character count without re-parsing messages.src/contextGuard.ts carries a live TOKEN_SCALING_FACTOR = 1 calibration constant applied to the fallback count β at 1 it is identity; non-1 values scale per Plugin Count Γ factor β Sidebar Display.typeof m.content === 'string' β array blocks: join each block's .text with newlines β ChatMessage objects: .getText() first, then .text, JSON.stringify() only as absolute last resort. Keeps the promptString passed to model.countTokens() semantically complete on the fallback path.RecodeConfig.ruleConfigs| β Yes |
| Enabled |
| Execution | executionTools.ts | 5 | β Yes | Mixed (JS/Python: enabled, Terminal/Shell: disabled) |
| Backup Operations | backupTools.ts + cleanupBackupsTool.ts (typo ".js" fixed 05.09) | 5 | β Yes | Utility toggle |
| Data Visualization | dataVisualizationTools.ts | 1 | β Yes | Utility toggle |
| Markdown Preview | markdownPreviewTools.ts | 1 | β Yes | Utility toggle |
| Task Planning | taskPlanningTools.ts | 3 | β Yes | Enabled (default) |
| Total Registered | Code-defined total (full re-audit 28.09): 113 live tool definitions across the 22 registered module files; exposed set is toggle-dependent β GOD MODE max β 113 (read_document, rag_web_content are each served by two categories) Β· locale data: 115 entries / 113 distinct names in all five locales (29.09 ghost purge + unlisted-tool entries β as of that audit) Β· 07.11 delta (CLOSED, re-audit done): apply_patch shipped (+1 definition β GOD MODE max β 116); locale sets re-measured and updated to 120 entry lines / 118 distinct names per set (all five), apply_patch entries added β every toolName documented in TOOLS_REFERENCE now has an entry in all five sets |
hubExclusionClustering.ts β Hub-Exclusion Clustering algorithm (Louvain community detection, hub identification, majority-vote reattachment)simulation.ts REMOVED (self-executing dev script, Tier-1 dead code β clustering assertions live on in tests/hubExclusionClustering.test.ts)toolPayloadCompaction.ts β C compaction family policy (PURE): pre-summarization prune of oversized role-tool payloads (>16384 UTF-8 bytes strict) into head-2048 + tail-512 preview under the [ai_toolbox compaction] marker, opaque locator compaction://<sha256hex>; digest-pairing fail-loud skip (never mints fake digests); in-place prune with byte-savings accounting. Contract: docs/tool-consolidation-draft.md Β§9c.toolPayloadStorage.ts β C compaction family storage backend: verbatim <cwd>/.ai_toolbox/compaction/<sha256>.payload, 0o600 'wx' exclusive create + symlink refusal on every path segment, EEXIST β byte-compare idempotent reuse (mismatched bytes throw); store-before-prune default ON in promptPreprocessor.ts β any store failure aborts that turn's prune.ToolExecutionPipeline.ts β tool-execution pipeline (PURE): outcome taxonomy success/error/deny/abstain; full contract in the Pipeline Hygiene section below.withPipeline.ts β process-level pipeline wiring; detail in the Pipeline Hygiene section below.[ToolExecutionPipeline]defaultFinalizer is not a function.getPipeline()); exports withPipeline(), registerToolFinalizer(toolName, fn), resetToolGuard().toolsProvider.ts calls resetToolGuard() on EVERY provider invocation β the monotonic seen-set is per-turn by construction (turn = one toolsProvider call; test suites get isolation for free).withPipeline() yet (behavior-neutral until wired).toolsProvider.ts requires a per-file moduleNameMapper entry in jest.config.cjs β same convention as every other .js-suffixed relative import; generic rules remain banned.read_documentrag_web_contententests/i18n.test.tssrc/locales/i18n.ts | The I18nManager class (API below). |
DEFAULT_CONFIG.language = 'en'.field('language', 'select', { displayName: 'π Language', options: [English, Deutsch (German), EspaΓ±ol, Simplified Chinese, Traditional Chinese] }, DEFAULT_CONFIG.language) is the first field of createConfigSchematics() β and because schematics declaration order defines LM Studio's settings-menu order, π Language sits at the top of the menu by design (no HTML involved; the panel is generated from these declarations).I18nManager, Zod config field, settings-panel entry, parity test guards β all committed (90f7a66 zh Tier-1, 6ddb7a8 es + Tier-2 full coverage).src/locales/ is consumed from production code at two points β (a) per-turn: promptPreprocessor.ts resolves the active language (resolveTurnLanguage(): configured β host locale β OS first tag β en) and renders S-banners + answer-language line; (b) boot: uiLabelBake.ts bakes settings-panel labels from the same chain via a state file. The config enum includes 'auto'; parity is regression-locked by byte-pin tests (tests/ulv2BytePin.test.ts).npm run build # Compiles src/ β dist/ with sourcemaps
npm run typecheck # Validates types without emitting (tsc --noEmit)
npm run lint # ESLint static analysis
// index.ts
export function main(context: PluginContext) {
// 1. Register config schematics (UI toggles) β GLOBAL scope since Option A (02.11): settings persist across chats/app restarts
context.withGlobalConfigSchematics(configSchematics);
// 2. Register prompt preprocessor (Document RAG + ContextGuard)
context.withPromptPreprocessor(preprocess);
// 3. Register tools provider (all registered categories based on config)
context.withToolsProvider(toolsProvider);
// 4. Setup cleanup handlers
process.on('SIGTERM', cleanupBrowserSession);
process.on('SIGINT', cleanupBrowserSession);
}
toolsProvider() called by LM Studio SDK
β
βΌ
createToolsProvider(config, stateManager, bgCommandManager)
β
βββ StateManager(config) βββββββΊ Load state from disk
βββ BackgroundCommandManager βββΊ Initialize process tracker
βββ Declarative Registry Pattern (v1.8.2+):
β
βββ TOOL_REGISTRIES array (21 in-array `{ key: }` entries; execution tools still registered separately AFTER the loop)
β βββ Each entry captures dependencies at definition time
β βββ Single for...of loop iterates all entries
β βββ Config key gating + GOD MODE bypass
β
βββ registerFileSystemTools() βββΊ 24 tools (enabled by default)
βββ registerWebResearchTools() βββΊ 3 tools (enabled by default)
βββ registerGitTools() βββΊ 15 tools (disabled by default)
βββ registerBrowserTools() βββΊ 5 tools (disabled by default)
βββ registerDatabaseTools() βββΊ 1 tool (disabled by default)
βββ registerDocumentTools() βββΊ 1 tool (enabled by default)
βββ registerBackgroundCommandTools() ββΊ 3 tools (disabled by default)
βββ registerImageProcessingTools() ββΊ 4 tools (enabled by default)
βββ registerHttpClientTools() βββΊ 3 tools (disabled by default)
βββ registerRagTools() βββΊ 7 tools (enabled by default: rag_index_files/pdf/docx/xlsx, rag_query_vector, rag_clear_index, rag_web_content β since v1.9.2/v1.9.10)
βββ registerUiGenerationTools() βββΊ 3 tools (enabled by default)
βββ registerContextManagementTools() ββΊ **22 tools** (12 core memory/session + 9 session-index & project-registry + composite `restore_session_context`; enabled by default)
βββ registerTextProcessingTools() βββΊ 3 tools (enabled by default)
βββ registerRefactorCodeTools() βββΊ **1 tool** (`refactor_code`; `unused_import_cleanup` (a recode-engine rule object) is not a registered tool β audited 05.09, confirmed in the 28.09 re-audit; enabled by default)
βββ registerExecutionTools() βββΊ 5 tools (mixed defaults)
β
βΌ
Return Tool[] to SDK βββΊ **Code-registered total: 113 live tool definitions across the 22 registering module files** (full re-audit 28.09 β `src/toolsProvider.ts` registry + per-module `name:` scan; exposed set is toggle-dependent, GOD MODE max β 113) Β· **locale data: 115 entries / 113 distinct names in all five locales** (full parity; `read_document` + `rag_web_content` served by two categories each) Β· **07.11 delta (CLOSED, re-audit done):** + `apply_patch` shipped in `fileSystemTools.ts` β GOD MODE max β 116 (canon); all five locale sets now carry 120 entry lines / 118 distinct names each β every toolName documented in TOOLS_REFERENCE has an entry in all five sets (diff-verified, gap closed)
Session Activity Occurs
β
βΌ
auto_summarize_context() called
β
βββ Analyze tool usage patterns
βββ Detect configuration changes
βββ Identify important decisions
βββ Generate summary
βΌ
ContextStorageManager.addEntry(entry)
β
βββ Load existing entries from .ai_toolbox_memory.msgpack (Working Dir β Plugin Root fallback)
βββ Apply default scope: 'global' (v1.9.1+) unless explicitly specified
βββ Set TTL for session-scoped entries: 24h (v1.9.1+)
βββ Increment frequency counter on existing entries with matching ID
βββ Append new entry to beginning of array
βββ Limit to 1000 entries (prevent unbounded growth)
βββ Save atomically (temp file + rename)
βΌ
Persistent Storage (.ai_toolbox_memory.msgpack)
β
βββ get_context_memory(limit, type?) β Retrieve recent entries
β βββ Prune expired session entries (TTL check: 24h threshold, v1.9.1+)
β βββ Apply heuristic scoring: Recency(70%) + Frequency(30%) sort
β βββ Return top N scored entries
β
βββ search_context(query, maxResults) β Text-based search
β βββ Prune expired session entries (v1.9.1+)
β βββ Filter by title/content/tags match
β βββ Apply heuristic scoring to results
β βββ Return top N scored matches
β
βββ context_summary() β Statistics & counts
βββ delete_context_entry(id) β Remove specific entry
Session Activity Occurs
β
βΌ
auto_summarize_context() called
β
βββ Analyze tool usage patterns
βββ Detect configuration changes
βββ Identify important decisions
βββ Generate summary
βΌ
ContextStorageManager.addEntry(entry)
β
βββ Load existing entries from .ai_toolbox_memory.msgpack β .session_context/.ai_toolbox_memory.msgpack
βββ Append new entry to beginning of array
βββ Limit to 1000 entries (prevent unbounded growth)
βββ Save atomically (temp file + rename)
βΌ
Persistent Storage (.ai_toolbox_memory.msgpack)
β
βββ get_context_memory() β Retrieve recent entries
βββ search_context(query) β Text-based search
βββ context_summary() β Statistics & counts
βββ delete_context_entry(id) β Remove specific entry
// Simplified registration pattern (actual implementation uses closure-based registry)
export async function toolsProvider(ctl: ToolsProviderController): Promise<Tool[]> {
const pluginConfig = ctl.getGlobalPluginConfig(configSchematics); // Option A (02.11): global scope β settings persist across chats/app restarts
// Construct typed PluginConfig from .get() calls
const config: PluginConfig = { /* ... */ };
// Initialize managers (singleton pattern)
if (!stateManager) stateManager = new StateManager(config);
if (!backgroundCommandManager) backgroundCommandManager = new BackgroundCommandManager(config);
const tools: Tool[] = [];
// --- Declarative Registry Definition (v1.8.2+) ---
const TOOL_REGISTRIES: ToolRegistryEntry[] = [
{ key: 'fileSystem', register: () => registerFileSystemTools(config, stateManager) },
{ key: 'webSearch', register: () => registerWebResearchTools(config) },
// ... 19 more entries (21 total β audited 05.09; execution tools handled separately, not via the array)
];
// --- Registry Loop (replaces ~80 lines of if/else blocks) ---
for (const entry of TOOL_REGISTRIES) {
if (config[entry.key] || isGodMode) {
tools.push(...entry.register());
}
}
return tools;
}
class StateManager {
private state: Map<string, StateEntry>;
private maxSize: number;
private persistenceEnabled: boolean;
private memoryFile!: string; // Resolved at runtime
set(key, value): void // In-memory + async disk write
get<T>(key): T | undefined // In-memory retrieval
delete(key): boolean // In-memory + async disk update
getAllKeys(): Promise<string[]> // Waits for initialization
clear(): void // Resets in-memory state
}
// save_session_summary writes:
await stateManager.set(`${summaryId}_data`, compressed); // Base64-encoded gzip stream < 10k chars
await stateManager.set(`${summaryId}_timestamp`, Date.now());
// get_session_summary reads with backward-compatible fallback:
const keys = await stateManager.getAllKeys(); // Waits for loadFromFile(), returns all keys
const compressedData = stateManager.get(summaryKey);
try {
const decompressed = zlib.gunzipSync(Buffer.from(compressedData, 'base64')).toString('utf-8');
sessionSummary = JSON.parse(decompressed); // New format (v1.5.15+)
} catch (parseErr) {
// Fallback for legacy uncompressed summaries (pre-v1.5.15)
if (typeof compressedData === 'string' && compressedData.startsWith('{')) {
try {
sessionSummary = JSON.parse(compressedData); // Legacy format
} catch (legacyErr) {
throw new Error(`Legacy summary parsing failed: ${String(legacyErr)}`);
}
} else {
throw parseErr; // Corrupted or unknown format
}
}
export type MemoryScope = 'global' | 'project' | 'session';
class ContextStorageManager {
private storagePath: string; // .ai_toolbox_memory.msgpack
load(): Promise<ContextEntry[]>
save(entries: ContextEntry[]): Promise<void>
addEntry(entry: ContextEntry): Promise<void> // Applies default scope + TTL
getRecentEntries(limit, type?): Promise<ContextEntry[]> // Heuristic scoring applied
searchEntries(query, maxResults): Promise<ContextEntry[]> // Heuristic scoring applied
deleteEntry(id): Promise<boolean>
clearAll(): Promise<void>
getSummary(): Promise<ContextSummary>
pruneExpiredSessionEntries(): Promise<number> // TTL pruning (v1.9.1+)
}
interface ContextEntry {
id: string;
timestamp: number;
date: string;
type: 'decision' | 'pattern' | 'configuration' | 'file_change' | 'error' | 'summary';
title: string;
content: string;
tags?: string[];
scope?: MemoryScope; // NEW (v1.9.1): Context isolation
frequency?: number; // NEW (v1.9.1): Access count for scoring
ttl_ms?: number; // NEW (v1.9.1): Expiration threshold
}
// Recency Decay: Exponential decay based on age (lambda = 1 day)
const recencyFactor = Math.exp(-ageMs / (24 * 60 * 60 * 1000));
// Frequency Saturation: Prevents infinite bias toward frequently accessed entries
const frequencyFactor = freq / (freq + 5);
// Weighted composite score
return (recencyFactor * 0.7) + (frequencyFactor * 0.3);
Input β Path Validation β Binary Detection β Command Sanitization β SQL Validation
(validatePath) (isBinaryFile) (sanitizeCommand) (validateSQLQuery)
let currentWorkingDir: string = BASE_DIR;
getWorkingDir(): string
setWorkingDir(newDir: string): boolean
resetWorkingDir(): void
resolvePath(userPath: string): string
getAllowedBases(): string[]
// former src/tools/gatewayTools.ts (REMOVED from codebase 24.08; shown for historical reference)
export async function getGatewayTools(
provider: ToolsProvider,
config: PluginConfig
): Promise<Tool[]> {
const exploreTools = tool({
name: 'explore_tools',
description: 'Discover available tools and their categories...',
parameters: { category: z.string().optional() },
implementation: async (params) => {
await provider.getAvailableTools(); // Ensure registry loaded
return { success: true, categories: [...] }; // Returns category names only
}
});
const executeGatewayTool = tool({
name: 'execute_gateway_tool',
description: 'Executes a specific tool by its name...',
parameters: {
toolName: z.string(),
arguments: z.record(z.unknown())
},
implementation: async (params) => {
return await provider.executeTool(params.toolName, params.arguments); // Delegates to registry
}
});
return [exploreTools, executeGatewayTool];
}
User Message β AI calls explore_tools(category="fileSystem")
β Returns: { success: true, categories: ["read_file", "write_file", ...] }
β AI decides to use read_file
β AI calls execute_gateway_tool(toolName="read_file", arguments={file_name: "example.txt"})
β Gateway delegates to provider.executeTool("read_file", args)
β Tool executes with full validation, security checks, error handling
User Path Input
β
βββ Empty check βββββββββββββββββΊ Reject
β
βββ UNC path check (\\\) βββββββΊ Reject
β
βββ Relative path?
β β
β βββ Yes: Resolve against basePath
β β β
β β βββ Within base? ββββΊ Allow
β β βββ Outside base? βββΊ Reject
β β
β βββ No (absolute):
β β
β βββ In allowed bases? βββΊ Allow
β βββ Outside allowed? ββββΊ Reject
Command String
β
βΌ
Layer 1: Dangerous Pattern Blocking
β
βββ Null byte injection ββββββββββΊ Reject
βββ IFS tampering ββββββββββββββββΊ Reject
βββ Dangerous patterns (rm -rf, sudo, etc.) ββΊ Reject
βββ Too many pipes (>2) ββββββββββΊ Reject
βββ Multiple semicolons (>1) ββββββΊ Reject
βββ Command substitution ($(), ``) ββΊ Reject
βββ Environment modification ββββββΊ Reject
β
βΌ
Layer 2: Tool-Category Enforcement
β
βββ classifyCommand() β Set<string>
β β
β βββ git * / api.github.com β 'gitOperations'
β βββ duckduckgo / google / bing β 'webSearch'
β βββ puppeteer / playwright / chromium β 'browserAutomation'
β βββ sqlite3 / mysql / psql β 'databaseQueries'
β βββ curl / wget / http β 'httpClient'
β βββ nohup / disown / & β 'backgroundCommands'
β β
β βΌ
β Check against config toggles
β β
β βββ Category disabled + !godMode ββΊ Reject
β βββ Category enabled or godMode βββΊ Allow
β
βΌ
Allow Execution
JavaScript Code
β
βββ require() detection ββββββββββΊ Reject
βββ eval() detection βββββββββββββΊ Reject
βββ fs/child_process access ββββββΊ Reject
βββ Function constructor βββββββββΊ Reject
βββ Dynamic import() βββββββββββββΊ Reject
βββ __proto__ access βββββββββββββΊ Reject
// Stops calculating if minimum possible score drops below threshold
function levenshteinSimilarity(a: string, b: string, minScore: number): number | null {
// Quick rejection for very different lengths
if (lenDiff / maxLen > (1 - minScore)) return null;
// Two-row optimization (saves memory vs full matrix)
// Early exit when row minimum exceeds threshold
}
// Concurrency-controlled batch processing
async function findFilesAsync(dirPath, pattern, maxDepth, concurrencyLimit = 4) {
// Process directories in batches
for (const batch of batches) {
await Promise.all(batch.map(dir => searchDir(dir, depth + 1)));
}
}
User Message
β
βΌ
promptPreprocessor()
β
βββ Check temporalAwareness config
β β
β βββ Enabled?
β β
β βββ Yes: Get cached datetime (5min TTL)
β β β
β β βββ Format: Standard ([Zeit: ...]) or HEUTE IST Mode
β β β
β β βββ Append timestamp to message end
β β
β βββ No: Skip
β
βΌ
Final Prompt sent to LLM (with timestamp suffix)
User Message Arrives β Step 0: Attachments + Checkpoint Suffix
β
βΌ
Step 0.5: ContextGuard Token Counting & Auto-Tracker Threshold Check
β
βββ Count tokens via History Text Length Γ 0.25 (+10% buffer)
βββ autoTracker.checkAndGeneratePrompt() β threshold warning if β₯75% used
βββ Inject checkpointSuffix into all return paths (unified injection)
β
βΌ
Step 0.6: Auto-Tracker Checkpoint Reply Handling + Message Analysis
β
βββ Detect "YES"/"NO" (or German "JA"/"NEIN") reply to pending checkpoint prompt β flush/save memory
βββ Analyze message for tracking triggers (decisions, completions, errors)
βββ Buffer actions for later flush at next threshold checkpoint
β
βΌ
Step 0.7: Project Keyword Detection β NEW (v1.9.8+)
β
βββ Read project_registry.json from disk
βββ Extract candidate words from user message (filter stop-words)
βββ Match against registered projects with fuzzy normalization (hyphenβunderscore)
βββ If matched β Inject "β οΈ REGISTERED PROJECT DETECTED" confirm-first banner
β (Does NOT change CWD this turn; one-shot switch only after an explicit YES/JA reply in a later message)
β
βΌ
Step 1: Directory Path Detection β Unchanged
β
βββ Detect Windows/Unix/Relative paths in message
βββ If found β Inject "WORKING DIRECTORY DETECTED" confirmation prompt
β
βΌ
Step 2: Document RAG (if enabled)
β
βββ Yes: Load embedding model
β β
β βββ Process files β chunks
β β
β βββ Semantic retrieval
β β
β βββ Filter by affinity threshold
β β
β βββ Inject relevant chunks into prompt
β
βββ No: Pass through unchanged
browser_open_page(url)
β
βΌ
BrowserSessionManager.getBrowser()
β
βββ Browser exists & connected? ββββΊ Reuse
β
βββ No: Launch new Puppeteer instance
β
βββ Retry with exponential backoff (max 2)
β
βββ Reset inactivity timer (5 min)
β
βΌ
Navigate to URL
β
βββ Wait for selector (optional)
β
βββ Take screenshot (optional)
β
βββ Extract text content
Session Activity Detected
β
βΌ
auto_summarize_context(sessionEvents, configChanges)
β
βββ Analyze tool usage patterns (>3 uses = frequent pattern)
βββ Track configuration changes
βββ Identify important decisions
βββ Generate session summary
βΌ
ContextStorageManager.addEntry(entry)
β
βββ Load existing entries from .ai_toolbox_memory.msgpack
βββ Prepend new entry to array
βββ Enforce 1000-entry limit
βββ Atomic save (temp file + rename)
βΌ
Persistent Storage (.ai_toolbox_memory.msgpack)
β
βββ get_context_memory(limit, type?) β Retrieve entries
βββ search_context(query, maxResults) β Text-based search
βββ context_summary() β Statistics & counts
βββ delete_context_entry(id) / clearContextMemory(confirm) β Management
User calls get_session_summary() or get_memory()
β
βΌ
1οΈβ£ Check Working Directory File:
{current_working_dir}/.session_context/.ai_toolbox_memory.msgpack
β
βββ Found? Decode msgpack β Return β
(Local-first hit)
β
βββ Missing/Empty/Corrupt? Continue...
β
βΌ
2οΈβ£ Check Plugin Root File:
{plugin_root}/.session_context/.ai_toolbox_memory.msgpack
β
βββ Found? Decode msgpack β Return β οΈ (Fallback hit)
β
βββ Missing/Empty/Corrupt? Continue...
β
βΌ
3οΈβ£ Check In-Memory State (RAM):
stateManager.get('session_summary_latest') or memory_* keys
β
βββ Found? Return β οΈ (Last resort)
βββ Not found? Return β error
User Message Arrives β Step 0.5: ContextGuard Token Counting
β
βββ autoTracker.checkAndGeneratePrompt(tokenCount, maxTokens):
β ββ Calculate usagePercentage = (effectiveTokens / maxTokens) * 100
β ββ Compare against threshold (default: 75%)
β ββ If >= threshold β Generate warning + FSM: IDLE β THRESHOLD_REACHED
β
βΌ
Warning injected into user message prompt:
β οΈ SESSION WARNING: You have reached {usage}% of your token limit...
User replies "YES" (German "JA" normalized) β Step 0.6: Checkpoint Reply Detection
β
βββ autoTracker.hasPendingWarning()? YES β
βββ replyMatch === 'YES'? YES β
βββ autoTracker.processUserReply('YES') β FSM: THRESHOLD_REACHED β CONFIRMED
βββ autoTracker.checkAndSaveTokenThreshold():
ββ flushActionsToMemory() β Save buffered decisions/completions/errors
ββ autoSaveSessionMemory() β Create checkpoint entry with token stats
User replies "NO" (German "NEIN" normalized) β Step 0.6: Checkpoint Reply Detection
β
βββ autoTracker.hasPendingWarning()? YES β
βββ replyMatch === 'NO'? YES β
βββ autoTracker.processUserReply('NO') β FSM: THRESHOLD_REACHED β DECLINED β IDLE (reset)
tool result β tokenStatsManager.recordToolResult()
βββ measures payload size β running mid-loop delta for this turn
βββ logs [AutoTracker] [DELTA] β¦ | chat used β N tok
where N = turnBaselineTokens (TokenCheck baseline at turn start) + midLoopEstTokens
next preprocess()/threshold evaluation
βββ compares against historyCount + running deltas (FIX #20),
so 75% / 90% triggers fire inside long multi-tool turns, not only between messages
tool result β toolsProvider wrapper pass (src/toolsProvider.ts, per registered tool)
β
βββ shared projection: baseline + midLoopDeltaTokens + this payload's estimate + footer self-size
β (computed ONCE per pass and consumed by BOTH the usage-footer gate and tier 2 β one number, two independent gates)
β
βββ TIER-2 block (sibling of the footer gate inside `baseline > 0`, NOT nested under the contextUsageFooter toggle):
Gate: !forcedSaveFiredThisTurn β one-shot per turn; re-armed at provider entry (= new turn, resetToolGuard convention)
AND config.contextGuardEnabled === true β strict (undefined/false/0 skip β mirrors PART B)
AND limit > 0 AND percent is a finite number > 0 β read LIVE from pluginConfig each pass (UI toggle change takes effect this run)
AND projected >= limit * percent / 100 β INCLUSIVE boundary (== threshold fires; pinned by test)
β
βββ Gate/guard fails βββΊ Skip silently (guard instance null pre-startup, or a stub without the method β PART B covers it)
β
βββ Gate passes β LATCH forcedSaveFiredThisTurn = true BEFORE the first await (a failure must not re-arm mid-turn):
βΌ
turnMessages = TokenStatsManager.getTurnMessages() ?? [] β post-compression list published at PART B's single setTurnEvaluation finally point (Step 2 hook); an absent publish coalesces to []
β
βββ Span guard: toSummarize = slice(0, len β keepLast=10) on the CALLER side (same exclusion as compressHistory), computed BEFORE its gate `toSummarize.length >= 10` (INCLUSIVE β a 20-message list fires exactly)
β βββ <10 summarizable messages (short list or absent publish) β deterministic [TIER-2] skip-log ("nothing summarizable yet"), latch still holds
β
βββ flushActionsToMemory() β buffered auto-tracked actions durable before the summary spans them (PART B parity)
βββ guard.generateSessionContinuity(toSummarize) β uncast call; same Arc-C machinery PART B uses at compression time
β βββ no structured summary (no usable model) β [TIER-2] warn, save skipped β compression still pending at next user message
β
βββ persistGeneratedSessionSummary(summary) β canonical save_session_summary writer path (.session_context/)
βββ saved=true β β
telemetry line Β· saved=false β β οΈ persist-failed telemetry line (never throws)
every failure inside the block is caught β [TIER-2] console.warn, tool result flows through byte-unchanged (non-fatal by spec)
Config (src/config.ts): nested budgetGuard block {default enabled=true since 06.11 owner order; explicit false β byte-identical legacy behavior}
+ five flat UI keys (toolsProvider.ts assembles the nested block from them β house pattern)
π‘οΈ Budget Guard master switch, advisory/spill/autopause thresholds (1β99 %), safety margin (0β50 %)
M1 β pre-call cost gate on the READ path (src/tools/fileSystemTools.ts read_file + read_file_chunked; src/tools/documentTools.ts):
projected = baseline + midLoopDelta + estimateTokensFromCharsOfClass(expectedChars, 'highEntropy')
fires when limit > 0 AND projected > limit Γ advisory_threshold_pct/100 β default 80 %
β self-describing ADVISORY returned INSTEAD of the file dump (what was withheld + numbers
+ controlled re-read hints: read_file_chunked small chunks / ripgrep / pattern_scan)
M2 β budget-aware spill cap in the toolsProvider wrapper 'A spill pattern' block (4th consumer):
edge = limit Γ (1 β safety_margin_pct/100) β default margin 8 β 92 % edge
fires when projected (incl. CTX-FOOTER self-size reserve) > edge
β inline preview capped at budgetChars = floor(remainingBudget / applied high-entropy density);
STRICTER bound wins: effectiveMax = min(absolute compaction cap, budgetChars); full payload still
stored verbatim via the existing spillTextIfNeeded() (disk store + SHA-256 locator pointer);
self-describing 'SPILLED by budget guard' marker line attached ONLY on budget spills β
absolute-cap spills stay byte-identical
M3 β insurance auto-persist at the autopause edge (latch + Arc-C pipeline in toolsProvider.ts next to its
tier-2/FIX-B siblings; pure threshold resolution + marker live in src/budgetGuard.ts):
fires when limit > 0 AND projected β₯ limit Γ autopause_threshold_pct/100 β default 85 %
AND !autoPersistFiredThisSession (session-scoped one-shot latch, deliberately never re-armed per provider run)
β Arc-C pipeline parity with tier-2: flushActionsToMemory() first, generateSessionContinuity(),
persistGeneratedSessionSummary(); marker PREPENDED to task_description ('[Budget guard] auto_saved=true β¦')
so the NEXT restore_session_context dossier shows it as its own note. M3 checkpoints only β
it caps nothing and never compresses; every failure non-fatal ([M3-AUTOSAVE] telemetry)
User Message Arrives
β
βΌ
promptPreprocessor()
β
βββ Check contextGuardEnabled config
β β
β βββ Enabled?
β β
β βββ Yes: Count tokens in history
β β β
β β βββ Below 90% threshold? βββΊ Skip compression
β β β
β β βββ Above 90% threshold?
β β β
β β βΌ
β β compressHistory(messages)
β β β
β β βββ Identify messages to compress (all except last 10)
β β βββ Send to summary model
β β β βββ Use contextGuardSummaryModel or current chat model
β β β
β β βββ Generate summary with preserved file paths/names
β β β
β β βββ Calculate tokens saved
β β β
β β βββ Inject visual indicator:
β β β
β β βββ π§ Emoji header
β β βββ Messages compressed count
β β βββ Tokens before β after (e.g., "~85k β ~42k")
β β βββ Percentage saved (e.g., "Saved ~43,000 tokens (~51%)")
β β βββ Timestamp
β β βββ Visual separator lines
β β
β βββ No: Skip ContextGuard processing
β
βΌ
Final Prompt sent to LLM (with or without compression indicator)
π§ **ContextGuard Compression Active**
βββββββββββββββββββββββββββββββββββββββββββββββ
β’ Compressed 15 message(s) into summary
β’ Tokens before: ~85k β after: ~42k
β’ **Saved ~43,000 tokens (~51%)**
β’ Timestamp: 19:15:32
βββββββββββββββββββββββββββββββββββββββββββββββ
### CONTEXT SUMMARY (from 15 messages)
[Summary content here...]
Above threshold β PART B-2 settlement (pending checkpoint warning consumed)
β
βΌ
Gate: contextGuardForceSummaryOnCompress === true β strict; undefined/false skips the step entirely
AND typeof guard.generateSessionContinuity === 'function' β legacy/stub guards skip byte-identically to pre-Arc C
AND safeMessages.length > 10 β keepLast=10 exclusion guarantees a non-empty span
β
βββ Gate fails βββΊ Skip (legacy path, unchanged)
β
βββ Gate passes (all failures below are caught β compression still runs):
βΌ
flushActionsToMemory() β buffered auto-tracked actions durable first
βΌ
generateSessionContinuity(safeMessages.slice(0, len-10), { fallbackModelId, abortSignal })
β (summary model = contextGuardSummaryModel or current chat model)
βββ Summary βββΊ persistGeneratedSessionSummary() via the canonical save_session_summary writer
β βββ saved β log β
β βββ not durable β telemetry warn (fallback)
βββ null (unparsable / no usable model) βββΊ telemetry warn (fallback)
βββ throw βββΊ caught, telemetry warn (non-fatal)
βΌ
compressHistory(safeMessages, ctl.abortSignal) β history destruction strictly last β runs in every case
User Message Arrives (ContextGuard Enabled)
β
βΌ
promptPreprocessor() β Native History API Iteration
β
βββ history.getLength() β Get message count
βββ For each message i from 0 to length-1:
β βββ msg = history.at(i) β Retrieve message by index
β βββ msg.getText() β Extract text content via getter method
β βββ msg.getToolCallRequests() β Serialize tool calls if present
β βββ msg.getToolCallResults() β Serialize tool results if present
β
βΌ
contextGuard.countTokens(messages, imageCount, modelId, systemPrompt, historyTextLength)
β
βββ PRIMARY METHOD: History Text Length Γ 0.25 ratio (v1.8.8+) β effective ~0.275 with +10% buffer
β βββ If historyTextLength provided from native API iteration:
β β β
β β βββ primaryTokenCount = Math.ceil(historyTextLength * 0.25)
β β βββ Add image tokens if applicable (+500 per image)
β β βββ Return totalTokens β β
Matches LM Studio sidebar exactly
β β
β βββ Verified at ~130K tokens for 544,578 chars β <0.5% deviation from sidebar
β
βββ VERIFIED at ~130K tokens for 544,578 chars β <0.3% deviation from sidebar (improved from <0.5%)
βββ FALLBACK: SDK-native countTokens() Γ calibration (if history unavailable)
β βββ Format messages into prompt string
β βββ Call model.countTokens(promptString) via LM Studio SDK
β βββ Apply TOKEN_SCALING_FACTOR = 1 for overhead calibration β β οΈ Legacy fallback
β
βΌ
Threshold Check: totalTokens >= tokenLimit * 0.9?
β
βββ Yes: compressHistory(messages) β Uses History Text Length Γ 0.24 for compressedPreview too
βββ No: Skip compression
index.ts
βββ toolsProvider.ts
β βββ config.ts
β βββ stateManager.ts
β βββ backgroundCommands.ts
β βββ tools/*.ts (22 live registration modules + helpers β audited 05.09)
β βββ security.ts (shared)
β βββ workingDir.ts (shared)
β βββ performanceUtils.ts (shared)
βββ config.ts
βββ promptPreprocessor.ts
βββ config.ts
ConfigSchema (Zod)
βββ Tool Gating (13 booleans)
βββ Execution Tools (4 booleans)
βββ Search Settings (3 fields)
βββ Browser Settings (2 fields)
βββ Git Settings (2 fields)
βββ Document RAG (3 fields)
βββ Security Settings (4 fields)
βββ State Management (2 fields)
βββ i18n (1 field)
βββ Notifications (1 field)
βββ Temporal Awareness (2 fields: temporalAwareness, dateFormatStyle)
βββ ContextGuard (7 fields): v1.4.2
βββ contextGuardEnabled (boolean) β Master toggle
βββ contextGuardCompressionPercent (int 50β100, default 90) β Compression trigger as % of the effective context window (07.10: replaced absolute contextGuardTokenLimit; built-in 30k fallback when no model window reported)
βββ contextGuardForceSummaryOnCompress (boolean, default true) β forced structured session-memory save at the compression trigger β flushes auto-tracked actions, then generates + persists a structured SessionSummaryData via the canonical save_session_summary writer BEFORE history destruction; strict `=== true` gate + typeof check on the guard method; non-fatal telemetry fallback on any failure
βββ contextGuardSmartReading (boolean) β Keyword-based file reading
βββ contextGuardSummaryModel (string) β Dedicated summary model name
βββ contextGuardTerminalFilterEnabled (boolean) β Terminal output filtering
βββ contextGuardTerminalFilterLength (number 100-20K) β Max terminal chars
βββ BudgetGuard (5 flat UI keys): budgetGuardEnabled (boolean, ON by default (explicit `false` β byte-identical legacy behavior); all five fields localized), budgetGuardAdvisory/Spill/AutopauseThresholdPct (int 1β99; defaults 80 / 92 / 85) + budgetGuardSafetyMarginPct (int 0β50, default 8 β M2 edge = limitΓ(1βmargin)); the nested consumer-facing `budgetGuard` block is assembled from these five in toolsProvider.ts. Full design: Β§ Budget-Guard Payload Protection above
src/
βββ index.ts # Plugin entry point
βββ toolsProvider.ts # Tool registration (conditional config gating)
βββ config.ts # Zod schema + UI schematics
βββ security.ts # Path/SQL/command validators
βββ stateManager.ts # Persistent state management
βββ workingDir.ts # Working directory manager
βββ performanceUtils.ts # Caching, async search, Levenshtein
βββ promptPreprocessor.ts # Document RAG + ContextGuard integration
βββ backgroundCommands.ts # Background process manager
βββ fuzzySearch.ts # Fuzzy file search implementation
βββ locales/ # i18n subsystem (see "Localization (i18n)" section)
β βββ types.ts # LanguageCode, Translation, ToolCategoryTranslations, FullTranslationSet
β βββ en.ts # English β reference locale (order + coverage baseline)
β βββ de.ts # German
β βββ es.ts # Spanish (added 04.09 with the i18n Tier-2 epic)
β βββ zh-CN.ts # Simplified Chinese
β βββ zh-TW.ts # Traditional Chinese
β βββ i18n.ts # I18nManager (language switching + accessors)
βββ tools/ # Tool category modules (26 source files β audited 05.09 vs v1.9.15 rev 27: 22 live register fns; utilityTools.ts exports 2 more but neither is ever called [dead]; patternScan/fileModTracker/toolPriority = helper-only)
β βββ fileSystemTools.ts # File system operations (24 tools β REGISTERED, incl. pattern_scan; + line_operations folded in 23.09, Q6)
β βββ patternScan.ts # pattern_scan search engine (clean-room module, ReDoS-gated; tool registered in fileSystemTools.ts)
β βββ webResearchTools.ts # Web research & search (3 tools β REGISTERED; rag_web_content served by vectorRagTools.ts since v1.9.10)
β βββ browserAutomationTools.ts # Browser automation (5 tools β REGISTERED)
β βββ gitGithubTools.ts # Git local ops + GitHub API (15 tools β REGISTERED)
β βββ databaseTools.ts # Database queries (1 tool β REGISTERED)
β βββ documentTools.ts # Document parsing (PDF/DOCX) (1 tool β REGISTERED)
β βββ backgroundCommandTools.ts # Background process management (3 tools β REGISTERED)
β βββ executionTools.ts # Code execution JS/Python/Terminal (5 tools β REGISTERED)
β βββ # utilityTools.ts β REMOVED (flagged dead code in tool-consolidation-draft Β§7: unregistered registerUtilityTools aggregator + duplicate memory/session tools; its live functions now sit in the dedicated modules listed below, registered under the 'utility' toggle)
β βββ imageProcessingTools.ts # Image processing & OCR (4 tools β REGISTERED)
β βββ httpClientTools.ts # HTTP client operations (3 tools β REGISTERED)
β βββ vectorRagTools.ts # Vector RAG semantic search (7 tools β REGISTERED: rag_index_files, rag_index_pdf, rag_index_docx, rag_index_xlsx, rag_query_vector, rag_clear_index, rag_web_content)
β βββ textProcessingTools.ts # Text transformation (3 tools β REGISTERED; β line_operations folded to fileSystemTools.ts 23.09, Q6)
β βββ uiGenerationTools.ts # UI component generation (3 tools β REGISTERED)
β βββ contextManagementTools.ts # Context management & tracking (12 tools β REGISTERED)
β βββ refactorCodeTools.ts # AST-based code refactoring (2 tools β REGISTERED)
β βββ dataVisualizationTools.ts # Chart generation (1 tool β REGISTERED under 'utility' toggle)
β βββ backupTools.ts # Backup & restore operations (4 tools β REGISTERED under 'utility' toggle)
β βββ cleanupBackupsTool.ts # Cleanup backups utility (1 tool β REGISTERED under 'utility' toggle)
β βββ # lineOperations.ts β REMOVED 23.09 (NEXT-REV: deprecated `delete_lines` alias stub, registered under the `utility` key since Q5=A; module + registration deleted, see docs/tool-consolidation-draft.md Β§9a)
β βββ taskPlanningTools.ts # Task planning & execution tracking (3 tools β REGISTERED)
β βββ markdownPreviewTools.ts # Markdown preview generation (1 tool β REGISTERED under 'utility' toggle)
β βββ fileModTracker.ts # File modification tracker (REGISTERED)
β βββ # networkToolsRegistry.ts β REMOVED 24.08 (was an orphan file with zero imports; deletion was tracked backlog since v1.9.3 and completed in the rag_web_content fix suite)
β βββ toolPriority.ts # Cluster-aware tool priority ranking (REGISTERED)
β βββ # backupUtils.ts / executionRegistry.ts / toolProtocolWarnings.ts / utilityRegistry.ts β REMOVED 01.09.2026 (Tier-1 dead code: zero referencers; see CHANGELOG_v2.md entry ~18:30)
β βββ restoreFromBak.ts # Backup restoration utility (REGISTERED)
β βββ attachmentManager.ts # β οΈ audited 05.09: lives in src/, NOT src/tools/ β attachment handling
β βββ browserActions.ts # β οΈ lives in src/ β browser action execution & validation
β βββ findLMStudioHome.ts # β οΈ lives in src/ β LM Studio home directory detection
β βββ lmStudioApi.ts # β οΈ lives in src/ β LM Studio REST API integration layer
β βββ budgetGuard.ts # Budget-Guard-Arc decision core (M1/M2/M3 pure gates; lives in src/ β see Β§ Data Flow β Budget-Guard Payload Protection)
βββ tokenStatsManager.ts # β οΈ lives in src/ β token statistics tracking (manager, not a tool); C family 24.09: active-turn tracking + bounded drain (`drainActiveToolTurns`, poll 250 ms / cap 15 s)
βββ types/ # Type definitions
βββ dom-augment.d.ts # DOM type augmentations for browser automation
βββ node-notifier.d.ts # Node.js notifier type declarations
βββ types.d.ts # Core shared type definitions
tests/ # Jest test suite β 84 suites / 1262 tests (v1.9.24)
βββ security.test.ts # Core security validation tests
βββ security.edge-cases.test.ts # Security boundary & edge case testing
βββ config.test.ts # Zod schema + UI schematics validation
βββ stateManager.test.ts # Persistence, path resolution, atomic writes
βββ fileSystemTools.test.ts # File system operation tests (module now 24 tools after line_operations fold 23.09 Q6; incl. pattern_scan β the folded tool has no dedicated suite yet)
βββ webResearchTools.test.ts # Multi-engine search & fetch tests
βββ browserAutomationTools.test.ts # Puppeteer session management tests
βββ gitGithubTools.test.ts # Git local ops + GitHub API tests
βββ databaseTools.test.ts # SQLite query validation tests
βββ executionTools.test.ts # JS/Python/Terminal sandboxed execution tests
βββ utilityTools.test.ts # Utility tools (backup, chart, line ops) tests
βββ backgroundCommands.test.ts # Background process management tests
βββ toolsProvider.test.ts # Declarative registry pattern integration tests
βββ performanceUtils.test.ts # Caching, async search, Levenshtein tests
βββ fuzzySearch.test.ts # Fuzzy file search similarity scoring tests
βββ workingDir.test.ts # Working directory manager path resolution tests
βββ findLMStudioHome.test.ts # LM Studio home detection & fallback tests
βββ i18n.test.ts # Translation file loading & formatting tests
βββ autoTracker.test.ts # Token threshold checkpointing & session memory tests (v1.6.6+)
βββ browserActions.test.ts # Browser action execution & validation tests
βββ fileSearch.test.ts # Recursive file search with exclusion patterns tests
βββ ripgrepEngine.test.ts # Standalone ripgrep tool + engine contract β REAL-timer suite incl. 3 s watchdog cap-fire (new 14.09)
βββ ripgrepTools.test.ts # ripgrep response-shape pins: mode echo, auto-demotion hint gating, no-match pattern_mode (new 14.09; former grep_files suites deleted in the same window)
βββ refactorCodeTools.test.ts # AST-based refactoring & dry-run diff tests (v1.5.30+)
βββ hubExclusionClustering.test.ts # Hub-exclusion clustering algorithm verification (83 tests) β NEW v1.9.8
βββ projectAutoDetect.test.ts # Project auto-detection & registration workflow tests β NEW v1.9.8
src/tools/recodeTool/
βββ rules/
β βββ unusedImports.ts β Tier 1: Implemented β
(extracted from refactorCodeTools.ts) β the only live rule as of 01.09.2026
β βββ # deadCodeDetection / modulePathNormalization / typeInference / asyncModernizer β REMOVED 01.09.2026 (Tier-1 dead code: never wired into any tool operation; see CHANGELOG_v2.md entry ~18:30)
βββ recodeEngine.ts β AST transformation orchestrator with dry-run diff support (LCS-based)
βββ recodeTypes.ts β Shared interfaces & schemas (RuleContext, RuleResult, RecodeRule)
export type Confidence = 'EXTRACTED' | 'INFERRED' | 'AMBIGUOUS';
interface ToolResultMetadata {
confidence: Confidence; // EXTRACTED (deterministic), INFERRED (semantic), AMBIGUOUS (uncertain)
provenance?: string; // e.g., "file:src/utils.ts L42", "rag_query_vector"
note?: string; // Additional context for confidence assessment
}
// Helper functions
function determineConfidence(operationType, success, fallbackUsed): Confidence;
function createToolResult<T>(data: T, confidence: Confidence, options?): { success: true; data: T & ToolResultMetadata };
function createErrorResult(message: string, provenance?): { success: false; error: string; data: ToolResultMetadata };
Build Dependency Graph β Calculate Degrees β Identify Hubs (80th percentile) β Create Non-Hub Subgraph β Louvain Community Detection β Majority-Vote Hub Reattachment β Cluster Density & Modularity Calculation
function buildDependencyGraph(sourceDirs: string[]): Map<string, Set<string>>;
function addEdge(adjacency, source, target): void;
function calculateDegrees(adjacency): Map<string, number>;
function identifyHubs(degrees, hubThresholdPercentile = 80): Set<string>;
function louvainCommunityDetection(adjacency): Map<string, number>;
function reattachHubsByMajorityVote(hubs, adjacency, nonHubCommunities): Record<string, number>;
function calculateClusterDensity(members, adjacency): number;
function calculateModularity(edges, nodeDegrees, clusterAssignments): number;
function performHubExclusionClustering(adjacency, hubThresholdPercentile = 80): HubExclusionResult;
function analyzeAiToolboxDependencies(): HubExclusionResult;
interface HubExclusionResult {
nodes: ModuleNode[]; // All modules with degrees (sorted by degree descending)
edges: Edge[]; // All connections in the graph
hubs: string[]; // Identified hub module IDs
nonHubs: string[]; // Non-hub modules for clustering
clusters: ClusterInfo[]; // Community clusters with density metrics
hubAssignments: Record<string, number>; // Hub β cluster ID mapping via majority-vote
hubThresholdPercentile: number; // Threshold used (default: 80th percentile)
modularity?: number; // Overall clustering quality [0-1]
}
interface ClusterInfo {
clusterId: number; // Sequential cluster identifier (0-indexed)
members: string[]; // Module IDs in this cluster
size: number; // Number of members
density?: number; // Internal edge density [0-1]
}
interface ProjectDetectionResult {
path: string; // Absolute path to detected project
isValid: boolean; // Whether this looks like a valid project (β₯0.3 confidence)
name?: string; // Detected project name from package.json or fallback
sourceDirs?: string[]; // Source directories within the project
confidence: number; // Detection confidence score [0-1]
}
// Confidence signals:
// - package.json exists: +0.4 (strongest signal)
// - src/ or lib/ directory exists: +0.3
// - .git directory exists: +0.1
// - tsconfig.json or jest.config.* exists: +0.2
function normalizeProjectName(name: string): string; // "ai-toolbox" β "ai_toolbox", "@lmstudio/ai-toolbox" β "lmstudio_ai_toolbox"
function generateNameVariants(name: string): string[]; // "aitoolbox" β ["aitoolbox", "ai-tool-box"]
async function searchWithAutoRegister(query, cwd, maxResults = 10): Promise<Array<{ name: string; path: string }>> {
let results = await enhancedSearchProjects(query, maxResults);
if (results.length === 0) {
const autoDetected = autoDetectAndRegister(cwd, query, true /* explicitConfirmation */);
if (autoDetected.registered) {
results = await enhancedSearchProjects(query, maxResults);
}
}
return results;
}
function initializeProjectDetection(cwd: string): void; // β οΈ DEPRECATED (v1.9.8+): No longer called from index.ts at startup. Registration requires explicitConfirmation=true via register_project tool. See src/index.ts comment: "NO AUTO-REGISTRATION ON STARTUP"
// Step 0.7 in promptPreprocessor.ts (v1.9.8+) β NEW
async function detectProjectKeywords(message: string): Promise<string | null> {
// 1. Read project_registry.json from disk
const registry = await readProjectRegistry();
// 2. Extract candidate words from user message
const words = extractCandidateWords(message); // Filter stop-words, lowercase
// 3. Fuzzy-match against registered projects (hyphenβunderscore normalization)
for (const word of words) {
for (const project of registry.projects) {
if (normalizeProjectName(word) === normalizeProjectName(project.name)) {
return `REGISTERED PROJECT DETECTED: ${project.name} at ${project.path}`;
}
}
}
return null; // No match β fall through to directory detection (Step 1)
}
// _syncFromSessionMemory() in registry manager β NEW (v1.9.8+)
async function _syncFromSessionMemory(): Promise<void> {
// Read .ai_toolbox_memory.msgpack from working dir and plugin root
const entries = await loadContextEntries();
for (const entry of entries) {
if ('decision' in entry.data && typeof entry.data.decision === 'string') {
const decision = entry.data.decision as string;
// Match project names from past decisions (e.g., "switched to ai-toolbox")
const match = extractProjectNameFromDecision(decision);
if (match) {
await registerProject(match.name, match.path);
}
}
}
}
export type ContextOrigin = 'ast' | 'semantic';
interface ContextNode {
id: string;
_origin: ContextOrigin; // "ast" (raw file/AST) or "semantic" (derived insight)
label?: string; // Human-readable label
source_file?: string; // Original file path (for ast origin)
data?: unknown; // Payload/data
timestamp?: number; // Optional timestamp for ordering
}
function replaceTier(oldNodes: ContextNode[], newNodes: ContextNode[]): ContextNode[] {
const oldAst = oldNodes.filter(n => n._origin === 'ast');
const oldSem = oldNodes.filter(n => n._origin === 'semantic');
const newAst = newNodes.filter(n => n._origin === 'ast');
const newSem = newNodes.filter(n => n._origin === 'semantic');
return [
...oldAst.filter(a => !newAst.some(n => n.id === a.id)), // Old AST not replaced
...newAst, // New AST
...oldSem.filter(s => !newSem.some(n => n.id === s.id)), // Old Sem not replaced
...newSem // New Sem
];
}
function createAstNode(id: string, data: unknown, sourceFile?: string): ContextNode;
function createSemanticNode(id: string, data: unknown, label?: string): ContextNode;
export type PriorityTier = 'critical' | 'high' | 'standard' | 'optional' | 'background';
const PRIORITY_TIER_VALUES: Record<PriorityTier, number> = {
critical: 1, // File system tools (module now 24 after 23.09 Q6 fold; line_operations deliberately kept 'standard' β owner flag)
high: 2, // Web research, execution, git operations (30+ tools β essential workflows)
standard: 3, // Browser automation, image processing, RAG, HTTP client (25+ tools β useful but not essential)
optional: 4, // Context management tools (12 tools β specialized or low-usage)
background: 5 // Backup, cleanup, chart generation, markdown preview (8+ tools β utility/maintenance)
};
interface ClusterAwarePriority extends ToolPriority {
moduleDegree?: number; // Number of dependencies/connections
isHub?: boolean; // Whether this tool's module is identified as a hub
clusterId?: number; // Assigned cluster ID from Hub-Exclusion clustering
centralityScore?: number; // Centrality score [0-1] β higher = more architecturally important
}
function computeCentralityScores(tools: ToolPriority[], clusteringResult: HubExclusionResult): Map<string, number>;
function sortToolsByClusterAwarePriority(tools: { name: string }[], clusteringResult?: HubExclusionResult): typeof tools;
function generateClusterAwareFilterReport(tools: { name: string }[], limit: number, clusteringResult?: HubExclusionResult): string;
const CATEGORY_TO_MODULE: Record<string, string | readonly string[]> = {
fileSystem: ['fileSystemTools.ts', 'tools/fileSystemTools.ts'],
webResearch: ['webResearchTools.ts', 'tools/webResearchTools.ts'],
// ... 20 categories mapped to source files with dual-name support (bare + path-prefixed)
};
// _syncFromSessionMemory() β Called lazily when search_projects or get_project_info is invoked
async function _syncFromSessionMemory(): Promise<void> {
// Load context entries from working dir + plugin root
const entries = await loadContextEntries();
for (const entry of entries) {
if ('decision' in entry.data && typeof entry.data.decision === 'string') {
const decisionText = entry.data.decision as string;
// Extract project names from past decisions
// e.g., "switched to ai-toolbox at C:\Source Code\..."
const match = extractProjectNameFromDecision(decisionText);
if (match) {
await registerProject(match.name, match.path);
}
}
}
}
// Called from search_projects tool β ensures registry is up-to-date before querying
async function searchProjects(query: string): Promise<ProjectInfo[]> {
await _syncFromSessionMemory(); // Lazy sync β NEW
return enhancedSearchProjects(query, 10);
}
promptPreprocessor() β detectProjectKeywords(message)
β
βββ If match found β Inject confirmation prompt
β ("REGISTERED PROJECT DETECTED: ai-toolbox at C:\...")
β
βββ User replies "YES" β AI calls register_project(workingDir, confirmed=true)
β
βΌ
registryManager._syncFromSessionMemory() β Ensures future searches find it
// New cross-module relationships:
toolPriority.ts β hubExclusionClustering.js (centrality scoring integration)
contextTiers.ts β (standalone β used by ContextStorageManager for tier-provenance)
projectAutoDetect.ts β DEPRECATED: No longer called from index.ts at startup. Registration requires explicitConfirmation=true via register_project tool. See src/index.ts comment: "NO AUTO-REGISTRATION ON STARTUP"
confidenceTypes.ts β all tool modules (via createToolResult<T>() helper functions)
hubExclusionClustering.ts β analysis utility (analyzeAiToolboxDependencies() pre-populated graph)
execute(toolName, args, impl, finalizer?) β { success, data?, error?, kind }
βββ monotonic guard: canonical key = tool name + deep-stable sorted-key JSON of args;
β identical call already seen this turn β kind "abstain" (no re-execution)
βββ legacy { success, data/error } return shapes normalized into the taxonomy
β (error text rendered by describeError() β see below)
βββ unhandled impl exceptions caught β kind "error" + code "UNHANDLED_EXCEPTION"
registerFinalizer(toolName, fn) // per-tool finalizeContent hook
resetGuard() // clears the monotonic seen-set (per-turn reset seam)
// module-level helpers: deny(reason), abstain(reason)
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Atomic Write Utility β
β (src/utils/atomicWrite.ts) β
β β
β ββββββββββββββββββββ ββββββββββββββββββββ β
β β atomicWrite() β β atomicWriteBinaryβ β
β β (text files) β β File() β β
β β β β (binary files) β β
β ββββββββββ¬ββββββββββ ββββββββββ¬ββββββββββ β
β β β β
β βΌ βΌ β
β βββββββββββββββββββββββββββββββββββββββββββββββ β
β β 1. Generate random temp filename β β
β β crypto.randomBytes(9) β 72-bit entropy β β
β β Format: {original}.{hex}.tmp β β
β ββββββββββββββββββββββ¬βββββββββββββββββββββββββ β
β β β
β βΌ β
β βββββββββββββββββββββββββββββββββββββββββββββββ β
β β 2. Write content to temp file β β
β β fs.promises.writeFile(tempPath, data) β β
β β (text: UTF-8 | binary: raw buffer) β β
β ββββββββββββββββββββββ¬βββββββββββββββββββββββββ β
β β β
β βΌ β
β βββββββββββββββββββββββββββββββββββββββββββββββ β
β β 3. Atomic rename (survives crashes) β β
β β fs.promises.rename(tempPath, original) β β
β β OS-level atomic operation β β
β ββββββββββββββββββββββ¬βββββββββββββββββββββββββ β
β β β
β βΌ β
β βββββββββββββββββββββββββββββββββββββββββββββββ β
β β 4. Cleanup on failure (if rename fails) β β
β β fs.promises.unlink(tempPath) β β
β βββββββββββββββββββββββββββββββββββββββββββββββ β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
Original file remains intact even if process crashes
between steps 2 and 3 β temp file orphaned but safe.
import * as crypto from 'crypto';
import { writeFile, rename, unlink } from 'fs/promises';
export async function atomicWrite(filePath: string, content: string): Promise<void> {
// Generate randomized temp filename (72-bit entropy)
const tempFile = `${filePath}.${crypto.randomBytes(9).toString('hex')}.tmp`;
try {
await writeFile(tempFile, content, 'utf-8'); // Step 1: Write to temp
await rename(tempFile, filePath); // Step 2: Atomic rename
} catch (err) {
await unlink(tempFile).catch(() => {}); // Step 3: Cleanup on failure
throw err; // Re-throw original error
}
}
export async function atomicWriteBinaryFile(
filePath: string,
buffer: Buffer
): Promise<void> {
const tempFile = `${filePath}.${crypto.randomBytes(9).toString('hex')}.tmp`;
try {
await writeFile(tempFile, buffer); // Raw buffer write (no encoding)
await rename(tempFile, filePath); // Atomic rename
} catch (err) {
await unlink(tempFile).catch(() => {}); // Cleanup on failure
throw err; // Re-throw original error
}
}
// BEFORE atomic write attempt β create .bak backup
await fs.copyFile(originalPath, `${originalPath}.bak`);
try {
await atomicWrite(originalPath, newContent); // Attempt async atomic write
} catch (err) {
// Rollback: restore from .bak backup
await fs.copyFile(`${originalPath}.bak`, originalPath);
throw new Error(`Atomic write failed β restored from backup: ${err.message}`);
}
class I18nManager {
// constructor: registers all five locale sets in a Map<LanguageCode, FullTranslationSet>; default 'en'
setLanguage(language: LanguageCode): void // throws Error on unsupported language code
getTranslations(): FullTranslationSet // the active translation set
get(category: keyof FullTranslationSet, key?: string): unknown // whole category block, or one tool entry by name
getAvailableLanguages(): LanguageCode[] // ['en', 'de', 'es', 'zh-CN', 'zh-TW']
getCurrentLanguage(): LanguageCode
}