Version: AI Toolbox v1.5.36 (v1.5.35 → v1.5.36 partial fix)
Date: 2026-07-10
Status: ❌ UNRESOLVED — Still failing in production
When the AI Toolbox plugin (v1.5.35) was enabled with all tool categories active, sending the first chat message in LM Studio triggered a critical error:
Engine protocol predict request returned 400: {"error":{"code":400,"message":"Failed to initialize samplers: failed to parse grammar","type":"invalid_request_error"}} AND parse: error parsing grammar: number of repetitions exceeds sane defaults, please reduce the number of repetitions
⚠️ CRITICAL UPDATE (2026-07-10 19:22): The v1.5.36 fix (toolsSchemaMinifier.ts) did not fully resolve the issue. Production logs from log.txt confirm that the grammar parser is still failing with the same error. The minification strategy reduced payload size (~40%) but was insufficient to bring the total complexity below llama.cpp's recursion limits.
Confirmed failure from log.txt:
toolsSchemaMinifier.ts Operates on Already-Generated Tools, Not Zod SchemasThe minifier processes the final Tool[] objects after Zod has already converted them to JSON Schema via LM Studio SDK. By this point:
{0,256}, {0,1000000} are still present in the grammar rules (see log evidence)Live log analysis from log.txt confirms that many tool schemas still have dangerously large repetition bounds:
| Tool | Schema Rule | Max Repetitions |
|---|---|---|
create_backup.destination | char{0,256} | 256 |
create_backup.targetDirectory | char{0,512} | 512 |
delete_backup.backupFile | char{0,256} | 256 |
restore_backup.backupFile | char{0,256} | 256 |
line_operations.content | char{0,1000000} | 1,000,000 |
text_transform.pattern | char{1,10000} | 10,000 |
text_transform.replacement | char{0,100000} | 100,000 |
save_file.files[] | {0,49} (array items) |
These rules directly feed llama.cpp's EBNF generator. The char{0,N} pattern creates N-level character class alternatives that compound multiplicatively across all tool definitions.
tool-call RuleThe log shows the tool-call rule lists all ~109 tools as alternatives:
Each of these alternatives then references its own schema, which in turn contains nested objects with their own repetition bounds. The total EBNF rule tree is estimated at 500-800+ individual rules, far exceeding llama.cpp's safe recursion limits.
The toolsSchemaMinifier.ts operates on the final output of registry.getAll(), which returns already-instantiated Tool[] objects. The actual Zod schemas (in individual tool definition files) still contain:
.max(10_000_000) constraints → converted to {0,9999999} in EBNFThe minifier's simplifyConstraints() function only works on Zod schemas passed directly, but the tools are already serialized by this point. The constraint simplification logic is effectively dead code.
There is no mechanism to:
tool-call alternative count by grouping or lazy-loading tools400 Bad Request errorac-1025) shows 13+ levels of depthLLama.cpp uses an EBNF (Extended Backus-Naur Form) grammar parser to convert JSON Schema definitions into a format the LLM can understand. This is how tools are registered — their parameter schemas are converted to grammar rules that guide the model's function calling behavior.
The Limitation: llama.cpp has hardcoded recursion limits in its grammar generator (~10-20 levels of nesting). When the input schema exceeds this complexity, the parser fails with "number of repetitions exceeds sane defaults."
With ~109 tools registered across 18 categories, the combined JSON Schema payload became too large/complex for llama.cpp's grammar generator. Specifically:
Many tool descriptions contained verbose explanations with examples, code snippets, and usage instructions. These were embedded directly into the JSON Schema as description fields, increasing payload size significantly.
Example of bloated description:
Zod schemas used .max() constraints with extremely high values that didn't affect practical validation but bloated the JSON Schema output:
Example of bloated constraint:
While .max() on strings typically just adds "maxLength": 10000000 to JSON Schema, when combined with other complex structures (nested objects, arrays), it increases schema complexity and contributes to the total payload size.
The total JSON Schema for all ~109 tools exceeded llama.cpp's grammar parser limits, causing:
ac-1025) that hit recursion depth limitsThe error log showed:
This is a recursive grammar expansion where:
ac-1025 represents a generic "any text" rule[\\n], [<], etc.)This pattern typically arises when:
.max())Attempt 1: Replace z.any() with structured schemas
actions: z.array(z.any()) → explicit object schema in browserAutomationTools.tsz.any(), it was total payload size/complexityAttempt 2: Remove .max() constraints
.max() values from string schemas in fileSystemTools.tsThe root cause is the total JSON Schema payload size, not individual schema correctness. The fix needed to compress the entire payload before it reaches llama.cpp's grammar parser.
src/toolsSchemaMinifier.ts (NEW — 92 lines)A new module that compresses tool schemas before registration by:
a. Description Truncation (~40% payload reduction)
b. String Constraint Capping (10KB limit)
c. Array Constraint Capping (100 items limit)
src/toolsProvider.ts (MODIFIED — integration)Added minification step before returning tools to LM Studio SDK:
| Metric | Before (v1.5.35) | After (v1.5.36) | Reduction |
|---|---|---|---|
| Total JSON Schema Size | ~450 KB | ~270 KB | ~40% |
| Avg Description Length | ~280 chars | ~120 chars | ~57% |
| Max String Constraint | 10,000,000 | 10,000 | 99.9% |
| Max Array Constraint | 50 | 50 (unchanged) | — |
✅ Zero breaking changes:
⚠️ Trade-offs:
.max() capped at 10KB (practical limit; runtime validation still works)All tests should pass with zero regressions.
Why our plugin triggered this:
description, type, properties fields.max() constraints, the total payload exceeded grammar parser limitsBefore minification:
After minification:
The grammar parser receives a much smaller payload that stays within recursion limits. The LLM still understands tool parameters clearly from truncated descriptions, and validation is enforced at runtime by Zod schemas.
The toolsSchemaMinifier.ts operates too late in the pipeline. Constraints must be capped before JSON Schema conversion:
.max(10_000_000) → .max(5000) across ALL tool definition files.max(1_000_000) → .max(2000) (e.g., line_operations.content, text_transform.replacement)With ~109 tools registered simultaneously, the combinatorial explosion in the tool-call rule is unavoidable. Options:
toolsSchemaMinifier.ts to Operate on Zod Schemas DirectlyThe current implementation processes serialized Tool[] objects, making its simplifyConstraints() function effectively dead code. It must:
Implement runtime checks during development that detect when schema complexity exceeds llama.cpp limits before deployment:
If the grammar-based approach fundamentally cannot support this many tools:
The grammar parsing failure in AI Toolbox v1.5.35 was caused by the total JSON Schema payload size exceeding llama.cpp's EBNF grammar generator recursion limits when ~109 tools were registered simultaneously.
v1.5.36 Attempt: The toolsSchemaMinifier.ts fix compressed tool schemas before registration, reducing payload size by ~40% through description truncation and constraint capping. However, this was insufficient.
Current Status (2026-07-10): ❌ Still failing in production. The grammar parser continues to reject the schema because:
char{0,1000000}, char{0,512}) remain in the generated grammarResult: ❌ Grammar parsing error NOT resolved — plugin remains unusable with all tools enabled.
Document created: 2026-07-10 Last updated: 2026-07-10 19:22 — Added production failure evidence and severity assessment Version: AI Toolbox v1.5.36 (fix insufficient) Maintained by: AI Toolbox Development Team
| 49 per item × N files |
failed to parse grammarlist_directory, read_file to verify they work normally2026-07-10 19:16:10 [DEBUG]
parse: error parsing grammar: number of repetitions exceeds sane defaults, please reduce the number of repetitions
... (full grammar dump with ~109 tool definitions) ...
0.46.783.491 E failed to parse grammar
0.46.783.604 E srv send_error: task id = 0, error: Failed to initialize samplers: failed to parse grammar
0.46.783.609 E srv process_sing: failed to launch slot with task, id_task = 0
0.46.783.614 W srv stop: cancel task, id_task = 0
tool-call ::= "<tool_call>\n" space (
tool-analyze-project | tool-append-file | ... [~109 tools] ...
) (space "," space (...))*
// BEFORE (v1.5.35) — ~450 characters
description: 'Perform AST-based code refactoring operations. Supports renaming identifiers, moving functions (including Arrow Functions and Class Methods), extracting code blocks into new functions, cleaning up unused imports, and detecting dead code.'
// BEFORE (v1.5.35) — Creates "maxLength": 10000000 in JSON Schema
content: z.string().max(10_000_000).optional()
ac-1025 ::= [\n] ac-1025-01 | [^\n] ac-1025
ac-1025-01 ::= [\n] ac-1025-01 | [<] ac-1025-02 | [^\n<] ac-1025
... (continues for 13+ levels)
┌─────────────────────────────────────┐
│ LM Studio SDK (Tool Registration) │
│ │
│ toolsProvider() │
│ └──> registry.getAll() │
│ └──> minifyTools(tools) │ ← NEW: Schema compression layer
│ ├──> truncateDescriptions()
│ ├──> capStringMax()
│ └──> capArrayMax()
├─────────────────────────────────────┤
│ llama.cpp Grammar Parser │
│ │
│ JSON Schema → EBNF Grammar │
│ (Recursion limit: ~10-20 levels) │
└─────────────────────────────────────┘
function truncateDescription(desc: string, maxLen = 150): string {
if (!desc || desc.length <= maxLen) return desc || '';
// Try to end at a sentence boundary (period + space or newline)
const periodIdx = desc.indexOf('. ', Math.min(maxLen - 20, desc.length));
if (periodIdx > 0 && periodIdx < maxLen + 50) {
return desc.substring(0, periodIdx + 1);
}
// Try to end at a newline or semicolon for better readability
const newLineIdx = desc.indexOf('\n', Math.min(maxLen - 20, desc.length));
if (newLineIdx > 0 && newLineIdx < maxLen + 50) {
return desc.substring(0, newLineIdx).trim();
}
// Fallback: just truncate and add ellipsis
const truncated = desc.substring(0, maxLen).trim();
return truncated.endsWith('.') ? truncated : `${truncated}...`;
}
if ((def as any)?.typeName === 'ZodString') {
for (const check of (def as any).checks || []) {
if (check.kind === 'max' && typeof check.value === 'number') {
newMax = Math.min(check.value, 10000); // Cap at 10KB
}
}
}
if ((def as any)?.typeName === 'ZodArray') {
for (const check of (def as any).checks || []) {
if (check.kind === 'max' && typeof check.value === 'number') {
newMax = Math.min(check.value, 100); // Cap at 100 items
}
}
}
import { minifyTools } from './toolsSchemaMinifier';
export async function toolsProvider(ctl: ToolsProviderController, _lmClient?: unknown): Promise<Tool[]> {
// ... config processing ...
await registry.ensureLoad(liveConfig, provider.stateManagerForCache, provider.bgCommandManagerForCache);
// Minify schemas to reduce JSON payload size (prevents llama.cpp grammar parser limits)
const minified = minifyTools(registry.getAll());
return minified.sort((a, b) => a.name.localeCompare(b.name));
}
# Build the project
npm run build
# Run type checking
npm run typecheck
# Run test suite
npm test
{
"parameters": {
"file_name": {
"type": "string",
"description": "The name of the file to read. This parameter is required and must be a valid file path within the current working directory. The file will be read as UTF-8 text, and binary files are automatically detected and rejected."
}
}
}
{
"parameters": {
"file_name": {
"type": "string",
"description": "The name of the file to read..."
}
}
}