Testing Guide

This guide covers local validation for Big RAG v1.1.0.

Prerequisites

• LM Studio installed and running for end-to-end plugin tests.
• At least one embedding model available in LM Studio.
• Node.js and npm installed.
• Dependencies installed with npm install.

Automated Checks

Run TypeScript compilation:

npx tsc

Run the current automated smoke tests:

node --test dist/tests/parseDocument.test.js dist/tests/fileExcludePatterns.test.js

Use npm run test when .lmstudio/entry.ts exists, because that script runs the full build before executing tests.

Manual Test Data

Create a small mixed-format dataset:

Optional files to add:

• A text-based PDF.
• A DOCX document.
• An ODT document.
• A CSV or TSV export.
• An image with readable text if OCR is enabled.

Basic Plugin Test

• Run the plugin in LM Studio development mode.
• Configure:
  • Documents Directory: ~/test-documents
  • Vector Store Directory: ~/.lmstudio/big-rag-test-db
  • Chunking Strategy: Character

Expected result:

• Sanity checks pass.
• Files are scanned and indexed.
• Retrieval returns relevant passages.
• The final prompt includes citations.

Chunking Tests

Character Chunking

Set:

Expected:

• Chunks are based on estimated tokens, not word counts.
• Long text is split around the configured size.
• Chunk boundaries avoid cutting a word when a nearby space exists.

Sentence Chunking

Set:

Expected:

• Prose chunks contain complete sentences where possible.
• A single sentence longer than the configured chunk size is kept whole.
• Overlap re-includes trailing sentences from the previous chunk.

Minimum Chunk Length

Set:

Expected:

• Short fragments, page numbers, and boilerplate chunks are filtered before embedding.
• Very small test files may produce zero chunks. Lower this setting to 0 for tiny fixtures.

Format Coverage Tests

Verify that these file types are discovered and parsed:

Expected:

• Text-like formats parse as plain text.
• DOCX parses through mammoth.
• ODT parses from content.xml when adm-zip is available.
• Unsupported or failed files are logged without stopping the whole indexing job.

Performance Tests

Parallel Embedding

Index a large text or PDF file that produces many chunks.

Expected:

• Embeddings are requested in batches of 8.
• Indexing is faster than serial per-chunk embedding.
• Aborting the request stops between batches.

Vector Store Cache

After indexing, send several chat messages without changing settings.

Expected:

• Repeated stats checks do not repeatedly scan every shard within the 10-second stats cache TTL.
• File hash checks use the in-memory index populated during vector store initialization.

Config Change Reset

Change one of these settings:

• Documents Directory
• Vector Store Directory
• Embedding Model

Expected:

• The in-memory vector store is closed and reset.
• Sanity checks run again.
• Retrieval uses the new configuration.

Max Query Length

Paste a long multi-paragraph query and set:

Expected:

• The query used for embedding is truncated to roughly 512 estimated tokens.
• Logs include a truncation message.
• The original user prompt remains available in the final prompt template.

CLI Test

Compile and run:

Optional CLI environment variables:

Expected:

• CLI indexing completes.
• The embedding manifest is written when chunks exist.
• The final summary reports successful, failed, skipped, updated, and new files.

Troubleshooting Checklist

• If no chunks are created, lower minChunkLength or inspect parser output.
• If retrieval returns noisy passages, raise minChunkLength or .

Reporting Issues

Include:

• Plugin version.
• LM Studio version.
• Operating system.
• Embedding model id.
• Dataset size and file types.
• Chunking, retrieval, OCR, and concurrency settings.
• Relevant logs and reproduction steps.