Project Context: Web Tools Plugin for LM Studio

Overview

lms-plugin-web-tools is an LM Studio plugin that exposes four web-oriented tools to local LLMs:

Web Search — DuckDuckGo web search returning ranked [title, url] pairs.
Image Search — DuckDuckGo image search; matching images are downloaded to the working directory so the assistant can display them.
Visit Website — fetches a URL and returns its title, first-level headings, and a search-term-aware slice of its readable content as Markdown (default) or plain text.
View Images — downloads images from a list of URLs and/or scraped from a page so the assistant can display them; each record carries filename, alt, and title metadata.

Built on @lmstudio/sdk, it requires Node.js >= 22 and targets ES2023 / CommonJS. The project is licensed under MIT. It descends from Daniel Sig's original lms-plugin-duckduckgo and lms-plugin-visit-website plugins, merged and extended by Nigel Packer. The plugin is at revision 5 (see manifest.json).

Directory Structure

Request Flow

Key Dependencies

Dependency	Purpose
`@lmstudio/sdk`	Plugin/tool registration with LM Studio
`impit`	HTTP client with TLS + header fingerprinting (required to bypass DuckDuckGo anti-bot)
`jsdom`	HTML parsing for web search results and website content
`@mozilla/readability`	Article text extraction for Visit Website (boilerplate removal)
`turndown`	HTML → Markdown conversion for Visit Website's content field
`cacache`	Disk-backed cache store for all three TTL caches
`zod`	Runtime validation of tool parameter schemas
`bottleneck`	Backs the shared `RateLimiter`
`file-type`	MIME sniffing for downloaded images

Build & Development Commands

Note: No test suite is currently configured.

Pre-commit Hook

A local pre-commit hook at .git/hooks/pre-commit runs lint, format:check, and knip sequentially and aborts the commit on any failure. The hook is not committed to the repo — fresh clones need to reinstall it. Bypass with git commit --no-verify when necessary.

TypeScript Configuration

Target: ES2023
Module: ES modules in source → compiled to CommonJS (outDir: "dist")

ESLint / Code Quality

The project uses a comprehensive ESLint configuration (ESLint v10+) with these notable rules:

Plugin Configuration (UI Fields)

All config fields default to 0 or "Auto", meaning the LLM assistant decides at runtime:

Field	Type	Default	Range	Purpose
`limitWebResults`	boolean	true	true/false	When enabled, caps web-search results at `webMaxResults`; when disabled, returns the full DDG page (~30).
`webMaxResults`	numeric	10	1–30	Max web-search results per page; hidden when `limitWebResults` is off (plugin-only, not exposed as tool parameter).
`limitImageResults`	boolean	true	true/false	When enabled, caps image-search results at `imageMaxResults`; when disabled, returns the full DDG page (~100).
`imageMaxResults`	numeric	10	1–100	Max image-search results per page; hidden when `limitImageResults` is off (plugin-only, not exposed as tool parameter).
`safeSearch`	select	Auto	strict/moderate/off	DuckDuckGo safe search level
`maxImages`	numeric	-1 (auto)	-1–200	View Images: max images scraped when a `websiteURL` is provided
`contentLimit`	numeric	0 (auto)	0–100000	Visit Website: max characters of text (plugin-only, not exposed as tool parameter)
`contentFormat`	select	markdown

Content Extraction (Visit Website)

src/parsers/page/page-text.ts feeds raw HTML to @mozilla/readability to strip boilerplate (nav, sidebars, comments), then routes Readability's .content HTML through either:

Markdown (default) — src/text/html-to-markdown.ts via a shared turndown service (ATX headings, bullets, fenced code, inline links, inline images). /// are stripped before conversion.

Both paths share src/text/normalize-blank-lines.ts for trailing-whitespace and blank-line collapsing. Both contentFormat (plugin select field, default "markdown") and contentLimit are plugin-only — neither is exposed as a tool parameter so the model cannot override the user-set or default values. The tool still returns contentLength, the pre-truncation character count, so the model can detect truncation and refine with findInPage.

Image Extraction (View Images)

src/parsers/page/page-images.ts scrapes <img> tags in document order, resolves relative src against the page URL, deduplicates, and returns { src, alt, title } tuples (up to maxImages). View Images then downloads each via downloadImages and returns { filename, alt, title, image } on success or { filename, alt, title, error } on failure. filename is derived from the URL's last path segment via src/fs/url-filename.ts.

Caches

Three disk-backed TTL caches (via cacache) are constructed once in toolsProvider and shared across tools. They persist across plugin reloads — clearing requires removing the cacache directory at ~/.lmstudio/plugin-data/lms-plugin-duckduckgo-cache:

Cache	Max entries	TTL config	Default
Search results	100	`searchCacheTtlSeconds`	15 min
VQD token	50	`vqdCacheTtlSeconds`	10 min
Website HTML	50	`websiteCacheTtlSeconds`	10 min

Cache sizes and subdirs are defined in src/tools-provider.ts; the TTLCache implementation is in src/cache/ttl-cache.ts. TTL defaults live in src/config/resolve-config.ts.

Error Hierarchies

Three error hierarchies are load-bearing:

formatToolError in src/errors/tool-error.ts converts these into user-facing strings per tool kind (web-search, image-search, website, image-download), including abort-detection via DOMException.name === "AbortError".

Important Implementation Notes

src/ ├── index.ts # Plugin entry point (registers config + tools provider) ├── tools-provider.ts # Registers all four tools via SDK │ ├── tools/ # Tool implementations │ ├── web-search-tool.ts │ ├── image-search-tool.ts │ ├── visit-website-tool.ts │ └── view-images-tool.ts │ ├── duckduckgo/ # DuckDuckGo API logic │ ├── index.ts │ ├── search-web.ts # Web search request + response handling │ ├── search-images.ts # Image search request + response handling │ ├── fetch-vqd-token.ts # Scrape VQD token from DuckDuckGo homepage │ ├── vqd-token-error.ts │ ├── safe-search.ts # Encodes safe-search setting to DuckDuckGo `p` param │ └── build-urls.ts # URL construction for search requests │ ├── config/ # Configuration resolution │ ├── auto-sentinel.ts # Special value (0/Auto) meaning "let the LLM decide" │ ├── config-schematics.ts # LM Studio UI configuration schema │ └── resolve-config.ts # Merges plugin UI settings with per-call overrides │ ├── cache/ # Cache infrastructure │ ├── index.ts │ ├── ttl-cache.ts # Disk-backed TTL cache via cacache │ ├── cached-search-results.ts # Cached search results payload type │ └── search-cache-key.ts # Deterministic cache key generation │ ├── errors/ # Error hierarchies │ ├── index.ts │ ├── no-results-error.ts # NoResultsError, NoWebResultsError, NoImageResultsError │ └── tool-error.ts # formatToolError, isAbortError, errorMessage │ ├── http/ # HTTP layer │ ├── index.ts │ ├── fetch-ok.ts # Fetch with error wrapping │ ├── fetch-error.ts # FetchError class (carries url + optional cause) │ ├── impit-client.ts # Shared impit client creation │ └── retry.ts # withRetry wrapper (maxRetries, backoff config) │ ├── images/ # Image download utilities │ ├── index.ts │ ├── download-image.ts # Single image download │ └── download-images.ts # Batch image download │ ├── parsers/ # HTML/JSON parsing │ ├── index.ts │ ├── search-results-parser.ts # Web search: HTML via jsdom, .result__a │ ├── image-results-parser.ts # Image search: JSON parsing │ ├── vqd-parser.ts # VQD token: homepage input[name="vqd"] │ └── page/ # Page-level extraction │ ├── page-images.ts # Scrapes <img> tags for View Images │ └── page-text.ts # Feeds HTML to @mozilla/readability │ ├── text/ # Text scraping utilities │ ├── index.ts │ ├── html-to-markdown.ts # HTML → Markdown via turndown service │ ├── html-to-text.ts # HTML → plain text (DOM walk, block-boundary newlines) │ ├── normalize-text.ts # Text normalization utilities │ └── normalize-blank-lines.ts # Trailing-whitespace and blank-line collapsing │ ├── timing/ # Rate limiting and delay utilities │ ├── index.ts │ ├── rate-limiter.ts # Shared RateLimiter (backed by bottleneck) │ └── sleep.ts # Async delay utility │ ├── fs/ # Filesystem utilities │ ├── index.ts │ ├── url-filename.ts # Derives filename from URL's last path segment │ ├── lmstudio-home.ts # Finds LM Studio home directory │ └── markdown-path.ts # Converts file paths to markdown references │ └── website/ # Website fetching logic ├── index.ts └── fetch-website.ts