Project Context: Web Tools Plugin for LM Studio

Overview

lms-plugin-web-tools is an LM Studio plugin that exposes four web-oriented tools to local LLMs:

Web Search — DuckDuckGo web search returning ranked [title, url] pairs (with optional snippet/date/type/description enrichment).
Image Search — Bing image search; discovery-only, returning per-image records with the full-resolution remote URL plus Bing's title and source-page metadata. No files are written to disk.
Visit Website — fetches a URL and returns its title, first-level headings, and a search-term-aware slice of its readable content as Markdown (default) or plain text.
Fetch Images — downloads images from a list of HTTP(S) URLs and/or scraped from a page into the chat's working directory so the assistant can embed them in a reply; each record carries filename, alt, and title metadata plus a Markdown image reference.

Built on @lmstudio/sdk, it requires Node.js >= 22 and targets ES2023 / CommonJS. The project is licensed under MIT. It descends from Daniel Sig's original lms-plugin-duckduckgo and lms-plugin-visit-website plugins, merged and extended by Nigel Packer. The plugin is at revision 5 (see manifest.json).

Directory Structure

Request Flow

Key Dependencies

Dependency	Purpose
`@lmstudio/sdk`	Plugin/tool registration with LM Studio
`impit`	HTTP client with TLS + header fingerprinting (required to bypass DuckDuckGo and Bing anti-bot)
`jsdom`	HTML parsing for web search results and website content
`@mozilla/readability`	Article text extraction for Visit Website (boilerplate removal)
`turndown`	HTML → Markdown conversion for Visit Website's content field
`cacache`	Disk-backed cache store for all three TTL caches
`zod`	Runtime validation of tool parameter schemas
`bottleneck`	Backs the shared `RateLimiter`
`file-type`	MIME sniffing for downloaded images

Build & Development Commands

Note: No test suite is currently configured.

Pre-commit Hook

A local pre-commit hook at .git/hooks/pre-commit runs lint, format:check, and knip sequentially and aborts the commit on any failure. The hook is not committed to the repo — fresh clones need to reinstall it. Bypass with git commit --no-verify when necessary.

TypeScript Configuration

Target: ES2023
Module: ES modules in source → compiled to CommonJS (outDir: "dist")

ESLint / Code Quality

The project uses a comprehensive ESLint configuration (ESLint v10+) with these notable rules:

Plugin Configuration (UI Fields)

All config fields default to 0 or "Auto", meaning the LLM assistant decides at runtime:

Field	Type	Default	Range	Purpose
`limitWebResults`	boolean	true	true/false	When enabled, caps web-search results at `webMaxResults`; when disabled, returns the full DDG page (~30).
`webMaxResults`	numeric	10	1–30	Max web-search results per page; hidden when `limitWebResults` is off (plugin-only, not exposed as tool parameter).
`limitImageResults`	boolean	true	true/false	When enabled, caps image-search results at `imageMaxResults`; when disabled, returns the full Bing page (~35).
`imageMaxResults`	numeric	10	1–35	Max image-search results per page; hidden when `limitImageResults` is off (plugin-only, not exposed as tool parameter). Tops out at Bing's native page size of 35.
`safeSearch`	select	Auto	strict/moderate/off	Safe-search mode applied to both DDG web search and Bing image search.
`maxImages`	numeric	-1 (auto)	-1–200	Fetch Images: max images scraped when a `websiteURL` is provided
`contentLimit`	numeric	0 (auto)	0–100000	Visit Website: max characters of text (plugin-only, not exposed as tool parameter)
`contentFormat`

Content Extraction (Visit Website)

src/parsers/page/page-text.ts feeds raw HTML to @mozilla/readability to strip boilerplate (nav, sidebars, comments), then routes Readability's .content HTML through either:

Markdown (default) — src/text/html-to-markdown.ts via a shared turndown service (ATX headings, bullets, fenced code, inline links, inline images). /// are stripped before conversion.

Both paths share src/text/normalize-blank-lines.ts for trailing-whitespace and blank-line collapsing. Both contentFormat (plugin select field, default "markdown") and contentLimit are plugin-only — neither is exposed as a tool parameter so the model cannot override the user-set or default values. The tool still returns contentLength, the pre-truncation character count, so the model can detect truncation and refine with findInPage.

Image Extraction (Fetch Images)

src/parsers/page/page-images.ts scrapes <img> tags in document order, resolves relative src against the page URL, deduplicates, and returns { src, alt, title } tuples (up to maxImages). Fetch Images then downloads each via downloadImages and returns { filename, alt, title, image } on success or { filename, alt, title, error } on failure. filename is derived from the URL's last path segment via src/fs/url-filename.ts. The intended workflow is Image Search → Fetch Images → embed the returned Markdown in the reply; Image Search itself does not write to disk.

Caches

Two disk-backed TTL caches (via cacache) are constructed once in toolsProvider and shared across tools. They persist across plugin reloads — clearing requires removing the cacache directory at ~/.lmstudio/plugin-data/lms-plugin-duckduckgo-cache:

Cache	Max entries	TTL config	Default
Search results (subdir `search-enriched`)	100	`searchCacheTtlSeconds`	15 min
Website HTML (subdir `website`)	50	`websiteCacheTtlSeconds`	10 min

Image search is not cached — Bing's metadata payload is small and the rate limiter alone caps fetch rate.

Cache sizes and subdirs are defined in src/tools-provider.ts; the TTLCache implementation is in src/cache/ttl-cache.ts. TTL defaults live in src/config/resolve-config.ts.

Error Hierarchies

Two error hierarchies are load-bearing:

FetchError (src/http/fetch-error.ts) — HTTP/network failures, carries url and optional cause.
NoResultsError base with NoWebResultsError / ().

formatToolError in src/errors/tool-error.ts converts these into user-facing strings per tool kind (web-search, image-search, website, image-download), including abort-detection via DOMException.name === "AbortError".

Important Implementation Notes

src/ ├── index.ts # Plugin entry point (registers config + tools provider) ├── tools-provider.ts # Registers all four tools via SDK │ ├── tools/ # Tool implementations │ ├── web-search-tool.ts │ ├── image-search-tool.ts │ ├── visit-website-tool.ts │ └── fetch-images-tool.ts │ ├── duckduckgo/ # DuckDuckGo web search │ ├── index.ts │ ├── search-web.ts # Web search request + response handling │ ├── safe-search.ts # SafeSearch type + DuckDuckGo `p` param encoding │ └── build-urls.ts # URL construction for web-search requests │ ├── bing/ # Bing image search │ ├── index.ts │ ├── search-images.ts # Fetches Bing's image-search HTML page │ ├── parse-results.ts # Extracts image tiles from `<a class="iusc" m="...">` │ └── build-urls.ts # URL construction (q, first, adlt) │ ├── config/ # Configuration resolution │ ├── auto-sentinel.ts # Special value (0/Auto) meaning "let the LLM decide" │ ├── config-schematics.ts # LM Studio UI configuration schema │ └── resolve-config.ts # Merges plugin UI settings with per-call overrides │ ├── cache/ # Cache infrastructure │ ├── index.ts │ ├── ttl-cache.ts # Disk-backed TTL cache via cacache │ ├── cached-search-results.ts # Cached search results payload type │ └── search-cache-key.ts # Deterministic cache key generation │ ├── errors/ # Error hierarchies │ ├── index.ts │ ├── no-results-error.ts # NoResultsError, NoWebResultsError, NoImageResultsError │ └── tool-error.ts # formatToolError, isAbortError, errorMessage │ ├── http/ # HTTP layer │ ├── index.ts │ ├── fetch-ok.ts # Fetch with error wrapping │ ├── fetch-error.ts # FetchError class (carries url + optional cause) │ ├── impit-client.ts # Shared impit client creation │ └── retry.ts # withRetry wrapper (maxRetries, backoff config) │ ├── images/ # Image download utilities │ ├── index.ts │ ├── download-image.ts # Single image download │ └── download-images.ts # Batch image download │ ├── parsers/ # Generic HTML parsing │ ├── index.ts │ ├── search-results.ts # Web search: HTML via jsdom, .result__a │ ├── image-extensions.ts # Supported-image-extension predicates (URL, content-type) │ └── page/ # Page-level extraction │ ├── page-images.ts # Scrapes <img> tags for Fetch Images │ └── page-text.ts # Feeds HTML to @mozilla/readability │ ├── text/ # Text scraping utilities │ ├── index.ts │ ├── html-to-markdown.ts # HTML → Markdown via turndown service │ ├── html-to-text.ts # HTML → plain text (DOM walk, block-boundary newlines) │ ├── normalize-text.ts # Text normalization utilities │ └── normalize-blank-lines.ts # Trailing-whitespace and blank-line collapsing │ ├── timing/ # Rate limiting │ ├── index.ts │ ├── rate-limiter.ts # Shared RateLimiter (backed by bottleneck) │ └── per-host-rate-limiter.ts # Per-host limiter for the web-search enrichment fan-out │ ├── fs/ # Filesystem utilities │ ├── index.ts │ ├── url-filename.ts # Derives filename from URL's last path segment │ ├── lmstudio-home.ts # Finds LM Studio home directory │ └── markdown-path.ts # Converts file paths to markdown references │ └── website/ # Website fetching logic ├── index.ts └── fetch-website.ts