CONTEXT.md
Snapshot of what was built, the bugs hit, and why the code looks the way it does. Written so a future session can pick up without rereading the chat.
An LM Studio plugin that exposes Vercel agent-browser
(vercel-labs/agent-browser, npm agent-browser, native Rust CDP CLI)
as model tools. Owner mbagley, plugin name agent-browser.
Built against the canonical LM Studio scaffold ā verified by cloning
lmstudio/wikipedia and lmstudio/dice via lms clone and matching
manifest.json, package.json, tsconfig.json, and the
PluginContext.withConfigSchematics / withToolsProvider registration shape.
browser_open, browser_snapshot, browser_screenshot, browser_click,
browser_type, browser_press, browser_get_text, browser_page_info,
browser_wait, browser_find, browser_navigate, browser_eval,
browser_close. Curated from agent-browser's 100+ subcommand surface ā chosen
to be the set an LLM actually needs without polluting the model's tool list.
LM Studio's plugin runtime does not inherit the user's shell PATH.
spawn("npx", ...) fails because the plugin's process.env.PATH is something
like /usr/bin:/bin ā Homebrew's /opt/homebrew/bin (where npx lives on
Apple silicon) isn't on it.
Constraint from the user: do NOT bundle agent-browser as a plugin
dependency. The plugin must work via the host system's npx / agent-browser.
Fix (src/agentBrowser.ts): resolveAbsolute(cmd) resolves npx (or
whatever the user configures) to an absolute path before spawn:
/opt/homebrew/bin, /usr/local/bin),
system bins, plus ~/.volta/bin, ~/.fnm/current/bin,
~/.nvm/current/bin, ~/.asdf/shims, ~/.bun/bin, ~/.local/bin,
~/.npm-global/bin, and ~/node_modules/.bin./bin/sh -lc 'command -v <cmd>' (login shell sources the
user's profile) if scanning misses.Spawned children also receive an enriched PATH (enrichedPath()) so npx
itself can find node and resolve agent-browser.
Symptom: plugin loaded fine, every tool call failed silently with the WS-close message in LM Studio.
Root cause: agent-browser's CLI parses subcommands first and global flags after the subcommand. The wrapper was emitting
agent-browser --session lmstudio --timeout 30000 open https://...
agent-browser saw --timeout as the subcommand, exited with
Unknown command: --timeout in ~150ms, the tool returned an error string, and
LM Studio's tool-call layer surfaced "WebSocket closed by the client"
downstream.
Fix (src/agentBrowser.ts:152): subcommand args go first, global flags
last:
agent-browser open https://... --session lmstudio --timeout 30000
Verified by running the binary directly, then by the integration test below.
test/verify-nytimes.js, run via npm run test:verify:
process.env.PATH to /usr/bin:/bin to mimic the LM Studio plugin
runtime (so PATH discovery is exercised, not bypassed by the developer's
shell).open https://www.nytimes.com ā snapshot -c -d 3 ā get title
ā get url ā close.[ref=e; title matches
/new york times/i; URL contains nytimes.com.| Field | Type | Default | Notes |
|---|---|---|---|
binCommand | string | npx agent-browser | Override to absolute path of npx or agent-browser if discovery fails. |
session | string | lmstudio | agent-browser --session name. |
headed | boolean | false | Adds --headed if true. |
timeoutMs | numeric (1000ā300000, int) | 30000 | Passed as --timeout (ms). |
screenshotDir | string | "" (cwd) | Where browser_screenshot writes files. |
userAgent | string | Chrome 131 / macOS UA | --user-agent. Blank disables. |
viewport | string | 1440x900 | WxH. Parsed into --args "--window-size=W,H". |
acceptLanguage | string | en-US,en;q=0.9 | Becomes --headers '{"Accept-Language":"ā¦"}'. |
colorScheme | select | light | --color-scheme. no-preference is the CLI's own default, so it's omitted. |
extraBrowserArgs | string | --disable-blink-features=AutomationControlled | Comma-separated Chromium flags merged into --args. |
createConfigSchematics() field types verified against the SDK's
basicKVValueTypesLibrary declaration in @lmstudio/sdk/dist/index.d.ts:
numeric accepts min, max, int, step; select takes options as
strings or {value, displayName} objects.
The five human-emulation fields are appended to globalFlags in
runAgentBrowser, so they tail the subcommand and respect invariant #2.
agent-browser applies these when its daemon launches Chrome; once a session
is already running they're effectively no-ops, so re-sending on every command
is safe (matches how --session/--timeout already work).
viewport and extraBrowserArgs both target --args and are merged into a
single comma-separated value via buildLaunchArgs. Empty fields drop out
individually ā clearing userAgent doesn't affect viewport, etc.
runAgentBrowser: [...prefix, ...args, ...globalFlags].
Do not reorder ā the CLI parses subcommands first.resolveCommand is async because the shell-fallback path uses execFile.env: { ...process.env, PATH: enrichedPath() } so npx can
re-resolve descendants.signal.addEventListener("abort", ...) sends SIGTERM, and
the listener is removed on close/error. No SIGKILL escalation yet ā add one
if hangs surface.npm run test:verify is meant for local validation; it makes
real network calls to nytimes.com and depends on Chrome for Testing being
installed via agent-browser install.browser_screenshot
and assert the file exists.runAgentBrowser(settings, ["--version"], { includeSession: false, includeTimeout: false })
in toolsProvider (don't await) to pre-download.cd /Users/mbagley/working/lmstudio-agent-browser-plugin npm install agent-browser install # one-time: download Chrome for Testing (host-side) npm run dev # lms dev ā load into LM Studio npm run test:verify # network integration test
LM Studio Hub: mbagley/agent-browser (revision 1, not yet pushed).
manifest.json type=plugin, runner=node, owner=mbagley, name=agent-browser, revision=1
package.json @lmstudio/sdk 1.4.0, zod 3.24.1, typescript 5.6, @types/node
scripts: dev, push, build, test:verify
tsconfig.json CommonJS, ES2021, strict, rootDir=src, outDir=dist
.gitignore .lmstudio, node_modules, dist
README.md user-facing setup + tools table
CONTEXT.md this file
src/index.ts main(context) ā registers configSchematics + toolsProvider
src/configSchematics.ts binCommand, session, headed, timeoutMs, screenshotDir
src/agentBrowser.ts spawn-based CLI wrapper (PATH discovery + arg ordering)
src/toolsProvider.ts 13 curated tools wrapping agent-browser subcommands
test/verify-nytimes.js network integration test against nytimes.com