README.md
README.md
An LM Studio plugin that turns LM Studio into a client for any OpenAI-compatible API. Point it at a Base URL and a model, and you get full control over generation right inside the chat — sampling, reasoning, tools, and real-time diagnostics.
It is aimed chiefly at the Splash local model runner — a high-throughput, Apple-Silicon, OpenAI-compatible inference server. Its sampling, reasoning, request-control, and error-handling behaviours were shaped around how Splash responds; it will be equally happy to talk to any other OpenAI-compatible endpoint.
Built around @lmstudio/sdk, the official openai SDK, and zod.
Many powerful models live behind hosted, OpenAI-compatible endpoints rather than locally. This plugin lets you use them from within LM Studio without leaving the app, while keeping the familiar chat UX and exposing every knob that matters:
On each turn the plugin:
All diagnostics respect the selected Log Level; at debug the full effective request payload is dumped to the console.
All fields are editable from the plugin settings panel in LM Studio.
| Field | Type | Default | Description |
|---|---|---|---|
| Base URL | string | (empty) | OpenAI-compatible endpoint, e.g. https://api.sharti.ru/v1. Required. |
| API Key | string (protected) | (empty) | Bearer key. Leave empty if the endpoint needs no auth. |
| Model | string | (empty) | Model identifier to request, e.g. Qwen3.8-27B. Required. |
Tuning parameters below are sent only when Custom Sampling is enabled. When disabled, only structural fields (model, messages, tools, priority, timeout, preserve_thinking) are transmitted—all sampling knobs are omitted.
| Field | Range | Default | Description |
|---|---|---|---|
| Custom Sampling | bool | on | Gate for tuning parameters. Off → suppresses temperature, top-k/p/min-p, penalties, seed, max-tokens, reasoning effort/budget. |
| Temperature | 0–2 | 1.0 | Sampling temperature. |
| Top K | −1–100 | 20 | Highest-probability tokens considered. |
| Top P | 0–1 | 0.95 | Nucleus sampling threshold. |
| Min P | 0–1 | 0 | Minimum probability threshold. |
| Presence Penalty | −2–2 | 0 | Penalises tokens that already appear. |
| Frequency Penalty |
| Field | Options / Range | Default | Description |
|---|---|---|---|
| Reasoning Effort | default · none · minimal · low · medium · high · xhigh · max | default | Provider-specific reasoning level. |
| Thinking Token Budget | −1–128000 | -1 | Cap on thinking tokens (−1 = uncapped). |
Prior thinking is always preserved between turns (preserve_thinking).
| Field | Options | Default | Description |
|---|---|---|---|
| Tool Choice | auto · required · none | auto | How the model handles tool calls. |
| Parallel Tool Calls | bool | off | Allow multiple simultaneous tool calls. |
| Field | Options / Range | Default | Description |
|---|---|---|---|
| Priority | foreground · normal · background | normal | Queue priority at the provider. Sent regardless of Custom Sampling. |
| Request Timeout (s) | 0–600 | 0 | Max seconds per request (0 = no timeout). Sent regardless of Custom Sampling. |
| Max Retries | 0–5 | 3 | Retry attempts on HTTP 503. |
| Field | Options / Type | Default | Description |
|---|---|---|---|
| Log Level | minimal · normal · debug | normal | Controls console verbosity. minimal shows only warnings/errors; normal adds request summaries and per-request stats; debug additionally dumps the full effective payload. |
| Show Turn Summary in Chat | bool | on | Displays the aggregated turn-end summary (calls, tokens, speed, time) inline in the chat window. Automatically stripped from history before resending. |
"runner": "node" in its manifest)| Package | Role |
|---|---|
@lmstudio/sdk | LM Studio plugin API (chat, generators, config schema) |
openai | Client for OpenAI-compatible endpoints |
zod | Schema validation |
Dev: typescript, @types/node.
After pushing, enable the plugin in LM Studio's Plugins area and fill in Base URL, API Key, and Model under its settings.
Common provider errors are translated into clear, actionable messages:
| Symptom | Likely cause / fix |
|---|---|
| Prompt too long: exceeds the model's context limit | Start a new chat or trim the context. |
| Server out of memory / ran out of memory while processing | Reduce context size; retry later. |
| Invalid or missing API key | Double-check the API Key setting. |
| Model not found | Verify the Model name. |
| Rate limit exceeded (HTTP 429) | Wait and retry. |
| Access forbidden (HTTP 403) | Check host/origin allow-lists on the server side. |
| Server unavailable (HTTP 503) | Handled automatically — retried with backoff up to Max Retries. |
| Response truncated: max token limit reached. | Increase Max Tokens or start a shorter prompt. |
| Response blocked by content filter. | Adjust the prompt; the provider's safety filter intervened. |
Tip: raise Log Level to debug to see the full effective request payload in the console — invaluable when diagnosing unexpected provider behaviour.
Distributed under the Apache License 2.0.
An LM Studio plugin that turns LM Studio into a client for any OpenAI-compatible API. Point it at a Base URL and a model, and you get full control over generation right inside the chat — sampling, reasoning, tools, and real-time diagnostics.
It is aimed chiefly at the Splash local model runner — a high-throughput, Apple-Silicon, OpenAI-compatible inference server. Its sampling, reasoning, request-control, and error-handling behaviours were shaped around how Splash responds; it will be equally happy to talk to any other OpenAI-compatible endpoint.
Built around @lmstudio/sdk, the official openai SDK, and zod.
Many powerful models live behind hosted, OpenAI-compatible endpoints rather than locally. This plugin lets you use them from within LM Studio without leaving the app, while keeping the familiar chat UX and exposing every knob that matters:
On each turn the plugin:
All diagnostics respect the selected Log Level; at debug the full effective request payload is dumped to the console.
All fields are editable from the plugin settings panel in LM Studio.
| Field | Type | Default | Description |
|---|---|---|---|
| Base URL | string | (empty) | OpenAI-compatible endpoint, e.g. https://api.sharti.ru/v1. Required. |
| API Key | string (protected) | (empty) | Bearer key. Leave empty if the endpoint needs no auth. |
| Model | string | (empty) | Model identifier to request, e.g. Qwen3.8-27B. Required. |
Tuning parameters below are sent only when Custom Sampling is enabled. When disabled, only structural fields (model, messages, tools, priority, timeout, preserve_thinking) are transmitted—all sampling knobs are omitted.
| Field | Range | Default | Description |
|---|---|---|---|
| Custom Sampling | bool | on | Gate for tuning parameters. Off → suppresses temperature, top-k/p/min-p, penalties, seed, max-tokens, reasoning effort/budget. |
| Temperature | 0–2 | 1.0 | Sampling temperature. |
| Top K | −1–100 | 20 | Highest-probability tokens considered. |
| Top P | 0–1 | 0.95 | Nucleus sampling threshold. |
| Min P | 0–1 | 0 | Minimum probability threshold. |
| Presence Penalty | −2–2 | 0 | Penalises tokens that already appear. |
| Frequency Penalty |
| Field | Options / Range | Default | Description |
|---|---|---|---|
| Reasoning Effort | default · none · minimal · low · medium · high · xhigh · max | default | Provider-specific reasoning level. |
| Thinking Token Budget | −1–128000 | -1 | Cap on thinking tokens (−1 = uncapped). |
Prior thinking is always preserved between turns (preserve_thinking).
| Field | Options | Default | Description |
|---|---|---|---|
| Tool Choice | auto · required · none | auto | How the model handles tool calls. |
| Parallel Tool Calls | bool | off | Allow multiple simultaneous tool calls. |
| Field | Options / Range | Default | Description |
|---|---|---|---|
| Priority | foreground · normal · background | normal | Queue priority at the provider. Sent regardless of Custom Sampling. |
| Request Timeout (s) | 0–600 | 0 | Max seconds per request (0 = no timeout). Sent regardless of Custom Sampling. |
| Max Retries | 0–5 | 3 | Retry attempts on HTTP 503. |
| Field | Options / Type | Default | Description |
|---|---|---|---|
| Log Level | minimal · normal · debug | normal | Controls console verbosity. minimal shows only warnings/errors; normal adds request summaries and per-request stats; debug additionally dumps the full effective payload. |
| Show Turn Summary in Chat | bool | on | Displays the aggregated turn-end summary (calls, tokens, speed, time) inline in the chat window. Automatically stripped from history before resending. |
"runner": "node" in its manifest)| Package | Role |
|---|---|
@lmstudio/sdk | LM Studio plugin API (chat, generators, config schema) |
openai | Client for OpenAI-compatible endpoints |
zod | Schema validation |
Dev: typescript, @types/node.
After pushing, enable the plugin in LM Studio's Plugins area and fill in Base URL, API Key, and Model under its settings.
Common provider errors are translated into clear, actionable messages:
| Symptom | Likely cause / fix |
|---|---|
| Prompt too long: exceeds the model's context limit | Start a new chat or trim the context. |
| Server out of memory / ran out of memory while processing | Reduce context size; retry later. |
| Invalid or missing API key | Double-check the API Key setting. |
| Model not found | Verify the Model name. |
| Rate limit exceeded (HTTP 429) | Wait and retry. |
| Access forbidden (HTTP 403) | Check host/origin allow-lists on the server side. |
| Server unavailable (HTTP 503) | Handled automatically — retried with backoff up to Max Retries. |
| Response truncated: max token limit reached. | Increase Max Tokens or start a shorter prompt. |
| Response blocked by content filter. | Adjust the prompt; the provider's safety filter intervened. |
Tip: raise Log Level to debug to see the full effective request payload in the console — invaluable when diagnosing unexpected provider behaviour.
Distributed under the Apache License 2.0.
Base URL and an optional API Key (leave the key empty if the endpoint doesn't require auth).none → max), set a thinking-token budget, and keep prior thinking preserved between turns. Reasoning is detected from reasoning_content and … blocks and surfaced as proper reasoning fragments in the UI.auto / required / none), supports parallel tool calls, and streams tool-call deltas incrementally.503s, per-request timeouts, request priority, graceful interruption, and table-driven error mapping (rate limits, context overflow, bad key, etc.).minimal / normal / debug). Warnings and errors reach the chat with visual badges; detailed request dumps and payload introspection stay in the console at higher levels.tool results so malformed histories don't break the provider.preserve_thinking) are always included; tuning parameters (sampling, reasoning) are included only when Custom Sampling is on.503) it retries with exponential backoff (1s → 2s → 4s → 8s → 10s), up to the configured maximum.finish_reason), the aggregator emits a consolidated summary to the console—and optionally to the chat.| −2–2 |
0 |
| Penalises tokens by how often they appear. |
| Repetition Penalty | 0.1–2 | 1 | Penalises repeated tokens. |
| Seed | −1–999999999 | -1 | Random seed (−1 = random). |
| Max Tokens | 0–128000 | 4000 | Maximum tokens to generate. |
sharti-connector/
├── manifest.json # LM Studio plugin manifest (type, runner, owner, revision)
├── package.json # Scripts & dependencies
├── tsconfig.json # TypeScript config
└── src/
├── index.ts # Entry point: registers config schematics + generator
├── config.ts # Schema definition + CONFIG_DEFAULTS + ShartiConfig type
├── constants.ts # Shared delimiters (reasoning tags, turn-summary markers)
├── types.ts # Wire types, request body, stream chunk, stats interfaces
├── generator.ts # Orchestrator: validate → convert → request → stream → stats
├── request.ts # Client factory, payload assembly, debug-dump helper
├── errors.ts # Table-driven provider error classification & messaging
├── stats.ts # Per-turn accumulator, session-scoped LRU store
├── convert/
│ ├── messages.ts # History↔OpenAI conversion, sanitization, conv-key derivation
│ └── stream.ts # Delta rendering, reasoning blocks, tool-call assembly, metrics
└── utils/
├── logger.ts # ShartiLogger: leveled channels (debug/info/warn/error/chat)
├── coerce.ts # Safe runtime type guards (asString/asBool/asNumber)
└── block.ts # Structured multi-line log block formatter
cd sharti-connector
# Install dependencies
npm install # or: bun install
# Run in watch mode against a connected LM Studio instance
npm run dev # → lms dev
# Push/build the plugin
npm run push # → lms push
Base URL and an optional API Key (leave the key empty if the endpoint doesn't require auth).none → max), set a thinking-token budget, and keep prior thinking preserved between turns. Reasoning is detected from reasoning_content and … blocks and surfaced as proper reasoning fragments in the UI.auto / required / none), supports parallel tool calls, and streams tool-call deltas incrementally.503s, per-request timeouts, request priority, graceful interruption, and table-driven error mapping (rate limits, context overflow, bad key, etc.).minimal / normal / debug). Warnings and errors reach the chat with visual badges; detailed request dumps and payload introspection stay in the console at higher levels.tool results so malformed histories don't break the provider.preserve_thinking) are always included; tuning parameters (sampling, reasoning) are included only when Custom Sampling is on.503) it retries with exponential backoff (1s → 2s → 4s → 8s → 10s), up to the configured maximum.finish_reason), the aggregator emits a consolidated summary to the console—and optionally to the chat.| −2–2 |
0 |
| Penalises tokens by how often they appear. |
| Repetition Penalty | 0.1–2 | 1 | Penalises repeated tokens. |
| Seed | −1–999999999 | -1 | Random seed (−1 = random). |
| Max Tokens | 0–128000 | 4000 | Maximum tokens to generate. |
sharti-connector/
├── manifest.json # LM Studio plugin manifest (type, runner, owner, revision)
├── package.json # Scripts & dependencies
├── tsconfig.json # TypeScript config
└── src/
├── index.ts # Entry point: registers config schematics + generator
├── config.ts # Schema definition + CONFIG_DEFAULTS + ShartiConfig type
├── constants.ts # Shared delimiters (reasoning tags, turn-summary markers)
├── types.ts # Wire types, request body, stream chunk, stats interfaces
├── generator.ts # Orchestrator: validate → convert → request → stream → stats
├── request.ts # Client factory, payload assembly, debug-dump helper
├── errors.ts # Table-driven provider error classification & messaging
├── stats.ts # Per-turn accumulator, session-scoped LRU store
├── convert/
│ ├── messages.ts # History↔OpenAI conversion, sanitization, conv-key derivation
│ └── stream.ts # Delta rendering, reasoning blocks, tool-call assembly, metrics
└── utils/
├── logger.ts # ShartiLogger: leveled channels (debug/info/warn/error/chat)
├── coerce.ts # Safe runtime type guards (asString/asBool/asNumber)
└── block.ts # Structured multi-line log block formatter
cd sharti-connector
# Install dependencies
npm install # or: bun install
# Run in watch mode against a connected LM Studio instance
npm run dev # → lms dev
# Push/build the plugin
npm run push # → lms push