Project Files
docs / DEVELOPENT.md
Image generation plugin for LM Studio using a Draw Things backend (HTTP or gRPC).
Tools provided:
generate_image — Text-to-Image and Image-to-Image with variants, previews, and file outputsreview_image — Request a one-shot re-view of specific earlier images (attachments/variants/images/pictures)Draw Things gRPC uses strict TLS with bundled certs.
This repo is intended to be loaded locally via the runner script at .lmstudio/production.js.
npm install && npm run build.lmstudio/production.js (it loads dist/index.js)The script checks for toolchain prerequisites (on macOS), installs dependencies, and runs a smoke test for sharp.
The build pipeline uses SWC to transpile to dist-temp/src, then Rollup bundles it to dist/index.js (for the plugin) and dist/cli.js (for command-line use). Runtime assets (protos, TLS certs, helpers) are copied to dist/interfaces/** and dist/helpers/**.
CLI usage (optional):
node dist/cli.jsnpm start script; the plugin is normally run via LM Studio.The shared library draw-things-chat-core is temporarily not included in this repository.
Since revision 21, the core source files are bundled into a single src/core-bundle.mjs via esbuild.
This was necessary because the LM Studio Hub enforces a 128-file limit per plugin, and the plugin's own sources combined with the core files exceed that budget.
Our request to raise the limit (sent to team@lmstudio.ai on 15 January 2026) has not been answered yet.
To set up the core bundle for local development:
After the first setup, any change in core only requires npm run build in draw-things-chat-core — the build script automatically re-bundles and syncs core-bundle.mjs + fpzip files into this repo's src/.
The following generated files are gitignored and must be synced from core:
src/core-bundle.mjs — bundled core library (~300 KB)src/core-bundle.d.mts — TypeScript type stubTo run in development mode, which requires the LM Studio CLI:
LM Studio removes ~/.lmstudio/conversations/<Chat-ID>.conversation.json and attachments, but can leave
~/.lmstudio/working-directories/<Chat-ID> behind.
This repo includes a small helper to find such orphan folders and (optionally) move them to the macOS Trash.
If you run a local HTTP server that serves files from LM Studio chat working directories, the plugin can return HTTP URLs (instead of file:// URLs) in tool responses.
Configuration:
HTTP_SERVER_PORT (default: 54760)http://127.0.0.1:<port>/__healthz and only emits HTTP links when:
x-mcp-image-server equals 1.File Layout:
~/.lmstudio/working-directories/{chatId}/generated-image-*.png~/.lmstudio/working-directories/{chatId}/preview-*.jpgPreview Generation:
preview-generated-image-{timestamp}-v{N}.jpgpreview-attachment-image-{timestamp}.jpgAll user-configurable settings are defined in src/config.ts (single source of truth).
Tool output / links
PREVIEW_IN_CHAT (boolean; default: false; UI label: Simple Previews in Chat): When enabled, tool responses include inline preview images (simple mode).HTTP_SERVER_PORT (number; default: 54760): If a compatible local image server is healthy, the tool returns http://127.0.0.1:<port>/<chatId>/<file> URLs instead of file:// URLs.Draw Things connection
DRAW_THINGS_HOST (string; default: 127.0.0.1)DRAW_THINGS_HTTP_PORT (number; default: 7860)DRAW_THINGS_GRPC_PORT (number; default: 7859)Orchestrator (agent model)
model (select; default: qwen/qwen3.5-35b-a3b): The local vision-capable model used by the generator/orchestrator.baseUrl (string; default: http://127.0.0.1:1234/v1): LM Studio OpenAI-compatible server base URL.apiKey (string; default: empty): Only needed if your local server requires auth.Only Draw Things is supported.
On startup, the plugin probes both transports and auto-selects:
DRAW_THINGS_HOST (plugin setting)DRAW_THINGS_HTTP_PORT (plugin setting; default 7860)DRAW_THINGS_HOST (plugin setting)DRAW_THINGS_GRPC_PORT (plugin setting; default 7859)src/interfaces/tls/** (copied to dist/).generate_imageParameters (minimal UI):
prompt: stringmodel: 'auto'|'z-image'|'qwen-image'|'flux'|'custom' (selects model overlay with optimized parameters)imageFormat: 'square'|'landscape'|'portrait' (maps to width/height)mode='edit' enables multi-reference image editing using a canvas image plus optional moodboard references.
Important constraints:
source, sourceVariant, sourceAttachment are removed; use canvas and moodboard.| Parameter | Type | Description | Example |
|---|---|---|---|
mode | string | Must be "edit" | "edit" |
prompt | string | Editing instruction; keep it short and action-oriented | "add sunglasses" |
canvas | string | Primary reference image (required unless there is exactly one source available). Notation: a1, v2, p1 (a/v/p shorthand → *1). | "a1", "v2" |
moodboard | string[] | Additional reference images (style/context). Same notation as canvas. | ["a2","v1"] |
Reference limit:
The selected model enforces a maxReferenceImages limit (canvas + moodboard). For qwen-image this is currently 4 (and flux is also 4).
Note: qwen-edit is not a user-selectable model id. Edit capabilities/limits depend on the effective backend model used (e.g. qwen-image edit models).
Examples:
Model Overlays:
The model parameter accepts symbolic names that map to complete parameter sets optimized for specific models:
auto: Use default parameters (no overlay; defaults currently use qwen-image edit models for edit mode and z-image for most other modes)z-image: Maps to z_image_turbo_1.0_q8p.ckpt with optimized settingsqwen-image: Maps to qwen_image_2512_bf16_q8p.ckpt with optimized settingsModel overlays define all necessary parameters including:
.ckpt file)The overlay system works for both HTTP and gRPC backends. When a model is selected, the symbolic name is used only for overlay lookup—the actual model filename from the overlay is sent to the backend.
MODEL_IDS vs MODEL_FAMILIES (architecture):
The codebase distinguishes between two lists:
MODEL_IDS (internal): All model identifiers including sub-categories like qwen-edit. Used for overlay lookups and internal routing (e.g. selectAutoModel("image2image") → "qwen-edit").MODEL_FAMILIES (user-facing): Only the selectable families (, , , , ). Used for reports, snapshots, and UI model selection.Sub-categories (like qwen-edit) are resolved internally but not exposed as separate user-selectable families. For example, when the user selects qwen-image and the mode is image2image, the system internally routes to qwen-edit overlays.
_dt_i2i_profile (internal routing):
For image2image and edit modes, the core layer sets _dt_i2i_profile on the service input:
_dt_i2i_profile: "img2img" → use defaultParamsDrawThingsImg2Img and getEffectiveOverlay(modelId, "img2img")_dt_i2i_profile: "edit" → use defaultParamsDrawThingsEdit and getEffectiveOverlay(modelId, "edit")This ensures that image2image + moodboard uses img2img defaults (e.g. strength: 0.9) instead of edit defaults (e.g. strength: 1), even though both modes share the generateImageEdit() code path for multi-reference inputs.
The field is typed in ImageGenerationParams (see src/services/schemas.ts) and stripped from user input via stripInternalToolKeys().
Model used → family (display-only metadata):
This repo can build a dtc.model-mapping-snapshot.v1 payload from the currently loaded Draw Things overlays + custom configs.
That snapshot can be prefixed into a ceveyne/draw-things-index/index_image query so the index plugin can enrich search results:
model stays raw (machine-safe basename)model_display becomes a human-friendly display stringmodel_use_hints.model_use_by_mode provides VALID mode+model tool args for generate_imageConstraints:
Token economy note (snapshot prefix):
When the orchestrator injects a leading model-mapping snapshot JSON object into index_image.query for tool execution, it strips that prefix back out when serializing the tool call into the agent-model prompt history.
The stripping is signature-based (only JSON objects with schema: "dtc.model-mapping-snapshot.v1" are removed), and the remaining query text (the part authored by the model) is preserved verbatim.
Optional verification:
npx tsx scripts/model-mapping-report.tsscripts/model-mapping-report.md (supports --custom-configs and --out).PREVIEW_IN_CHAT Behavior:
PREVIEW_IN_CHAT=true: Tool returns preview images as Markdown in the response.PREVIEW_IN_CHAT=false (default): Tool returns only text/links (preview + original URLs and a compact JSON summary). The orchestrator can inject Markdown for generated images into the chat stream after the tool call completes.Vision Promotion (pixels to the agent model):
Independently of UI display, the generator/orchestrator can “promote” recent images into the same tool-continuation turn by injecting a synthetic role=user message containing image_url parts (placed directly after the tool result).
How it’s controlled (see src/config.ts):
visionPromotionPersistent=false (default): idempotent mode — promote only when there are new promotable items.visionPromotionPersistent=true: persistent mode — promote up to 5 attachments + 3 variants every turn.Tool-Result Image Harvesting (external tool images):
When a non-generate_image tool returns image URLs (for example from a web/image search tool), the orchestrator can “harvest” those images into the current chat working directory and inject a compact Markdown table with previews.
Source selection (canvas):
Canvas / moodboard notation:
a1 = attachment 1, v2 = variant 2, p1 = picture 1a// shorthand means //Returns:
PREVIEW_IN_CHAT=true: writes previews into the active chat folder and returns Markdown images.Variants:
variants (or aliases n / num_images) and validated centrally (1–3).batch_size and returns all images in one call.generated-image-<timestamp>-v1..vN.png and previewed as preview-generated-image-...-v1..vN.*.FPS Resolution (Video Models):
When the output is a video (e.g. LTX-2), the fps value for the gRPC payload is resolved in this exact priority order (highest → lowest):
| Priority | Source | Notes |
|---|---|---|
| 1 | Tool parameter — filtered.fps | Explicit fps sent in the generate_image call. If the LLM provides it, it always wins unconditionally. |
| 2 | Capability map — video.defaultFps in IMAGE_MODEL_CAPABILITY_MAP | Set per backend model file in src/capabilities.ts (e.g. ltx-2-distilled → 25, derived from the Draw Things Model Zoo framesPerSecondForModel). Applies when no tool-parameter value was given. |
| 3 | defaultParams.fps (via effective.fps) | From , currently . Applies when neither the tool parameter nor the capability map supplies a value. |
Custom Config / Model Overlay values for fps are blacklisted:
fps is stripped from overlayParamsNoSize alongside width, height, batch_count, batch_size, batchCount, batchSize, upscaler, upscaler_scale, and upscalerScale. Overlay sources (both Custom Configs and Model Overlays) therefore have no effect on fps resolution.
Summary of the resolution expression (all three gRPC call sites):
effective.fps reflects defaultParams.fps (= 24) because overlay fps is blacklisted and ...filtered (which may carry tool-param fps) is spread over defaultParams in the effective object — but that path is shadowed by the filtered.fps check in priority 1 anyway.
File Outputs:
~/.lmstudio/working-directories/{chatId}/generated-image-<timestamp>-vN.png~/.lmstudio/working-directories/{chatId}/preview-<basename>.jpg|webpreview_imagereview_image schedules a one-shot re-view/promotion of existing media items already present in the chat working directory state (chat_media_state.json).
Accepted targets (canonical):
aN = attachmentvN = generated variantiN = harvested tool-result imagepN = picture (e.g. Draw-Things index result)Strict validation (default):
strict=true by default.chat_media_state.json is rejected.Tolerated aliases (not advertised):
Some models tend to reference provenance fields (e.g. values taken from tool results like imagePaths or httpPreviewUrls) instead of using aN/vN/iN/pN.
We deliberately do not advertise these aliases in the tool schema/description, but we tolerate them for robustness:
The plugin may rewrite tool results only for the agent model input, to keep the model aligned with stable notations and to avoid accidental reliance on unstable paths/URLs.
ceveyne/draw-things-index/index_image)The original tool payload contains images[] entries with imagePaths[] and/or httpPreviewUrls[].
Each images[] entry may also include optional provenance metadata:
sourceInfo.type: one of generate_image_variant | attachment | saved_image | draw_things_projectsourceInfo.imageType (optional): producer-provided origin string (recent addition; used primarily for saved_image)When rendering the Draw-Things index markdown table, sourceInfo.imageType is surfaced as an additional metadata field:
**Origin:** <imageType>For the agent model, we add a strictly additive field to each images[] entry:
index: ["pN", ...]The array is aligned by position with the union of imagePaths/httpPreviewUrls (same length as the max of those arrays). Each entry is either:
review_image target (pN), derived from the stable picture index in chat_media_state.json, ornull when there is no path/preview at that position.This is intentionally additive: all original fields remain unchanged.
logs/generate-image-plugin.log, logs/error.log, logs/generate-image-plugin.audit.jsonlWhen using mode='image2image' or mode='edit', the plugin normalizes every input image before it is sent to the backend. This applies uniformly to all source pools:
a1, a2, ...)v1, v2, ...)The processing is identical across transports and modes; only the number of inputs differs (moodboard multi-reference is gRPC-only).
Normalization Rules (per input image):
Output sizing: The backend renders at a constrained requested_effective size. If requested_raw exceeds render limits, the upscaler is enabled and the final output is postprocessed back to exact requested_raw dimensions.
Processing Flow:
Configuration:
All img2img/txt2img limits are centralized in src/services/drawthingsLimits.ts:
Error Handling:
/tmp/debug-i2i-buffer-*.png (the main TypeScript plugin code does not)See src/core/tools.ts and src/helpers/imageUtils.ts for implementation details.
This plugin exports the LM Studio plugin settings into process.env internally for compatibility with existing service code.
Treat environment variables as internal wiring; user-facing configuration lives in src/config.ts and is set via LM Studio.
dist/interfaces/tls/server_crt.crt + root_ca.crt exist.dist/helpers/GRPCBin2PNG.js + dist/helpers/PNG2GRPCBin.js should exist (executed via ). Optional native binaries can be provided via /.MIT
variants | n | num_images: 1..3 (canonical requested number of variants; centrally validated)mode: 'text2image'|'image2image'|'edit'canvas: 'a1'|'v2' etc. (priority-1 reference for edit mode)fluxflux_2_klein_9b_q6p.ckptautoz-imageqwen-imagefluxcustomcanvasmoodboardpNp1mode='text2image': ignores canvas/moodboard.mode='image2image' or mode='edit':
canvas explicitly.canvas is omitted and there is exactly one source available in the chat state, it is auto-selected.canvas is omitted and there are multiple sources, the tool returns an error asking you to specify canvas.vpa1v1p1"2") is only accepted when unambiguous (exactly one pool populated)defaultParamsDrawThings{Txt2Img,Img2Img,Edit}.ts24| 4 | Hardcoded 24 | Literal last-resort fallback in the ?: 24 expression. Unreachable in practice as defaultParams.fps is always a number. |
chat_media_state.json only (no network fetches, no filesystem probing).sourceUrl, preview, filename → pNpreview, filename → iNpreview, filename, sourceUrl → vNoriginAbs, preview, filename → aNstrict=true).p1p2i1, i2, ...)width/height, inputs should not exceed that)drawthingsLimits.minDim)nodeDT_DECODER_CMDDT_ENCODER_CMDHTTP_SERVER_PORT controls probing and URL building.# Using npm (default)
npm install
npm run build
# 1. Clone and build draw-things-chat-core (sibling directory)
cd ../draw-things-chat-core
npm install
npm run build # tsc → bundle → sync-core (copies into draw-things-chat/src/)
# 2. Back in this repo, build the plugin
cd ../draw-things-chat
npm install
npm run build
npm run dev
# list orphans (dry-run)
node scripts/cleanup-lmstudio-orphan-working-directories.mjs
# dry-run + print directory tree of each orphan
node scripts/cleanup-lmstudio-orphan-working-directories.mjs --dry-run
# move to Trash (asks for confirmation)
node scripts/cleanup-lmstudio-orphan-working-directories.mjs --trash
// Basic edit: use attachment 1 as the canvas
{ mode: "edit", canvas: "a1", prompt: "add sunglasses" }
// Edit with style references
{ mode: "edit", canvas: "a1", moodboard: ["a2"], prompt: "in watercolor style" }
// Edit a generated variant with an attachment as a reference
{ mode: "edit", canvas: "v1", moodboard: ["a1"], prompt: "match the lighting" }
payloadFps =
(typeof filtered.fps === "number" ? filtered.fps : undefined) ??
getDefaultFpsForModel(payloadModel) ?? // capability map
(typeof effective.fps === "number" ? effective.fps : 24); // defaultParams fallback
Input source (any pool: aN / vN / pN)
→ Determine render target (requested_effective) from tool params or canvas size
→ Normalize each input image (cap → sum constraint → align → minDim → PNG)
→ Generate with backend
→ Postprocess to exact requested_raw size (with upscaler when required)
→ Return to user
export const drawthingsLimits = {
min: 256,
minDim: 256, // Minimum dimension per side
maxWidth: 1536,
maxHeight: 1536,
targetSum: 1344, // Sum constraint: width + height ≤ 1344px
align: 64, // Dimensions must be multiples of 64
upscaler: "4x_ultrasharp_f16.ckpt",
upscalerScaleFactor: 2,
};