App

Developer Docs

lmstudio-js

lmstudio-python

CLI

Introduction

Project Setup

Basics

Chat

Working with Chats

Cancelling Predictions

Image Input

Structured Response

Speculative Decoding

Generate Completions

Configuration Parameters

Agentic Flows

The .act() call

Tool Definition

Plugins (Beta)

Introduction to Plugins

Using npm Dependencies

Tools Provider

Prompt Preprocessor

Introduction

Examples

Custom Configuration

Custom Status Report

Handling Aborts

Generators

Custom Configuration

Publishing a Plugin

Text Embedding

Generating embedding vectors

Tokenization

Tokenizing text

Manage Models

List Local Models

List Loaded Models

Load and Access Models

API Reference

LLMLoadModelConfig

LLMPredictionConfigInput

Model Info

Get Context Length

Get Model Info

Feature In Beta

Plugins SDK is currently in private beta. Join the beta here.

Plugins (Beta)

Handling Aborts

A prediction may be aborted by the user while your generator is still running. In such cases, you should handle the abort gracefully by handling the ctl.abortSignal.

You can learn more about AbortSignal in the MDN documentation.

This page's source is available on GitHub