GGUF
•
granite
A small and capable LLM from IBM
Model info
Model
granite 3.2 8b
Author
ibm
Arch
granite
Parameters
8B
Format
gguf
Size on disk
about 4.94 GB
Download the model using lms
— LM Studio's developer CLI.
lms get granite-3.2-8b
curl http://localhost:1234/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "granite-3.2-8b",
"messages": [
{ "role": "system", "content": "Always answer in rhymes." },
{ "role": "user", "content": "Introduce yourself." }
],
"temperature": 0.7,
"max_tokens": -1,
"stream": true
}'
lms log stream
to see your prompts as they are sent to the LLM.lmstudio.js
- LM Studio SDK documentation (TypeScript)lms log stream
- Stream server logslms
- LM Studio's CLI documentation