•
phi3
Microsoft's latest Phi Mini model supports a whopping context length of 128k tokens in a small size, offering extremely long chats for cheap.
Model info
Model
Phi 3.1 Mini 128k
Author
Microsoft
Arch
phi3
Parameters
3.8B
Format
gguf
Size on disk
about 2.39 GB
Download the model using lms
— LM Studio's developer CLI.
lms get phi-3.1-mini-128k
curl http://localhost:1234/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "phi-3.1-mini-128k",
"messages": [
{ "role": "system", "content": "Always answer in rhymes." },
{ "role": "user", "content": "Introduce yourself." }
],
"temperature": 0.7,
"max_tokens": -1,
"stream": true
}'
lms log stream
to see your prompts as they are sent to the LLM.lmstudio.js
- LM Studio SDK documentation (TypeScript)lms log stream
- Stream server logslms
- LM Studio's CLI documentation