LM Studio on Windows - a graphical way to run local LLMs

1. Download

Go to lmstudio.ai, download the Windows installer, run it, and follow the wizard.

2. Discover models

Open the "Discover" tab and search for `qwen2.5-7b-instruct` (a good default) or `llama-3.1-8b-instruct`. Choose a Q4_K_M GGUF quantization to balance speed and quality.

3. Chat

Open the "Chat" tab, load the downloaded model, and start chatting. Adjust `Context Length` based on your VRAM.

4. Serve a local API

Open "Developer" -> Start Server. The endpoint runs at http://localhost:1234/v1 and is OpenAI-compatible.

curl http://localhost:1234/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"local-model","messages":[{"role":"user","content":"Hi"}]}'

Choosing a quantization

• •Q4_K_M - best balance (recommended)
• •Q5_K_M - more quality, about 15% slower
• •Q8_0 - near full precision, only if you have spare VRAM
• •Q2/Q3 - only for testing; quality drops a lot