Run LLMs locally on macOS with Ollama

Why Ollama

Ollama exposes a local OpenAI-compatible API. Point any client (Cursor, Continue, your app) to http://localhost:11434 and use the model for free, offline, and privately.

1. Install

Download the official installer from ollama.com/download or use Homebrew:

brew install ollama
ollama serve &

2. Pick a model for your RAM

• •8 GB -> llama3.2:3b or qwen2.5:3b (fast, basic)
• •16 GB -> llama3.1:8b, qwen2.5:7b, mistral:7b (recommended default)
• •32 GB -> qwen2.5:14b, gemma2:27b-q4
• •64 GB+ -> llama3.3:70b-q4_K_M, qwen2.5:32b

3. Download and run

ollama pull llama3.1:8b
ollama run llama3.1:8b

4. Call it through an OpenAI-compatible API

import OpenAI from "openai";
const client = new OpenAI({
  baseURL: "http://localhost:11434/v1",
  apiKey: "ollama",
});
const res = await client.chat.completions.create({
  model: "llama3.1:8b",
  messages: [{ role: "user", content: "Hello!" }],
});

5. Connect Cursor / Continue / Zed

In Cursor: Settings -> Models -> Add custom OpenAI model, URL http://localhost:11434/v1, model llama3.1:8b. In Continue (VS Code), add an `openai` provider with the same baseURL to ~/.continue/config.json.

Troubleshooting

• •Slow model -> try a smaller quantization (:q4_K_M)
• •"connection refused" -> confirm `ollama serve` is running
• •Out of RAM -> use a smaller model or close heavy apps