Home/Local AI setup

Local AI setup

Run Roastline's roast assistant entirely on your own machine with LM Studio or Ollama — private, offline, and no account required.

Runs on
Your machinedesktop app · no sign-in
Backends
LM Studio / Ollamaany OpenAI-compatible /v1
LM Studio port
localhost:1234/v1 endpoint
Ollama port
localhost:11434/v1 endpoint
SETUP GUIDE · Local AI · ~10 min

Local models work in the desktop app without sign-in. Cloud models (Claude, ChatGPT, Azure Foundry) stay available in the same dropdown when you're signed in.

How it fits together

Roastline talks to a small local server that you run — either LM Studio or Ollama. Both expose an OpenAI-compatible /v1 API. The app's daemon forwards your chat to that server and lists the models it has available.

Roastline desktop app
  └─ AI panel / Preferences
      └─ Roastline daemon (:19234)
          └─ Your local server
             ├─ LM Studio  http://localhost:1234/v1
             └─ Ollama      http://localhost:11434/v1

Your choice of preset (LM Studio, Ollama, or Custom URL) is saved in the app and sent with each request.

Quick start (desktop app)

  1. Install and start LM Studio or Ollama (sections below).
  2. Open Roastline → Preferences (gear) → AI.
  3. Under Local AI server, pick LM Studio or Ollama.
  4. Confirm Active: shows the expected URL (:1234 or :11434).
  5. Click Refresh on the model list (or reopen the AI panel).
  6. Choose a local model in Default model.
  7. Open the AI side panel and chat — no account required.

The status indicator tells you where you stand:

StatusMeaning
Connected + model countServer reachable; models listed
OfflineServer not running, or wrong port
Checking…The daemon is probing your endpoint

LM Studio

Install

  1. Download from lmstudio.ai for macOS, Windows, or Linux.
  2. Install and open LM Studio.

Load a model

  1. Open the Models tab (or search Hugging Face from inside LM Studio).
  2. Download a chat-capable model (see Model picks).
  3. Load it in the chat UI so it's ready to serve.

Start the API server

  1. Go to the Developer tab (or Local Server, depending on version).
  2. Click Start Server. The default address is http://localhost:1234.
  3. Enable CORS if LM Studio shows the toggle (usually optional — Roastline proxies through its daemon).

Roastline expects the OpenAI path /v1, so the preset URL is http://localhost:1234/v1.

Verify (terminal)

$ curl -s http://localhost:1234/v1/models | head -c 400

You should see JSON with a data array of model ids.

In Roastline: Preferences → AI → Local AI server → LM Studio. The Active URL should read http://localhost:1234/v1.

Ollama

Install

  1. Download from ollama.com and install.
  2. On macOS/Linux the menu-bar app usually runs ollama serve automatically. On Windows, open the Ollama app after installing.

Pull a model

$ ollama pull llama3.2
# or: ollama pull qwen2.5:7b
# or: ollama pull mistral

List what you've installed:

$ ollama list

API endpoint

Ollama's OpenAI-compatible API lives at http://localhost:11434/v1. If the service isn't running, start it:

$ ollama serve

Verify (terminal)

$ curl -s http://localhost:11434/v1/models | head -c 400

In Roastline: Preferences → AI → Local AI server → Ollama. The Active URL should read http://localhost:11434/v1.

Model picks

Roastline sends a structured roast report (curve, milestones, event log) and asks for phase analysis and actionable advice. Any solid instruction-tuned chat model works; larger models tend to give richer roast commentary.

We recommend openai/gpt-oss-20b — it's the best balance of quality, size, and speed for roast commentary. It's available in both LM Studio and Ollama. Other models are perfectly fine to test, too.

BackendExamples to try
LM Studioopenai/gpt-oss-20b (recommended), Qwen2.5-7B-Instruct, Llama 3.2 3B/8B Instruct, Mistral 7B Instruct, Phi-3
Ollamagpt-oss:20b (recommended), qwen2.5:7b, llama3.2, mistral, phi3
  • Start with gpt-oss-20b, then experiment — swap models any time from the dropdown.
  • Prefer instruct / chat variants over base models.
  • 7B–8B is a good balance on a typical laptop; use smaller quantizations (Q4/Q5) if RAM or VRAM is tight.
  • The model must appear in /v1/models and be loaded/ready on the server before chat works.

Custom URL (advanced)

Running vLLM, a llama.cpp server, or a model on another machine? Choose Preferences → AI → Custom URL and enter any OpenAI-compatible base URL. Roastline normalizes it to end with /v1.

ServerExample base URL
llama.cpp serverhttp://localhost:8080/v1
vLLMhttp://192.168.1.50:8000/v1
Remote LM Studiohttp://192.168.1.50:1234/v1

Click Apply custom URL, then refresh the model list.

Using local AI during a roast

  1. Connect a roaster or use the simulator.
  2. Start a session and run a roast (or load history).
  3. Open the AI panel.
  4. Select your local model (green provider badge).
  5. Ask for analysis, or enable auto-commentary in Preferences if available.

The assistant receives the current roast context (profile, events, decimated curve), so follow-up questions stay grounded in that report.

Troubleshooting

SymptomWhat to check
Local AI: OfflineIs the LM Studio server started? Is ollama serve running? Correct preset in Preferences?
No models in dropdownRun curl …/v1/models. In LM Studio, load a model and start the server. In Ollama, run ollama pull ….
Chat error: unreachableFirewall blocking localhost? Wrong port (1234 vs 11434)? Custom URL missing /v1?
Chat error: 4xx from serverModel id is stale — refresh the list and re-select. Model not loaded in LM Studio.
Works in curl, not in appConfirm the Active URL in Preferences. Toggle the preset (Ollama ↔ LM Studio) and back. Restart Roastline.
Web simulator has no local AIExpected — use the desktop app for LM Studio / Ollama.

Privacy & accounts

  • Local models — inference stays on your machine; no Roastline cloud account required on desktop.
  • Cloud models — require sign-in and route through configured providers (Anthropic, OpenAI, Azure Foundry).

You can use local AI alongside cloud models and switch between them in the model dropdown at any time.

Need a hand getting local AI running?

Send us your setup and the status indicator — we'll help you connect.

Contact supportAll supported devices