Local Models

Lattis connects to local model servers you run, rather than hosting models itself. Any OpenAI-compatible server works; the models it serves appear in GET /v1/models next to your cloud models and route the same way.

Supported servers

Adding a local provider starts from a picker of common platforms. The choice just pre-fills the server’s documented default address and labels the account — under the hood every platform is treated identically:

Platform	Default base URL	Note
LM Studio	http://localhost:1234/v1	Start the server from its Developer tab.
Ollama	http://localhost:11434/v1	ollama serve — running by default.
llama.cpp	http://localhost:8080/v1	llama-server from llama.cpp.
vLLM	http://localhost:8000/v1	High-throughput GPU serving. API key optional.
Unsloth	http://localhost:8000/v1	Served via llama.cpp or vLLM.
Custom	http://localhost:8000/v1	Any OpenAI-compatible /v1 endpoint.

The base URL is fully editable, so non-standard ports and servers on other machines work too.

Add a server

In Settings, add a local provider, pick the platform, and confirm the base URL. Lattis probes the server’s /v1/models before saving — a URL that doesn’t answer is rejected up front, so you know immediately whether the connection works.

Each server is a separate account with its own base URL and its own model catalogue, fetched live from the server. You can connect several at once (say, LM Studio and Ollama side by side) and their models are merged into one list.

Use the models

Local models show up in GET /v1/models alongside connected cloud models. Use one by passing its id as the model field on any request — see the Public API. Routing, format translation, and usage tracking work exactly as they do for cloud providers; requests served by your own hardware are costed at $0.

If a server goes down after it’s connected, its models simply drop out of the list until it’s back — reconnecting isn’t necessary.