Local Models
NimbleBrain can run its own agents on a model served from your machine or network, through the local provider. This page records what has been run end to end against NimbleBrain, what failed, and what has not been tried. Treat anything not listed as untested.
What “verified” means here
Section titled “What “verified” means here”A task is verified when the model completed it in a real NimbleBrain chat and the result was checked against what was stored: the file’s contents, or the saved task’s schedule, timezone, tools and run count. A reply that says it worked does not count. The tasks:
- Everyday: list the installed apps; create a file; create a file, read it back, list the workspace’s files, and answer from them.
- Scheduled tasks: create a task that runs every weekday at 8am, approve it, move it to 9am in a later message, and run it once.
Model servers
Section titled “Model servers”local speaks OpenAI’s Chat Completions API. A server’s Responses API is not used.
| Server | Status | Notes |
|---|---|---|
| Ollama 0.32.5 | Verified | Tool calling, streaming, and usage reporting. The server most results below come from. |
llama.cpp llama-server (build 11669, --jinja) |
Verified on Gemma 4 E4B | One multi-step task, correct. Tool calls work on both Chat Completions and Responses at the server; NimbleBrain uses Chat Completions. |
| LM Studio | Not tested | Serves Chat Completions. Its bug tracker has open Gemma 4 tool-calling issues (#1756, #1927). |
| vLLM | Not tested | Built for NVIDIA GPU servers rather than laptops. |
Any server through the openai provider with a baseURL |
Broken | That provider uses OpenAI’s Responses API. Ollama rejects the stored-item references sent on a tool loop’s second call (unknown input item type: "item_reference"), and llama.cpp’s streamed events fail the client’s schema check. Use local. |
Models
Section titled “Models”On Ollama, at a 32k context, unless noted.
| Model | Tool calling | Everyday tasks | Scheduled tasks | Resident memory at 32k |
|---|---|---|---|---|
Gemma 4 E4B (gemma4:e4b) |
Yes | Verified | Partial: of two runs on the current code, one created the task, moved it to 9am and ran it with no tool errors. The other, on an install made with npx nimblebrain against the v0.75.0 release candidate, created the task paused, then on each later turn called tasks__create again instead of tasks__update and asked a question, so the task was never moved or run. |
About 7 GB |
Qwen3 8B (qwen3:8b) |
Yes | Verified | Partial: it created the task, but on later turns it sometimes answered with nothing. Tested before the latest changes; not rerun since. | About 10 GB |
Qwen2.5 3B (qwen2.5:3b) |
Yes | Failed: empty replies, malformed arguments, and a claim of success after a refused call | Failed | Not measured |
| Gemma 3 (any size) | No, on Ollama | Not usable | Not usable | — |
| Gemma 4 26B / 31B, Qwen 3.6, Llama | — | Not tested | Not tested | 18 GB+ downloads |
For comparison, Claude Sonnet 4.6 passed every task above with no tool errors.
Known issues
Section titled “Known issues”- A context window smaller than the prompt fails silently. Ollama picks a default window from GPU memory: 4k below about 23 GiB, 32k up to about 47 GiB, 256k above that. A prompt longer than the window is cut without an error, and the model loses its tools. In testing, a 4k window turned a file request into “I do not have the capability to create files”, reported as a successful turn. Set the window explicitly (
OLLAMA_CONTEXT_LENGTH=32768, orPARAMETER num_ctx 32768in a Modelfile) and give the same number ascontextWindow. - Small models sometimes run dependent calls together. Gemma 4 E4B was seen to create a file and list the files in one step. The list can finish first and not show the new file.
- Small models can mistake one task for another. Asked not to run a task now, Gemma 4 E4B has created it paused. On later turns it has tried to create the same task again rather than change it. Check a task’s schedule and status in the tasks panel after setting it up on a small model.
- Small models may stop instead of retrying. When a call is refused (a wrong argument, or a tool held until its guide is read), a small model sometimes reports the problem or asks the person instead of correcting the call.
- Turns are slower than a hosted model. On the test machine, Gemma 4 E4B took about 10 to 15 seconds for a one-step turn and 20 to 60 seconds for a multi-step one; Claude Sonnet 4.6 took 5 to 15. The prompt (about 5k tokens on an ordinary turn) is processed in full on a conversation’s first call, and later calls reuse the server’s cache. Slower machines were not measured.
Setup and reachability
Section titled “Setup and reachability”- Verified:
npx nimblebrain --local-model gemma4:e4b(CLI 0.2.0) installed the v0.75.0 release candidate against Ollama on its default settings, prepared the model with a 32k window, and passed everydoctorcheck. The runtime offered only the local model. - Verified: Docker Desktop on macOS reaches a model server on the host at
host.docker.internal, including one listening only on127.0.0.1, as Ollama does by default.
Not yet verified
Section titled “Not yet verified”- Linux. The release’s
docker-compose.ymlmapshost.docker.internalto the host on Linux too, but a Linux host has not been tested. Ollama on Linux listens on127.0.0.1by default, which a container cannot reach; setOLLAMA_HOST=0.0.0.0. - Windows.
- Machines with less than 64 GB, apart from the 4k-window failure above.
- Long conversations. The tests ran up to four turns in a conversation.
Choosing a model
Section titled “Choosing a model”Use a model with tool calling (on Ollama, ollama show <model> lists tools under capabilities). Gemma 4 E4B is the smallest model tested end to end: verified on everyday tasks, partial on scheduled ones. Allow its resident memory plus about 2 GB for NimbleBrain’s containers and room for the operating system: 16 GB is tight, and 24 GB is a comfortable floor.