Skip to main content

Local GPU Stack

This guide describes the repository-supported local GPU setup: Ollama runs natively on the host for direct GPU access, and Libre WebUI runs in Docker with docker-compose.dev.external-ollama.yml.

What You Get​

  • Libre WebUI using the librewebui/libre-webui:dev image.
  • Native host Ollama for local model inference.
  • Persistent Libre WebUI data volumes.
  • A clean path for testing the dev image without containerizing Ollama.

Prerequisites​

ComponentRequirement
DockerDocker Desktop or Docker Engine with Compose v2
OllamaInstalled and running on the host
GPUOptional but recommended for local models
NVIDIACurrent driver and NVIDIA Container Toolkit if you containerize GPU workloads
DiskEnough space for Ollama model files

Install and Verify Ollama​

Linux:

curl -fsSL https://ollama.com/install.sh | sh

macOS:

brew install ollama
ollama serve

Pull a starter model:

ollama pull gemma3:4b

Verify the API:

curl http://localhost:11434/api/version

Start Libre WebUI​

From the repository root:

docker compose -f docker-compose.dev.external-ollama.yml up -d

Open http://localhost:8080.

Custom Ollama Endpoint​

If Ollama is on another machine:

OLLAMA_BASE_URL=http://192.168.1.100:11434 docker compose -f docker-compose.dev.external-ollama.yml up -d

For Tailscale:

OLLAMA_BASE_URL=http://100.x.y.z:11434 docker compose -f docker-compose.dev.external-ollama.yml up -d

For a first GPU workstation:

ollama pull gemma3:4b
ollama pull qwen3:8b
ollama pull deepseek-r1:8b
ollama pull nomic-embed-text

Use the Model Manager to discover larger models once the basic stack is working.

LAN Access​

Add your LAN or Tailscale origin to CORS_ORIGIN in the compose environment before exposing the app to another device:

CORS_ORIGIN=http://localhost:8080,http://192.168.1.50:8080

Then restart:

docker compose -f docker-compose.dev.external-ollama.yml up -d

Useful Commands​

docker compose -f docker-compose.dev.external-ollama.yml ps
docker compose -f docker-compose.dev.external-ollama.yml logs -f libre-webui
docker compose -f docker-compose.dev.external-ollama.yml pull
docker compose -f docker-compose.dev.external-ollama.yml up -d

Troubleshooting​

Container cannot reach Ollama

docker compose -f docker-compose.dev.external-ollama.yml exec libre-webui \
  wget -O- http://host.docker.internal:11434/api/version

On Linux, add:

extra_hosts:
  - 'host.docker.internal:host-gateway'

Model runs on CPU

Native Ollama uses the host installation. Check your Ollama install, GPU driver, and ollama ps.

Need TTS

Use the Qwen3-TTS or Kyutai TTS server guides, then enable the matching plugin in Settings.