By the end of this you will have a web app at https://ai.yourdomain.com that answers questions, reads your documents and keeps every prompt on hardware you control. No API keys, no per-token billing, no data leaving the box.

1. Pick the right box

You need 8 GB RAM minimum for a 7B model at 4-bit quantisation with headroom for the web UI. If you can afford 16 GB, take it — larger models and longer context both live in memory.

Create the VPS with Ubuntu 24.04 and add your SSH key. Then, from your machine:

ssh root@YOUR_SERVER_IP
apt update && apt upgrade -y

2. Point a DNS record at it

In your DNS provider, create an A record:

Type Name Value TTL
A ai YOUR_SERVER_IP 300

Wait until dig +short ai.yourdomain.com returns the IP. Caddy cannot issue a certificate before that resolves, and this is the single most common reason the last step fails.

3. Install Docker

apt install -y ca-certificates curl gnupg
install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg
chmod a+r /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
  https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" \
  > /etc/apt/sources.list.d/docker.list
apt update && apt install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin

4. Write the compose file

Create /opt/ai/docker-compose.yml:

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    restart: unless-stopped
    volumes:
      - ollama:/root/.ollama
    ports:
      - '127.0.0.1:11434:11434'
    environment:
      - OLLAMA_KEEP_ALIVE=30m

  openwebui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: openwebui
    restart: unless-stopped
    depends_on:
      - ollama
    ports:
      - '127.0.0.1:8080:8080'
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
      - WEBUI_AUTH=true
      - ENABLE_SIGNUP=false
    volumes:
      - webui:/app/backend/data

  caddy:
    image: caddy:2-alpine
    container_name: caddy
    restart: unless-stopped
    ports:
      - '80:80'
      - '443:443'
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile:ro
      - caddy_data:/data
      - caddy_config:/config

volumes:
  ollama:
  webui:
  caddy_data:
  caddy_config:

Two details worth noticing: both services bind to 127.0.0.1, so nothing is exposed until Caddy proxies it, and ENABLE_SIGNUP=false means nobody can create an account behind your back.

5. Add HTTPS with Caddy

Create /opt/ai/Caddyfile:

ai.yourdomain.com {
    encode zstd gzip
    reverse_proxy openwebui:8080
}

Caddy gets a certificate automatically on first start. That is the whole TLS configuration.

cd /opt/ai && docker compose up -d
docker compose ps

If caddy is restarting, DNS is not ready yet. Fix DNS, then docker compose restart caddy.

6. Pull a model

docker exec -it ollama ollama pull qwen2.5:7b-instruct-q4_K_M
docker exec -it ollama ollama run qwen2.5:7b-instruct-q4_K_M "In one sentence, what is an NPU?"

Expect 5–12 tokens per second on 4 vCPU. If it feels glacial, try a 3B model at q4_K_M — quality drops but latency improves a lot.

7. Create your account, then close the door

  1. Visit https://ai.yourdomain.com, sign up as the first user (this becomes the admin).
  2. In Admin settings → Users, confirm signup is disabled.
  3. Set a strong password and enable 2FA if your fork supports it.

8. Harden the box

# firewall: only SSH + web
apt install -y ufw
ufw allow OpenSSH
ufw allow 80/tcp
ufw allow 443/tcp
ufw --force enable

# unattended security updates
apt install -y unattended-upgrades && dpkg-reconfigure -plow unattended-upgrades

# fail2ban for SSH
apt install -y fail2ban && systemctl enable --now fail2ban

Backups: the ollama and webui volumes hold everything. Snapshot the VPS weekly, or:

docker run --rm -v ai_ollama:/data -v /root:/backup alpine \
  tar czf /backup/ollama-$(date +%F).tar.gz -C /data .

Troubleshooting and common mistakes

“502 Bad Gateway” from Caddy. The web UI takes 60–90 seconds on first boot. Check docker compose logs -f openwebui before changing anything.

Certificate errors. dig +short ai.yourdomain.com must return your server IP. If you are behind Cloudflare’s proxy, either set the record to DNS-only or use a Cloudflare origin certificate.

The model keeps reloading. You are out of RAM. OLLAMA_KEEP_ALIVE only helps if the model fits. Check with docker stats, then move to a smaller quantisation.

No GPU. Everything here is CPU inference. It works; it is just slower. A used 16 GB GPU in a desktop is a better long-term home for this stack than a VPS.

You exposed port 11434. If Ollama is reachable from the internet, anyone can use your box. Keep the 127.0.0.1: prefix in the compose file, always.