By the end of this you will have a web app at https://ai.yourdomain.com that answers questions, reads
your documents and keeps every prompt on hardware you control. No API keys, no per-token billing, no
data leaving the box.
1. Pick the right box
You need 8 GB RAM minimum for a 7B model at 4-bit quantisation with headroom for the web UI. If you can afford 16 GB, take it — larger models and longer context both live in memory.
Create the VPS with Ubuntu 24.04 and add your SSH key. Then, from your machine:
ssh root@YOUR_SERVER_IP
apt update && apt upgrade -y
2. Point a DNS record at it
In your DNS provider, create an A record:
| Type | Name | Value | TTL |
|---|---|---|---|
| A | ai |
YOUR_SERVER_IP |
300 |
Wait until dig +short ai.yourdomain.com returns the IP. Caddy cannot issue a certificate before that
resolves, and this is the single most common reason the last step fails.
3. Install Docker
apt install -y ca-certificates curl gnupg
install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg
chmod a+r /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" \
> /etc/apt/sources.list.d/docker.list
apt update && apt install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin
4. Write the compose file
Create /opt/ai/docker-compose.yml:
services:
ollama:
image: ollama/ollama:latest
container_name: ollama
restart: unless-stopped
volumes:
- ollama:/root/.ollama
ports:
- '127.0.0.1:11434:11434'
environment:
- OLLAMA_KEEP_ALIVE=30m
openwebui:
image: ghcr.io/open-webui/open-webui:main
container_name: openwebui
restart: unless-stopped
depends_on:
- ollama
ports:
- '127.0.0.1:8080:8080'
environment:
- OLLAMA_BASE_URL=http://ollama:11434
- WEBUI_AUTH=true
- ENABLE_SIGNUP=false
volumes:
- webui:/app/backend/data
caddy:
image: caddy:2-alpine
container_name: caddy
restart: unless-stopped
ports:
- '80:80'
- '443:443'
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile:ro
- caddy_data:/data
- caddy_config:/config
volumes:
ollama:
webui:
caddy_data:
caddy_config:
Two details worth noticing: both services bind to 127.0.0.1, so nothing is exposed until Caddy
proxies it, and ENABLE_SIGNUP=false means nobody can create an account behind your back.
5. Add HTTPS with Caddy
Create /opt/ai/Caddyfile:
ai.yourdomain.com {
encode zstd gzip
reverse_proxy openwebui:8080
}
Caddy gets a certificate automatically on first start. That is the whole TLS configuration.
cd /opt/ai && docker compose up -d
docker compose ps
If caddy is restarting, DNS is not ready yet. Fix DNS, then docker compose restart caddy.
6. Pull a model
docker exec -it ollama ollama pull qwen2.5:7b-instruct-q4_K_M
docker exec -it ollama ollama run qwen2.5:7b-instruct-q4_K_M "In one sentence, what is an NPU?"
Expect 5–12 tokens per second on 4 vCPU. If it feels glacial, try a 3B model at q4_K_M — quality drops
but latency improves a lot.
7. Create your account, then close the door
- Visit
https://ai.yourdomain.com, sign up as the first user (this becomes the admin). - In Admin settings → Users, confirm signup is disabled.
- Set a strong password and enable 2FA if your fork supports it.
8. Harden the box
# firewall: only SSH + web
apt install -y ufw
ufw allow OpenSSH
ufw allow 80/tcp
ufw allow 443/tcp
ufw --force enable
# unattended security updates
apt install -y unattended-upgrades && dpkg-reconfigure -plow unattended-upgrades
# fail2ban for SSH
apt install -y fail2ban && systemctl enable --now fail2ban
Backups: the ollama and webui volumes hold everything. Snapshot the VPS weekly, or:
docker run --rm -v ai_ollama:/data -v /root:/backup alpine \
tar czf /backup/ollama-$(date +%F).tar.gz -C /data .
Troubleshooting and common mistakes
“502 Bad Gateway” from Caddy. The web UI takes 60–90 seconds on first boot. Check
docker compose logs -f openwebui before changing anything.
Certificate errors. dig +short ai.yourdomain.com must return your server IP. If you are behind
Cloudflare’s proxy, either set the record to DNS-only or use a Cloudflare origin certificate.
The model keeps reloading. You are out of RAM. OLLAMA_KEEP_ALIVE only helps if the model fits.
Check with docker stats, then move to a smaller quantisation.
No GPU. Everything here is CPU inference. It works; it is just slower. A used 16 GB GPU in a desktop is a better long-term home for this stack than a VPS.
You exposed port 11434. If Ollama is reachable from the internet, anyone can use your box. Keep the
127.0.0.1: prefix in the compose file, always.



