Local AI Dashboard
Everything you'd otherwise track across many browser tabs and a couple of terminals: GPU and host telemetry, Docker stack control, live logs, and what the model library is costing in disk and electricity. Collapsed into one dark, single-page dashboard behind one port, with no external database and no cloud dependency.
Problem & approach
A 16GB GPU meant only one model could be loaded at a time to get the most out of the available hardware, so switching workloads meant juggling Portainer stacks by hand: stopping one, starting another, then checking nvtop to see whether the previous job had actually finished. On top of that, keeping track of homelab and in-development applications, remembering port numbers, staying on top of AI news, and watching local model bloat added up to constant background overhead.
A single containerised dashboard behind one port: live GPU and host telemetry communicated at a glance through a dynamic favicon, green, orange, or red depending on GPU utilisation. Start, stop, and log-tail control over existing Portainer stacks without opening Portainer, making it easy to switch workloads from a phone. A model library that fingerprints both the on-disk model directories and Ollama's blob store in one pass to surface duplicate and no-longer-needed models. A SQLite-backed usage history with real electricity cost, all driven by a single YAML config file that hot-reloads with no restart.
Challenges
Problem
Naive duplicate detection by filename missed the real problem, the same weights re-appearing as a GGUF on disk and separately imported into Ollama as content-identical multi-shard blobs, which du and ollama list each report as unrelated, unrecovered disk usage.
Solution
Built a content-fingerprint pass across both the raw model directories and Ollama's blob store, hashing the first and last 1 MiB of same-size files, since a full SHA-256 over dozens of multi-gigabyte GGUFs was too slow to run on every scan, and merging Ollama tags with on-disk files into one duplicate-detection pass, which is what surfaced roughly 100 GiB of duplicated weights on the machine it was built on.
Problem
nvtop's GPU utilisation and VRAM totals rendered fine inside the ttyd container, but the per-process breakdown, the part that actually tells you what's eating VRAM, came back blank.
Solution
Docker's default PID namespace isolation was blocking access to host process info. Fixed by adding pid: host and cap_add: SYS_PTRACE to the ttyd service specifically, scoped and documented as an acceptable trade-off for a private LAN host rather than something to route around.
Problem
Ollama's keep_alive: 0 unload doesn't free VRAM immediately: it waits for the current request to finish, so a fire-and-forget unload button reported success while the model was still resident.
Solution
Turned unload into an explicit state machine, idle to unloading to confirmed-or-timeout, that polls live Ollama status until VRAM is actually freed, instead of trusting the fetch() response as confirmation.
Stack
- Node
- Svelte 4 + Vite
- sqlite (built-in, no external DB)
- Docker Compose + Portainer API proxy
- ttyd (nvtop terminal, proxied same-origin)
Demonstrates
- Docker/Portainer stack lifecycle control via server-side proxied API
- Content-fingerprint deduplication across a filesystem and a separate blob store
- Hot-reloaded YAML configuration via fs.watch, no restart required
- GPU usage accounting on local SQLite, no external DB