A homelab breaks in interesting ways. These are real incidents from mine, what caused them, and what actually fixed them, kept here so the same mistake does not get made twice.
Ollama Was Silently Running on CPU for 11+ Days
Local LLM inference was noticeably slow for over a week before anyone noticed. No errors, no alerts, just a model that felt sluggish and got shrugged off.
Root Cause
The NVIDIA driver on the GPU host had drifted to a version that no longer matched what the Ollama container expected. Ollama fell back to CPU inference without ever surfacing an error, so nothing in monitoring flagged it.
Fix
Pinned the driver version explicitly instead of letting it float on the latest release, and confirmed GPU utilization directly rather than trusting response times as a proxy for whether the GPU was actually in use. Saw roughly a 90x speedup once the fix landed.
docker02's Storage Driver Didn't Match docker01
Two Docker hosts that were supposed to be identical had quietly diverged: one running overlayfs, the other running vfs, a legacy driver with real performance and stability downsides.
Root Cause
The vfs driver had been the default on that host since it was first provisioned and nobody had audited it against the other hosts. It ran fine day to day, which is exactly why it went unnoticed.
Fix
Rebuilt the Docker storage layer on docker02 to use overlayfs, matching docker01. A good reminder that hosts described as identical need periodic verification, not just an assumption.
apt upgrade Was Silently Detaching LXC Containers From the Network
Routine Proxmox package updates kept knocking LXC containers off the network. Services stayed up internally, but nothing outside the container could be reached.
Root Cause
Proxmox's apt upgrades restart the host networking service, which detaches LXC veth interfaces from the vmbr0 bridge. Nothing in the upgrade process reattaches them automatically.
Fix
Deployed a permanent if-up.d hook that reattaches veth interfaces whenever the bridge comes back up, so this is no longer something to remember to check after every upgrade.
Tower's br0 interface had been configured with a /16 subnet mask instead of /24 for an unknown amount of time, quietly broadening its view of the network far past what it needed.
Root Cause
Likely a leftover from early setup that was never revisited once the network settled into its real /24 layout.
Fix
Corrected the mask to /24. Flagged for a follow-up check after the next reboot, since interface config like this doesn't always announce itself as wrong until something depends on the boundary being correct.
pihole2 Was Dying at Midnight, Every Night
One of two Pi-hole instances was crashing right around midnight on a predictable schedule, which made it look intentional before it was diagnosed.
Root Cause
logrotate was sending FTL a SIGHUP as part of its nightly rotation, and FTL wasn't configured to survive that signal gracefully.
Fix
Set Restart=always on the FTL service so a SIGHUP-triggered exit just comes back up on its own instead of staying down until someone notices.
19 Ports Open to the Whole LAN, Zero Firewall Rules
An audit across the Docker hosts and Tower turned up 19 published container ports reachable from anywhere on the LAN, with no host firewall in front of any of them.
Root Cause
Ports get published as services get added, and without a standing firewall policy, exposure just accumulates silently in the background.
Fix
Locked down all four hosts. Tower needed a User Scripts 'Array Started' trigger rather than the usual /boot/config/go, since Unraid's boot sequence doesn't run go early enough for this to stick reliably.
Ansible Was Regenerating Zabbix PSK Keys on Every Run
Zabbix Agent PSK encryption worked fine right after setup, then started silently dropping back to unencrypted connections after Ansible playbook runs, with no errors anywhere.
Root Cause
Ansible's password lookup plugin was regenerating a new PSK value every single run instead of reusing the existing one, which meant the agent and server disagreed on the key after each deploy.
Fix
Tracked it down to one of Ansible's more confusing lookup plugin behaviors and fixed the playbook to persist and reuse the key instead of regenerating it.