Setting Up Prometheus and Grafana for Home Lab Monitoring

Self-hosted observability used to feel like a paid product in disguise. Pull-based metrics, exporter quirks, and alertmanager YAML scared off hobbyists who just wanted to know whether their NAS was still alive. Prometheus and Grafana are remarkably approachable once you separate the moving parts, and this walkthrough focuses on practical decisions for a small setup: a handful of servers, a NAS, maybe a couple of Raspberry Pi nodes humming in a cupboard, all running quietly behind an Aussie broadband link.

I built my first home metrics stack while renting a unit in Brisbane, with a second-hand Dell Optiplex next to the laundry and an aging router plugged into an NBN HFC connection. The point was never enterprise polish — it was to catch dying disks before they failed and to stop guessing whether a slow API was the application or the network. The same approach scales from a single node to a small fleet without changing the philosophy, and is the playbook I now reach for, including for clients through my consulting resume.

Planning your observability stack

Before installing anything, draw a small diagram on paper. List every machine you actually want to monitor: the hypervisor, the NAS, a Raspberry Pi running Pi-hole, the family router if it exposes metrics, maybe a small Kubernetes cluster for fun. Each becomes a scrape target in Prometheus, and naming them consistently now will save hours later when you build Grafana dashboards.

Think about retention early. Prometheus stores time-series data locally by default, and on a spinning disk the default fifteen-day retention can quietly eat tens of gigabytes. Two to four weeks is usually enough for a home lab; if you want more history, consider remote-write to a cheap object store. Several Australian providers with points of presence in Sydney and Melbourne offer low-cost storage tiers suitable for archival metrics.

Finally, decide what success looks like. If your goal is simply "know when things are down", a tiny configuration with two exporters and a single alerting rule is enough. If you want to chart UPS battery voltage during an AGL outage in Adelaide or track CPU temperatures against a Melbourne heatwave, you will want more exporters and a longer retention window. Match the scope to the question you want answered.

Setting up exporters and the Prometheus server

The Prometheus model is pull-based, so every machine you want to monitor needs to expose an HTTP endpoint that Prometheus can scrape. The de-facto standard for Linux hosts is node_exporter, which exposes CPU, memory, disk, network, and filesystem metrics. On Debian or Ubuntu, a single deb package from the Prometheus downloads page is enough; on RHEL-derived systems used by many Australian enterprise environments, the same binary works fine as a systemd unit.

Networking matters more than people expect. Routers supplied by Telstra, Optus, or Aussie Broadband often isolate guest networks and IoT VLANs, so a Prometheus server on your main LAN may not reach every node. Pin Prometheus to a VLAN that can see everything, or use the push gateway for devices behind double NAT. Static DHCP reservations in your router prevent the dreaded "scrape failed, target disappeared" problem when a device renews its lease.

Prometheus itself is a single static binary written in Go, which makes it wonderfully portable. Download the release tarball, drop it under /opt/prometheus, and create a dedicated system user. Put the data directory on a dedicated SSD or a small NVMe in a spare M.2 slot — local suppliers like Scorptec and MSY in Brisbane stock affordable drives, and a 256 GB NVMe is more than enough for several months of metrics from a modest home fleet.

Run Prometheus behind a reverse proxy with basic authentication if you want to access the UI from outside your home network — the built-in web server is fine for localhost but does not advertise itself as production-grade. Caddy or nginx with a small Let's Encrypt certificate handles this cleanly, even on residential NBN FTTP plans without a static IP. Pair it with dynamic DNS and you have a remote dashboard you can check from a cafe in Perth or a coworking space in Hobart.

Building dashboards that actually help

Grafana is where the magic becomes visible. Install it from the official APT or YUM repository — Grafana Labs maintains packages for most major distributions used in Australia, including Ubuntu LTS releases. Point its default datasource at your Prometheus instance, then build dashboards panel by panel. Resist the urge to copy community dashboards wholesale; they are usually bloated with panels you will never read.

Useful panels include CPU load per host, memory pressure, disk SMART temperature, and network throughput. SMART data is gold, because catching a drive creeping toward failure beats replacing it after the array has degraded. The smartctl_exporter project makes this trivial, and many ASRock and ASUS motherboards expose IPMI metrics that work with the ipmi_exporter.

Pulling services like Plex, Home Assistant, or Unifi Network into a single Grafana dashboard gives you a pane of glass no commercial router UI provides. With Australian energy prices being what they are, you might also add a panel showing UPS load and mains voltage — handy during summer brownouts in regional South Australia or planned outages in Canberra suburbs.

Alerting and keeping the stack healthy

Alertmanager is the third pillar and arguably the most underrated. A single rule that fires when any node_exporter target has been unreachable for more than two minutes is usually enough to start. Route alerts through your notification stack, whether that is email, Telegram, or Pushover, to keep the signal out of the spam folder. Keep rules in version control, even if it is just a private Git repo on the same NAS you are monitoring — silent alerting failure is the worst during an incident.

The biggest mistake long-term is letting the data directory grow unchecked. Watch the storage panel in Grafana and prune jobs you no longer care about. Rotate Grafana admin passwords and the basic-auth password on the Prometheus endpoint, and back up dashboards through Grafana provisioning rather than the UI. A weekly cron job exporting every dashboard to JSON, then committing it to Git, has saved me more than once when a config store blew out during an unrelated upgrade.

Once everything is humming, the home lab feels less like a pile of boxes and more like a small data centre. Pull metrics when something feels off, glance at the temperature panel before summer kicks in, and sleep better knowing the next disk failure will not be a surprise.

Practical tuning checklist

After the dashboards and alerts are in place, a handful of small tunings will keep the stack tidy over the long term. The following changes pay off the most in a home environment, and most take less than ten minutes each to apply.

• Pin Prometheus retention to a value you actually intend to keep, and watch the data directory size in Grafana.
• Use file_sd_configs for target lists so adding a new host does not mean editing prometheus.yml.
• Run node_exporter with --collector.filesystem.mount-points-exclude to ignore tmpfs and overlay mounts.
• Enable TLS and basic auth on both Prometheus and Grafana, even on a home LAN.
• Keep dashboards, alert rules, and Prometheus configuration in a private Git repository.
• Set up smartctl_exporter and ipmi_exporter early — they are cheap to add and painful to add during an incident.
• Document the network path to each target; future-you will forget.

The same approach scales from a single node to a small business environment without changing the philosophy. Drop this playbook into a starter environment this weekend, and a few exporters plus a shared Grafana instance will replace most paid monitoring subscriptions.