My DIY smart lighting stack with ESP8266 and MQTT

Walking through my Brisbane house at 2am used to mean a careful dance of toe-stubbing and wall-hunting for light switches. After one too many collisions with a hallway bookshelf, I finally committed to a real home lighting automation project instead of relying on a few scattered smart bulbs.

The brief was simple: every light needed to respond to presence, time of day, and a manual override that actually worked. I also wanted a system I controlled, not a cloud service. That pushed me toward an ESP8266 and MQTT setup, with a Raspberry Pi running the broker and Home Assistant on top.

I have spent the last few months wiring twelve nodes around the house, replacing dumb switches with smart ones, and tuning automations until the lights feel like part of the building rather than a science project. The result costs a few hundred dollars in parts, runs entirely on my own network, and has already paid for itself in convenience.

This is the full story of how I did it, the mistakes along the way, and what I would change if I were starting over with a fresh set of tools and a soldering iron.

Picking the wireless brain

The ESP8266 is not the newest chip, but it is cheap, well documented, and backed by a community of thousands who have done exactly what I wanted. For a lighting project, you do not need the extra horsepower of an ESP32. You need a WiFi radio, a handful of GPIO pins, and the ability to publish a message when a button is pressed.

I went with NodeMCU boards because they have a built-in USB-to-serial converter and a voltage regulator, so I can flash them from a laptop without extra hardware. Each board costs a few dollars from local suppliers, and the form factor fits inside a standard Australian wall plate with some careful Dremel work.

One thing to watch out for is the 3.3V logic levels. If you are driving a relay module that expects 5V, you will get flaky behaviour. I learned that the hard way when a bedroom light refused to turn off one evening and just sat there flickering at about four hertz.

Sourcing parts locally

Living in Australia means the usual trade-off: components cost a bit more, but they arrive in a week rather than a month, and local suppliers actually know their stock. My main sources were Jaycar for basics like resistors, terminal blocks, and project boxes, and Core Electronics for the more specialised relay boards and sensors.

For the mains side, I stuck with 230V-rated components throughout. Australian light switches run on a different wiring standard than the UK or US, so the physical swap-out needed thought. I bought spare plates from Bunnings so I could prototype in the garage before touching the real wiring.

The total parts cost came in around three hundred and fifty dollars, including the Raspberry Pi, the SD card, and a small UPS for the network gear. That is less than a single mid-range commercial smart switch would have cost for every fitting in the house.

Standing up the MQTT broker

MQTT is the glue that holds the whole thing together. Every switch, sensor, and light publishes its state to a topic, and Home Assistant subscribes to those topics to drive its automations. The broker itself is just Mosquitto, and it runs happily on a Raspberry Pi 4 with a few hundred megabytes of RAM to spare.

I already had a Pi cluster running in the cupboard under the stairs, originally set up for CI work. Reusing that infrastructure kept the broker close to my other self-hosted services. The setup process is much the same as the one I wrote about in Deploying a GitLab Runner on a Raspberry Pi Cluster, so the broker will not slow you down.

Security matters here. I locked the broker down to local network connections, required authentication, and set up a separate user for Home Assistant with a strong random password. There is no reason to expose MQTT to the wider internet.

Firmware on the microcontrollers

For the firmware, I tried two paths. First, I flashed Tasmota onto a couple of boards to get a feel for the out-of-the-box experience. It works well and is a great way to prototype. Eventually I switched to a custom Arduino sketch because I wanted finer control over the debounce logic on the wall switches and the LED indicator behaviour.

Each node publishes its state to a topic structured by room and function, like home/livingroom/mainlight/state. Commands come back on a parallel topic. This naming convention makes the automations easier to read, and it scales as you add more devices. Give every board a unique client ID and a static DHCP reservation so a dead node at 1am is easy to track down.

Wiring it into Home Assistant

Home Assistant is the layer that turns a pile of MQTT topics into something a person can actually use. The MQTT integration auto-discovers any device that publishes the right discovery messages, so a new switch shows up in the dashboard within seconds of being powered on.

I built a simple floor plan view with lights grouped by room, plus a few helper entities that expose the whole-house state. Voice control through a Google Home integration rounds things out for the rest of the family, who were skeptical about yet another gadget on the network. The most useful feature has been the logbook. When a light turns on at an unexpected time, I can scroll back and see exactly which sensor triggered the event, and that kind of transparency is what separates a home automation project from a flaky toy.

Automations that earn their keep

The real win is not turning lights on with a phone. It is lights that turn on by themselves, in the right way, at the right time, and stay out of the way the rest of the time. My automations handle the hallway lights after sunset, the bathroom fan when humidity spikes, and the porch light when the front door opens after dark.

I use the sun component for sunset, which works well in Brisbane where the difference between summer and winter sunset times is about two hours. A motion sensor in the pantry means the door no longer needs a pull cord, and a time-based scene in the living room slowly dims the main light over twenty minutes before bed.

Fail-safes matter too. Every automated switch can still be operated manually from the wall, and I have a global kill switch that drops all the lights back to dumb mode if the broker goes down. That kind of fallback is what keeps the household happy when something misbehaves.

What broke and what I'd change

The biggest source of failures has been WiFi range. The ESP8266 is not a powerhouse radio, and a few of my original node locations were too far from the access point. I solved it with a mesh of cheap TP-Link repeaters, but if I were starting over I would pull Ethernet to the ceiling voids during the next renovation.

The second surprise was how often the USB power supplies failed. Cheap five-dollar bricks are tempting, but they die within months when they are crammed behind a switch plate. I now use name-brand supplies and check them every six months. Other projects around the house, like the calibrate a Coolscan 9000 workflow I documented for film scanning, taught me the same lesson about cheap power and the value of a proper runbook.

Recommendations before you start

• Pick a single naming convention for MQTT topics and stick to it from day one. Refactoring later is painful.
• Buy one extra NodeMCU and a breadboard. Prototyping on the bench catches mistakes before they are hidden behind plaster.
• Test every switch on a bench supply at 230V before installing it. A short circuit in a ceiling void is not a fun Saturday.
• Keep the manual override as a first-class feature, not an afterthought. The system must work when the network is down.
• Plan your WiFi coverage before you buy a single board. A node that cannot reach the access point is just expensive plastic.

If you want a hand designing or deploying a similar setup, or you need someone to look over your home network and storage plan, get in touch through the consulting page. I take on a small number of infrastructure and cloud migration projects each year and would be happy to chat about yours.