state of the homelab h2 2025

what’s running?

I’m going to start by walking through the physical and virtual servers, then dive into networking and finally the physical space and support stuff. I’ve been picking at this homelab stuff for quite a while, so it’s good to check in with what’s going on and how I feel about it.

servers

physical

I’ve got a three node proxmox cluster, which acts as my lowest level building blocks for compute resources. Two are 2u ryzen 9 3900x boxes, each with 128 gigs of memory, a boot nvme disk and a local nvme disk for VM storage. They’ve got dual 10g nics, and some nonsense gpu in there so that they’ll boot. They’re in a pretty slick 2u silverstone case, that was actually quite fun to build in. The third node in the cluster is an 8th gen intel nuc (critical for intel quicksync for plex). I’m on proxmox 8.4.11 as of this writing.

The primary nas is a 4u rackmount build with a ryzen 5 3600 and 64 gigs of memory. It’s got a 500 gig sata ssd to boot from, two 4tb nvme drives for a fast mirrored pool, a pair of 10tb spinning disks for a time machine backup pool, and ten 6tb spinning disks in a raid 10 pool for general storage. There is a dual 10 gig nic and another nonsense gpu, and a LSI 9300-16i 16-Port 12Gb/s SAS Controller HBA Card for connecting all those drives. It’s running truenas scale. I went through a couple of cases before landing on this particular one, but I’m very happy with how it is working in there.

I’ve got some support machines as well - a pikvm v4 plus hooks into an 8 port tesmart kvm. That gives me kvm access to all the servers (and various miscellany). I’ve got a 2020 m1 mac mini in there, mostly interacting with AV stuff. A 4 port protectcli box runs bind and kea for dns and dhcp in my lab (deliberately outside of proxmox so I can avoid bootstrapping issues).

I run some AV gear in there as well. For video, I’ve got an atem constellation 1 m/e as my primary switcher. There’s a hyperdeck hd mini for record and playback, and the aforementioned mac mini has a blackmagic design ultrastudio 3g for playback. There are some sdi to fiber converters that send and receive video signals up the fiber lines to my office, and an Apple TV to be an airplay video source. For audio, I’ve got a behringer x32 rack with a dante card and some sdi de-embedders to get audio out of the video pipeline and into the mixer.

virtual

In proxmox, I’ve got two main deployment flows: stuff that needs a VM (ie, legacy stuff that is complicated to containerize or a service that is required to get my kubernetes cluster up and running), and stuff that can run in my Kubernetes cluster. All the workloads deployed to k8s are deployed via ansible playbooks executed by forgejo runner actions. To support that, I’ve got dedicated vms for forgejo (my self hosted git forge) and forgejo action runner. I used to run a separate docker registry and package repo, but now I take advantage of forgejo’s built in package and docker image hosting to keep it all in one place. There is a vm running unbound for my home’s DNS usage (that sits in front of the bind box which is authoritative for my internal domains), a valheim server, a development debian vm, a homeassistant vm, and an nginx proxy vm (largely to put valid certs in front of homelab services and devices). For the most part, they’re created from the same debian 12 template and managed via ansible playbooks.

The k8s cluster has two worker nodes (one on each of the beefier proxmox nodes) and three control plane nodes (one on each proxmox node). It’s bootstrapped and managed by an ansible playbook I put together, and uses kubeadm for bootstrapping, calico as the cni and metallb in l2 mode, local etcd cluster on the control plane nodes for cluster state, and haproxy plus keepalived for API server VIPs. The k8s cluster has its own vlan and subdomain, making it super easy to keep segregated and to get to arbitrary services. As I mentioned above, all the stuff in the k8s cluster is deployed via ansible playbooks, invoked by forgejo runner actions.

In the k8s cluster, I’m running nginx gateway fabric for proxying, which has been pretty fun to set up. I’m using cert-manager (mostly just for acme certs for the nginx gateway fabric), sonarr/lidarr/radarr/sabnzbd for media, karakeep for link/article saving, and some static sites (my internal docs site and a development copy of this blog). I’ve got a few pieces to build still (biggest missing piece is my dns operator to get records synced for domains not owned by coredns), but the forgejo action workflow has been really great here. I’m using iscsi from truenas for persistent storage (I have yet to configure the democracy csi, opting to manually manage these disks for the moment, but getting the csi wired up is also on my short list).

networking

I’m an all unifi household. I’ve been using Ubiquiti gear since 2016 (edgerouter lite and UAP-AC-PRO), so my router, switches, and access points are all ubiquiti. I have symmetrical gigabit fiber to the home, so there’s a verizon ONT chilling on my wall, running into my UDM SE. My primary core switch is a Hi-Capacity Aggregation switch (28 10 gig ports and 4 25 gig ports). I have a USW-24-POE for my video doorbells, access points, anything in the basement that needs PoE, and a USW-PRO-24 for general 1 gig connectivity. The Hi-Capacity aggregation switch connects to a second Hi-Capacity aggregation switch in my attic over four single mode fiber runs - there are three rooms on the first floor of my house with dual cat6 runs plumbed to 10 gig ports on the basement agg switch, and the same configuration (three rooms, dual ethernet runs in each) on the second floor goes to the attic agg switch.

I’ve been generally quite happy with the Unifi stuff, and after a fallow period they seem to be shipping features pretty frequently and with a reasonable quality. I’m not super impressed with the multicast implementations - some experiments I did with SMPTE 2110 were truly disastrous, but for my dante audio network it’s worked really well, and my actual lab/work/home usage has all been great.

physical layout, cabling

All of this rackmounted gear lives in a 42u startech rack in my basement. I have the unifi smart pdu and a tripp lite ups so I can power cycle things remotely and ride out any power issues, respectively. I put a small wall mounted network rack in to terminate the eight fiber runs and dozen or so cat6 runs.

what’s cooking?

I think the core elements of the lab are pretty stable. I’m happy with Unifi for networking, I’m happy with Truenas for storage, I’m happy with Proxmox for virtualization. Doubling down on forgejo has worked out really well - actions are great, the package manager and docker repo stuff being built in is excellent (two fewer services for me to have to run). I should definitely sort out my iscsi csi stuff so that I’m not manually provisioning iscsi volumes anymore, and I need to get my bind + kea stuff into a gitops flow that works for me (that’s probably my least reliable service at the moment, which is bad because it’s pretty critical to my lab functioning).

The proxmox servers are coming up on 5 years old, so I’ll need to start to plan to replace them if they fail. I think I’m still bought in on the strategy (build a machine in a 2u case with a desktop cpu for the balance of power and performance), so not too complicated to execute on a replacement.

I’m interested in doing more with this blog, and maybe with my various and sundry websites in general, so I’ll probably need (want) to build some tools to that end. I need to iron out the image pipeline for blogging with Zola.

I’ve got lots of stuff running in nginx, but I’m thinking maybe caddy would be a more straightforward option for a webserver. Given my previous point about noodling more on web stuff, I might want to look at swapping webservers.

Oh and I should get a reliable/performant postgres cluster up and running at some point. Forgejo can be backed by postgres (it would be very annoying to have to recover from sqlite backups!), and just having a relational db that I can rely on/operate will open up new tools and workflows for me. I think this is probably contingent on sorting the csi k8s storage stuff first, but still pretty possible in the near term.