Objective
Document the characteristics of the physical servers and the criteria used to choose them, balancing performance, power consumption, noise, and running costs.
Architecture
The infrastructure is built on three mismatched x86 servers (secondhand workstations and servers) dedicated to virtualization, with CPUs that have virtualization extensions, memory sized for real workloads, and solid-state disks. Two nodes have dedicated GPUs for passthrough to VMs. An extra network card separates cluster traffic from service traffic. An uninterruptible power supply protects the nodes and the network equipment.
Details
Hardware classes
The three nodes are not twins: they are secondhand workstations and servers collected over time, not bought as an identical kit. The balance between power, noise, and running cost matters more than the peak power of any single node: each machine plays the role it suits best (the most powerful server hosts the heavy workloads, the other two the light services and the labs). A second network card makes it possible to separate cluster traffic from service traffic.
Power consumption, noise, and cooling
A homelab stays on around the clock, so power consumption and noise weigh as much as performance. The most powerful node (a dual-processor rack server) is also the noisiest and the one with the highest consumption: it stays on because I need the compute capacity, but it is the first candidate for replacement with something more efficient. Assigning VMs only the resources they need avoids overloading the smaller nodes.
Power and redundancy
An uninterruptible power supply (UPS) protects the nodes and the network equipment from surges and short outages, and signals the nodes to shut down cleanly when the battery runs out. Redundancy relies mostly on the cluster nodes rather than on the internal components of a single server: the cluster setup is described in Three-node Proxmox VE cluster: setup, quorum, and first checks.
Current setup
| Node 1 | Node 2 | Node 3 | |
|---|---|---|---|
| Manufacturer and model | HP Z220 SFF Workstation | Dell PowerEdge R630 | Custom-built PC (OEM motherboard) |
| CPU | Intel Xeon E3-1245 V2 @ 3.40 GHz | 2× Intel Xeon E5-2680 v4 @ 2.40 GHz | Intel Core i7-4770 @ 3.40 GHz |
| Cores / threads | 4 / 8 | 28 / 56 | 4 / 8 |
| RAM | 32 GB | 80 GB | 24 GB |
| Storage | ~465 GB SSD | ~3.9 TB SSD + ~1.8 TB HDD | ~1.6 TB SSD |
| GPU | 1× NVIDIA RTX A2000 | 2× NVIDIA RTX A400 | — |
- CPU: all with virtualization extensions (VT-x/VT-d), so dedicated devices can be passed through to VMs when needed.
- GPU: the professional cards on nodes 1 and 2 are dedicated to passthrough to the virtual machines that need them (e.g. Immich for hardware acceleration, compute workloads), not to the host's graphics.
- Network per node: two Ethernet interfaces, one for service traffic and one dedicated to cluster traffic (Corosync) and ZFS replication.
- UPS: Eaton 5PX2200IRT2UG2, line-interactive, 2,200 VA / 2,200 W, pure sine wave, 8 C13 and 2 C19 outlets, hot-swappable batteries, 2U rack format. Monitored via USB HID and RS-232 serial, so the nodes receive the clean-shutdown signal when the battery runs out (it covers the nodes, switch, and firewall; it is not meant to keep things running for long without mains power, only to allow a clean shutdown).