This is an automated archive made by the Lemmit Bot.
The original was posted on /r/homelab by /u/Theduke322 on 2025-01-11 21:57:38+00:00.
Happy New Year! In response to u/osuno1 on my homelab setup from two years ago, My Personal Setup with XCP-ng, I decided it's a great time to post an update on my homelab.
Hardware
Over the years, my homelab has undergone many iterations. Currently, it features a functional, though perhaps slightly excessive, design. Here’s an overview, top to bottom (some items are stacked front and back):
- StarTech 25U Open Frame Rack
- Cable Matters 1U 24 Port Keystone Patch Panel
- UniFi Switch PRO 48
- Supermicro 1U Ryzen Server
- 4x HP DL360 Gen9
- 2x Arista 7050S 48 Port SFP+ Switch
- 2x CyberPower 15A PDU
- Unifi Network Video Recorder
- Isilon NL410
- Triplite SmartPro UPS 2.25 kW
- 2x APC Smart-UPS X 3000 2.7kW
- PfSense Router Whitebox Server
25U has been enough to fit all of the hardware I've needed. I like the breathable design of the StarTech rack and I haven't had any issues with it. However, I have considered a 42U rack to help with cable management and maintenance.
Networking
I'm using a pair of Arista 10/40G switches as my network backbone. They serve as the gateway for some trusted networks and house the VLANS for everything else. I have 10G connectivity between each server and these switches, with an 80G trunk between the two switches. The UniFi Switch Pro manages the access layer, providing PoE to the wireless access points and security cameras, as well as handling management and iDRAC connections. All UniFi security cameras record to the UNVR and are accessible remotely. My firewall software of choice is PfSense, running on a custom server with an i7-9700k, 16G of DDR4 RAM, and a 10G fiber card. All of my untrusted networks are housed here.
Compute
My most powerful server is a recently purchased Supermicro 1U Ryzen Server. It features a Ryzen 5950x, 128G of DDR4 RAM, and a 10G fiber card. The next two servers are identical HP DL360 Gen 9s, each equipped with dual E5-2630 v4 CPUs, 336G of DDR4 RAM, and a 10G fiber card. These three servers make up my Proxmox cluster, which hosts all of the services in my homelab. I have an additional HP DL360 that serves as a standalone Plex Media Server.
Storage
I have substantial storage in my homelab, thanks to the Isilon NL410. It has 36 hard drive bays, populated with a mix of 3TB to 22TB hard drives, providing a current capacity of 244TB. My second storage server is another HP DL360 with all-flash storage. It contains six 3.2TB SAS SSDs in a 3x mirror, two-wide configuration, giving me roughly 9TB of fast storage for Proxmox. This server also holds my file shares.
Power
To power my homelab, I needed to set up substantial infrastructure. I had an electrician install two 30A/120V circuits dedicated to the lab. Each circuit powers an APC Smart-UPS X 3000, which in turn powers a CyberPower 15A PDU. Additionally, a 20A/120V circuit powers the Triplite SmartPro UPS. Altogether, my homelab draws 10-12 amps continuously. Just don't ask about the power bill...
Software
My software and services are very similar to two years ago. For my sanity, I've really stuck to the expression, if it ain't broke, don't fix it, because if you do fix it it'll end up broke. Once I setup a key piece of my infrastructure I tend to leave it alone.
Networking
My network backbone is a pair of Arista 10/40G switches but for my router/firewall I am using PFSense. PFSense is a feature rich open-source firewall that practically runs on any piece of hardware. A majority of my VLANs terminate here. Those VLANs include:
- Management
- DMZ
- IOT
- Guest
- Camera
- VDI
I previously had a trusted device VLAN and server VLAN, but those were moved to the Arista switches for 10G routing capability. Aside from basic routing functions, these are the features and packages I utilize:
- ACME for automated LetsEncrypt certificates
- Avahi for Multicast DNS
- FRR for OSPF
- HAProxy for HTTP load balancing
- Tailscale for mesh VPN
- Dynamic DNS
- DHCP Relay
The benefit of PFSense for me is the ability to consolidate multiple capabilities into one rock solid device. Other products such as OPNsense are just as powerful but I'm not interested in switching any time soon.
Unifi handles everything WiFi and Security. I have the Unifi Controller hosted on-premises in a Debian VM. It manages my two access switches and three access points. I have four Wi-Fi networks for trusted, IOT, guest and security devices. Unifi devices are a little pricey, but they perform well and have a great feature set.
Compute
Proxmox is my hypervisor of choice and has been for some time. To briefly throw out honorable mentions, I've also used XCP-ng and ESXi. I settled on Proxmox primarily because of backup performance issues I was having with XCP-ng, and the cost of ESXi. For the most part Proxmox provides everything I need, and it's free!
The three servers I have provide an impressive 112 logical CPUs and 786GB of DDR4 ram in my cluster. I primarily use Linux with Debian as my preferred distro. I also have some Windows machines for domain services and jump boxes. Here is an overview of my virtual machines:
- proddc03 - Primary domain controller. Handles DNS and DHCP as well.
- mw11vm - Personal Windows 11 VDI.
- lw11vm - Friend's Windows 11 VDI.
- jumpbox01 - Debian jump box (allows access to servers while external via SSH)
- docker01 - Debian VM running majority of my docker containers.
- webmp - Debian VM running docker containers related to photography business.
My services include:
- Nextcloud - A file hosting service that has become my Dropbox replacement. I have "unlimited storage" with no fees, as many users as I want, and access from anywhere.
- Gitea - A Git repository where I store my code projects and my Ansible configurations.
- Ghost - A blog and newsletter platform.
- Bitwarden - A password manager that is my LastPass replacement. I store my own passwords here as well as share passwords with my family.
- Outline - A glorified note taking app, Outline is my Notion replacement. I use this as my digital brain and store everything that's important to me.
- Zitadel - An identity provider that primarily supports my Outline instance. It piggybacks off my Active Directory domain.
- Immich - A photo and video backup management solution that is my Google Photos replacement. Similar to Nextcloud I have "unlimited storage" with no fees, as many users as I want, and access from anywhere.
- Plex - A media streaming solution for my DVD and CD collection.
- Homepage - A dashboard to quickly access all of my services.
- UptimeKuma - A monitoring tool so I know if my services go down.
- Portainer - A container management tool that I use to manage containers on docker01.
- NocoDB - A no-SQL database.
- Headscale - An open-source implementation of Tailscale. Tailscale is a mesh VPN, and I use it to connect back to my home network when I'm away.
- Lychee - A photo management tool I use to share my photography work in galleries.
- Nginx - A web server that hosts my static photography website.
- Paperless-ngx - A document management system I use to eliminate paper in my life.
Storage
TrueNAS is my storage software of choice. Another popular storage software is Unraid, but I think it compromises on speed and stability for ease of use.
TrueNAS neptune pool dashboard
My first TrueNAS host, Neptune, has the most storage at a little under 250TB. I have a mix of 3, 12, 16, and 22TB drives separated into five RAIDZ1 V...
Content cut off. Read original on https://old.reddit.com/r/homelab/comments/1hz70cs/homelab_setup_xcpng_back_to_proxmox/