Skip to main content

Where it all began

2017

Gotta start somewhere.

Converted an old desktop into a NAS, using FreeNAS (before the name changed to TrueNAS). The system is pretty janky, consist of the following:

  • Random Lenovo Intel Motherboard, hacked together with an AMD cooler and fan
  • Intel i5-2500
  • 16GB (2x8) DDR3
  • 2x Syba 4-Port SATA PCIe x1 Expansion Card (This was before I learn about LSI HBA cards)
  • 12x 500GB Assortment of HDDs I salvage from work

IMG_20191125_224224.jpgIMG_20191125_224145.jpg

For the pfSense Firewall, it started out as an HP Elitedesk desktop, with holes cut in the front for the Ethernet NICs.

This was running on:

  • HP Intel Motherboard
  • 4th Gen Intel i5 Processor (Forgot which model)
  • 16GB (4x4) DDR3
  • 4x Gigabit PCIe NIC, mix of Intel and RealTek, that have a ribbon breakout cable to the physical NIC daughter boards
  • 1x 120GB SSD for Boot Drive

IMG_20190730_224053.jpg

IMG_20190730_223907.jpg

2018

Over time, as I accumulate more and more consumer hardware, my home lab grew and changed. Below is the upgraded janky hardware for FreeNAS:

  • Intel i7-4790K
  • 32GB (4x8) DDR3 RAM
  • Asus Z97 Motherboard
  • Syba SATA PCIe Expansion Card
  • 4x4TB HDD in RAID 10
  • 3x 480GB SSD in RAID 5
  • 2x120GB SSD as Mirrored Boot drives
  • Thermaltake Core P5 Open Air Case

It was also around this time I added an UPS to the setup (at last) to protect against random, momentary power outages.

20180419_231119.jpg

As I learn more and more about IT, I was slowly reaching the limit of the consumer-grade motherboards in my homelab. Around this time, I found a great deal on an AsRock Server motherboard, And so it begins my next phase of Homelab, where I use server grade hardware in a DIY mounting solution. Very jank. This time, the server runs ESXI (free license) as the hypervisor, with everything else virtualize underneath.

  • Dual Intel Xeon E3-2630L
  • 128GB (16x8) DDR3 ECC RAM
  • AsRock Rack EP2C602-4L/D16 Motherboard
  • 3x 1TB HDD on onboard SATA controller, pass-through to one FreeNAS VM
  • 3x 2TB HDD on PCIe SATA controller, pass-through to another FreeNAS VM
  • 3x 480GB SSD on onboard SATA controller, pass-through to a third FreeNAS VM
  • 3x 120GB SSD, used as ESXI VM Datastore

For networking, pfSense was transfer to a small NUC-like device, from Qotom Mini PC. I have also acquired a 16-Port gigabit Netgear Smart Switch that have basic VLAN functionality to start learning about VLANs.

Server case was too expensive at the time for me, and I had a lot of scrap wood around. So I decided to build a hanging wooden "case" to house the server on.

IMG_20190920_221903.jpg

20181129_234452.jpg

20181129_234505.jpg

20181129_234522.jpg

20181129_234540.jpg

2019

While open-air "case" can work, it's not a good idea to leave all the electronics exposed to elements. Sometime during the year, Newegg had a flash sale on a Rosewill L4000 4U server case for cheap, and so I upgraded the lab once more.

The components largely remind the same as before, just in a proper case now.

20200229_173514.jpg

After discover the r/homelabsales subreddit, I was able to score a good deal on a Cisco switch, thinking it will help me studying for the CCNA in the future. This switch is the SG200-52P, which is a huge upgrade over the previous 16-Port Netgear switch, in terms of both functionality and more ports.

But server racks are still out of the question, due to both cost and lack of space. Janky mounting it is.

IMG_20190731_002409.jpg

IMG_20190819_221009.jpg

2020

Lock down happened. What better way to spend the idle time than to DIY a wooden server rack?

The wooden server rack is made from 2x4s, which I got laying around, long before the pandemic skyrocket the cost of wood.

IMG_20200726_121229.jpg

IMG_20200727_142812.jpg

Drawer rails from Home Depot works well as rails for servers.

IMG_20200727_154316.jpg

From top to bottom:

  • 24-Port Patch Panel, using Keystone RJ45 Pass-through Jacks
  • Cisco SG200-52P PoE Gigabit Switch
  • 1U empty server case, used as a rack shelf, for holding the pfSense Mini PC
  • 2U ESXI DMZ Server
    • Runs Minecraft, Plex, Resume Website, and Wordpress Blog
    • Intel i7-4790K
    • 32GB (4x8) DDR3 RAM
    • Asus Z97 Motherboard
    • 3x 256GB SSD as ESXI Datastore
    • Nvidia P400 for Plex Transcoding
  • 4U ESXI Main Server
    • Runs TrueNAS, Unifi Controller, Pi-Hole, and a bunch of Winodws and Linux VMs
    • Dual Intel Xeon E3-2650L
    • 128GB (16x8) DDR3 ECC RAM
    • AsRock Rack EP2C602-4L/D16 Motherboard
    • LSI 9300 HBA card for all the HDDs
    • 2x 1TB SSD as ESXI Datastore
  • APC 1500VA UPS
  • CyberPower 1500VA UPS

IMG_20200728_175445.jpg

As COVID continues to rage on, my brother moved back home from college. His server equipment are added to the DIY rack.

Now there's one more 4U server, along with a few Raspberry Pi 3s.

Previously, I run Pi-Hole inside a VM on the ESXI host. With the Raspberry Pis as the physical Pi-Hole server, I don't have to worry about not having DNS resolution when I do maintenance on the ESXI Hosts.

IMG_20210514_214639.jpg

2021

Shortly after vaccination was available to the general public, I switched job. The company was in the middle of an physical office migration and was getting rid of half-height server racks.

Of course I took one home. It is a very nice 25U rack.

IMG_20210709_202407.jpg

All hardware transferred over.

IMG_20210726_231256.jpg

From top to bottom:

  • 24-Port Patch Panel
  • Cisco SG200-52P PoE Switch
    • Old, but very stable. Still running well.
  • 24-Port Patch Pannel
  • Netgear 8-Port Unmanaged PoE Switch
    • DMZ Switch, to physically separate the DMZ zone from the rest of the network
  • 1U Rack Shelf
    • Contains the pfSense Mini PC in the front, miscellaneous items in the back
  • Unifi Protect NVR
    • Expensive, but really nice surveillance system
    • One of the few camera systems that can be control locally and does not require any subscriptions
  • 1U 3D Print Pi-Rack
    • 2 Raspberry Pi 3 as Pi-Hole
  • 2U DMZ Server
    • ESXI 7.0
      • Minecraft, Plex, Wordpress, NGINX Reverse Proxy
      • Same hardware as before
  • 4U Main Server
    • ESXI 7.0
      • TrueNAS, Mumble Server, Unifi Controller, and a lot more
    • Same hardware as before
  • 4U Server
    • My brother's server, same hardware as my 4U
  • APC 1500VA UPS
  • CyberPower 1500VA UPS
  • Tripp Lite 1500VA UPS

IMG_20210726_235508.jpg

IMG_20210726_235447.jpg

Safety first. Gotta make sure the fire extinguisher is close by.

IMG_20210726_235349.jpg

2022

Electricity cost are on the rise. At this point, the Ivy Bridge Xeon Processors are over 8 years old, and are drawing more watts compared to a more modern processor.

Time for an internal upgrade. A LOT of upgrades.

From top to bottom:

  • 1U 24 Port Patch Panel
  • 1U 24 Port Patch Panel
  • Brocade ICX 7250
    • 48 Gigabit RJ45 Ports, 8x 10G SFP+ Ports
    • Going 10G for the servers network, because why not
    • Command Line operation for the switch at last; time for some automation
  • Netgear GS108LP PoE Switch
    • DMZ switch
    • Physical separation for the DMZ network
  • 1U Shelf, holding Raspberry Pi trays
    • From Left to Right:
      • Raspberry Pi 3B, Primary Pi-Hole
      • Raspberry Pi 3B, Secondary Pi-Hole
      • Raspberry Pi 4B 2GB, Home Assistant
      • Raspberry Pi 4B 2GB, Network Monitoring
      • Raspberry Pi CM4 8GB, Pi-Box, running NextCloud
  • 1U pfSense Server
    • Supermicro X10SDV-4C-TLN2F
      • Built-in Dual 10G Intel NIC
      • Xeon-D 1520 Low Power CPU
    • 16GB DDR4 ECC RAM
    • Intel i350-T4 Quad Gigabit PCIe NIC
    • 2x120GB SSD for mirrored Boot drive
  • Unifi Protect NVR
  • 2U Main ESXI Server
    • AsRock x470D4U AM4 Motherboard
      • One of the only Server grade motherboard for the consumer Ryzen CPUs
    • Ryzen 7 3800X
      • 8 Cores 16 Threads
    • 128GB (4x32GB) DDR4 RAM
    • 2x Nvidia Quadro P400 GPU
    • Supermicro Dual 10G SFP+NIC
      • One port is directly connected to the TrueNAS server for NFS share
  • 2U TrueNAS server
    • AsRock D1541D4U-2T8R Motherboard
      • Xeon-D 1541 Low Power CPU
        • 8 Cores 16 Threads
      • On-Board 10G RJ45 NIC
    • 4x12TB HDD in RAID 10
      • Bulk Media Storage
    • 2x 1.6TB SAS SSD in RAID 1
      • ESXI NFS Datastore
    • 2x 1.6TB SAS SSD in RAID 1
      • Backup Pool
    • 2x Dual 10G SFP+ PCIe NIC
      • One direct connection to ESXI server for NFS share
      • One SMB only connection to rest of network
      • One webGUI only connection to rest of network
  • 4U Server
    • Brother's server, also upgraded to Ryzen with Asrock Mobo

Is this even a home lab anymore? It feels more and more like a home data center instead.

IMG_20220916_205843.jpg

On the back, from top to bottom:

  • 1U PDU, connected to CyberPower UPS
  • 1U APC ATS, connected to both APC UPS and Tripp Lite UPS
    • Networking Equipment are connected to this ATS, for extended run time
    • With only the pfSense, Brocade Switch, Unifi Access Points, and Verizon ISP modems, the three UPS gives about 30 minutes of continuous run time during the recent power outage
  • 1U PDU with switches for quick ISP modem reboots
  • 1U VGA KVM

IMG_20220916_205941.jpg

What's next?

More hardware.

Currently planning on testing Tape backup with LTO4 drives.

Also want to test virtualization clustering using Lenovo M720q and M920q Tiny machines as compute nodes in a Proxmox Cluster.