All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
How would you virtualize this?
I have a Protectli Vault min computer I'd like to use as a home firewall and virtualization platform, but am not sure what the ideal virtualization software/storage config is.
Hardware
- i5-8250U CPU (1.6Ghz/3.4Ghz Kaby Lake)
- 32GB of RAM
- 256GB mSATA SSD
- 1TB 2.5" SSD
- 6 gig-E ports
My Internet is 600/10.
Goals
I'll be using it primarily as a firewall. Ethernet from my Internet cable modem will plug into one port and the others will be used to whack up my home network and connect various switches and WAPs.
I'm planning to turn on IDS, IPS, VPN, etc. - many network toys and security options that take CPU. However, talking with others, people say that even with an office of people using VPN + Snort + etc., a Celeron/i3 with 4-8GB of RAM is enough. I have 4-6 people max so I think there's a lot of extra horsepower I'd like to exploit.
I'd like the virtualization to prioritize the pfSense VM over all the others, if possible, but the others will be a lot less busy so this may not be critical.
VMs
I'd like to host:
a pfSense VM, which will be the main VM
a couple small (1 core, 1G RAM) OpenBSD systems that will be alive but idling 99% of the time
two Linux VMs...probably no more than 1 core and 2GB of RAM, and these will be significantly more active but not insane
Virtualization and Storage
I realize we're only talking a couple drives here, nothing is redundant, etc. Everything will be backed up and if something fails, a delay while I rebuild is acceptable. mSATA is a little faster but not dramatically so.
None of the VMs needs more than 20-30GB or so of storage.
Some options:
Put everything on the mSATA, software raid to SSD partition, use the spare SSD space for something else. I checked and mSATA tops at 750MB/sec and SSD at 580MB/sec. The two Linux VMs are running on Raspberry Pi and are fine on MicroSD so extreme performance isn't needed :-)
Put everything on SSD and use SSD solely for backups
Divide VMs over both
Put the pfSense VM alone on mSATA and then have all other VMs on the SSD to segregate I/O - in other words, no matter what the other VMs are doing, they can't slow down pfSense on storage. However, I think that rather wastes the storage speed because I don't think pfSense is that I/O-intensive.
Virtualization options
"Popular virtualization software that has been tested on the Vault include XCP-ng, antMan from Antsle, Proxmox, ESXi, and Unraid."
I'm thinking any virtualization that supports x86-64 will work.
Of these, XCP-ng is Protectli's favorite. Never used it. Have used VMware at work and that would be fine. I know many hear seem to like Proxmox.
So...in this situation, any recommendations?
Comments
Did I miss or where did you mention the NVMe before?
Apart from this, if you plan to go with something other than Linux in your VM, my advice is to use VMware because this shows less complications from experience.
Typo - I was referring to the mSATA. Corrected.
@raindog308
Some quick remarks:
I would advice against running pfSense in a VM. I know, I know, SDN and that processor good enough, but still, pfSense is the core job of that box and I'd recommend on it also serving as the node OS
I don't know how complete a FreeBSD pfSense is, but in case it doesn't support virtualization ("Bhyve", very nice Virt. about as fast as KVM/ProxMox) yet I'd simply build a kernel that does. It does run all BSDs and AFAIK all linuxes (albeit with some fumbling due to certain distro's peculiarities like e.g. booting into an uncommon partition).
Re "disk(s)": why not give 2 vdisks to the VMs, one mSata and one (larger) SSD based? Re. "redundancy": Just get an external USB 3 dual drive enclosure, put 2 SSDs into it et voilà you've got a (soft-)raid 1 drive in addition to or instead of the internal SSD.
Result could be 2 linux VMs + 2 OpenBSD VMs, each with a (presumable smallish) fast mSata vDrive + a (presumably generously) large "data" drive that is safe (mirrored). Plus a happily humming pfSense on the node itself.
Given the primary security role, you might disable hyperthreading to reduce the guest escape attack surface for the Internet facing pfsense VM. Alternatively, pin 2 logical cores associated with 1 physical core to that VM to achieve a similar effect. This latter approach leaves 6 threads for other VMs so may be preferred. In a similar spirit, consider using PCI passthrough for each of the NICs to benefit both from a security and performance perspective.
There's good support for these features in KVM these days.
Regarding disk allocation, I would base it on workload profile. From what you wrote, I might be tempted to put pfsense and one of the busy Linux VMs on the smaller drive and everything else on the bigger drive.
+1 for bhyve
Got a similarly oversized opnsense box where I'm planning to get that up and running. Eventually...
I know you have the CPU cycles to spare, but I wouldn't trade a virtio for hardware offloading myself on a router. I'm also not an expert on this LOL
Also the security considerations mentioned above
I've been running pfsense on multiple (10+) Proxmox nodes (on-prem and cloud) for a few years now without issue.
Should be fine for your home setup!
This is the way.
As someone else mentioned above, you can either dedicate/pin a number of cores to pfsense, or what I prefer doing is increasing my pfsense VM's
CPU Units
to 2048.There are a few settings/tweaks I would recommend for pfsense on Proxmox, so holler if you end up going down that route.