Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In Register

Quick Links


Categories

In this Discussion

› › How would you virtualize this?
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

How would you virtualize this?

raindog308raindog308 Administrator, Veteran
edited February 2022 in Help

I have a Protectli Vault min computer I'd like to use as a home firewall and virtualization platform, but am not sure what the ideal virtualization software/storage config is.

Hardware

  • i5-8250U CPU (1.6Ghz/3.4Ghz Kaby Lake)
  • 32GB of RAM
  • 256GB mSATA SSD
  • 1TB 2.5" SSD
  • 6 gig-E ports

My Internet is 600/10.

Goals

  • I'll be using it primarily as a firewall. Ethernet from my Internet cable modem will plug into one port and the others will be used to whack up my home network and connect various switches and WAPs.

  • I'm planning to turn on IDS, IPS, VPN, etc. - many network toys and security options that take CPU. However, talking with others, people say that even with an office of people using VPN + Snort + etc., a Celeron/i3 with 4-8GB of RAM is enough. I have 4-6 people max so I think there's a lot of extra horsepower I'd like to exploit.

  • I'd like the virtualization to prioritize the pfSense VM over all the others, if possible, but the others will be a lot less busy so this may not be critical.

VMs

I'd like to host:

  • a pfSense VM, which will be the main VM

  • a couple small (1 core, 1G RAM) OpenBSD systems that will be alive but idling 99% of the time

  • two Linux VMs...probably no more than 1 core and 2GB of RAM, and these will be significantly more active but not insane

Virtualization and Storage

I realize we're only talking a couple drives here, nothing is redundant, etc. Everything will be backed up and if something fails, a delay while I rebuild is acceptable. mSATA is a little faster but not dramatically so.

None of the VMs needs more than 20-30GB or so of storage.

Some options:

  • Put everything on the mSATA, software raid to SSD partition, use the spare SSD space for something else. I checked and mSATA tops at 750MB/sec and SSD at 580MB/sec. The two Linux VMs are running on Raspberry Pi and are fine on MicroSD so extreme performance isn't needed :-)

  • Put everything on SSD and use SSD solely for backups

  • Divide VMs over both

  • Put the pfSense VM alone on mSATA and then have all other VMs on the SSD to segregate I/O - in other words, no matter what the other VMs are doing, they can't slow down pfSense on storage. However, I think that rather wastes the storage speed because I don't think pfSense is that I/O-intensive.

Virtualization options

"Popular virtualization software that has been tested on the Vault include XCP-ng, antMan from Antsle, Proxmox, ESXi, and Unraid."

I'm thinking any virtualization that supports x86-64 will work.

Of these, XCP-ng is Protectli's favorite. Never used it. Have used VMware at work and that would be fine. I know many hear seem to like Proxmox.

So...in this situation, any recommendations?

Thanked by 1pan_ia0_net

Comments

  • webcraftwebcraft Member
    edited February 2022

    @raindog308 said:
    Put the pfSense VM alone on NVMe and then have all other VMs on the SSD to segregate I/O

    Did I miss or where did you mention the NVMe before?

    Apart from this, if you plan to go with something other than Linux in your VM, my advice is to use VMware because this shows less complications from experience.

    Thanked by 2raindog308 pan_ia0_net
  • raindog308raindog308 Administrator, Veteran

    @webcraft said:

    @raindog308 said:
    Put the pfSense VM alone on NVMe and then have all other VMs on the SSD to segregate I/O

    Did I miss or where did you mention the NVMe before?

    Typo - I was referring to the mSATA. Corrected.

    Thanked by 1pan_ia0_net
  • jsgjsg Member, Resident Benchmarker

    @raindog308

    Some quick remarks:
    I would advice against running pfSense in a VM. I know, I know, SDN and that processor good enough, but still, pfSense is the core job of that box and I'd recommend on it also serving as the node OS
    I don't know how complete a FreeBSD pfSense is, but in case it doesn't support virtualization ("Bhyve", very nice Virt. about as fast as KVM/ProxMox) yet I'd simply build a kernel that does. It does run all BSDs and AFAIK all linuxes (albeit with some fumbling due to certain distro's peculiarities like e.g. booting into an uncommon partition).
    Re "disk(s)": why not give 2 vdisks to the VMs, one mSata and one (larger) SSD based? Re. "redundancy": Just get an external USB 3 dual drive enclosure, put 2 SSDs into it et voilà you've got a (soft-)raid 1 drive in addition to or instead of the internal SSD.
    Result could be 2 linux VMs + 2 OpenBSD VMs, each with a (presumable smallish) fast mSata vDrive + a (presumably generously) large "data" drive that is safe (mirrored). Plus a happily humming pfSense on the node itself.

    Thanked by 2raindog308 pan_ia0_net
  • craigbcraigb Member

    Given the primary security role, you might disable hyperthreading to reduce the guest escape attack surface for the Internet facing pfsense VM. Alternatively, pin 2 logical cores associated with 1 physical core to that VM to achieve a similar effect. This latter approach leaves 6 threads for other VMs so may be preferred. In a similar spirit, consider using PCI passthrough for each of the NICs to benefit both from a security and performance perspective.

    There's good support for these features in KVM these days.

    Regarding disk allocation, I would base it on workload profile. From what you wrote, I might be tempted to put pfsense and one of the busy Linux VMs on the smaller drive and everything else on the bigger drive.

    Thanked by 2raindog308 pan_ia0_net
  • shadowbanshadowban Member

    +1 for bhyve
    Got a similarly oversized opnsense box where I'm planning to get that up and running. Eventually...
    I know you have the CPU cycles to spare, but I wouldn't trade a virtio for hardware offloading myself on a router. I'm also not an expert on this LOL
    Also the security considerations mentioned above

  • tjntjn Member
    edited February 2022

    I've been running pfsense on multiple (10+) Proxmox nodes (on-prem and cloud) for a few years now without issue.

    Should be fine for your home setup!

    @craigb said:
    In a similar spirit, consider using PCI passthrough for each of the NICs to benefit both from a security and performance perspective. [...]
    From what you wrote, I might be tempted to put pfsense and one of the busy Linux VMs on the smaller drive and everything else on the bigger drive.

    This is the way.

    @raindog308 said:

    • I'd like the virtualization to prioritize the pfSense VM over all the others, if possible, but the others will be a lot less busy so this may not be critical.

    As someone else mentioned above, you can either dedicate/pin a number of cores to pfsense, or what I prefer doing is increasing my pfsense VM's CPU Units to 2048.

    cpuunits:
This is a relative weight passed to the kernel scheduler. 
The larger the number is, the more CPU time this container gets. 
Number is relative to the weights of all the other running containers. 
The default is 1024. You can use this setting to prioritize some containers.

    There are a few settings/tweaks I would recommend for pfsense on Proxmox, so holler if you end up going down that route.

Sign In or Register to comment.