Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


SolusVM + KVM + ZFS?
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

SolusVM + KVM + ZFS?

MikePTMikePT Moderator, Patron Provider, Veteran

Hello folks,

Do any of you know if it's possible to create KVM containers using ZFS? It seems the IO improves a lot, which helps delivering a better IO for the clients. Any information about this? How would you guys do it?

Happy to share my thoughts so we can find a solution to achieve this! :)

Comments

  • I'm curious about the same, but with VIRTUALIZOR instead of SolusVM!

    Thanked by 1MikePT
  • matthewvzmatthewvz Member, Host Rep

    Hmm. Might have to find a VM or something to test it out on. If theres a will theres a way.

    Thanked by 1MikePT
  • ZFS on Linux in production - no thanks. ZFS on FreeBSD is good and damn reliable.

    But you don't want the storage on the same as the VPS's because I'm sure you'll want to enable the compression functionality in ZFS to give the illusion of higher performance. ZFS without compression enabled doesn't give magnificent performance, ZFS with compression enables gives slightly better performance but looks great on for DD aficionados.

    There are plenty of guides on building ZFS toasters but needless to say, lots of SAS ports, don't share ports or use extenders, don't use RAID controllers, use decent SSD for ZIL and so on.

    Thanked by 2MikePT lukesUbuntu
  • MikePTMikePT Moderator, Patron Provider, Veteran

    @ItsChrisG said:
    I'm curious about the same, but with VIRTUALIZOR instead of SolusVM!

    Yeah, I'll be providing test KVM to the LET community, to give it a try, now I'm trying to find a way to do it.

  • MikePTMikePT Moderator, Patron Provider, Veteran

    @matthewvz said:
    Hmm. Might have to find a VM or something to test it out on. If theres a will theres a way.

    I have a node here ready to test it.

  • MikePTMikePT Moderator, Patron Provider, Veteran

    @MarkTurner said:
    ZFS on Linux in production - no thanks. ZFS on FreeBSD is good and damn reliable.

    But you don't want the storage on the same as the VPS's because I'm sure you'll want to enable the compression functionality in ZFS to give the illusion of higher performance. ZFS without compression enabled doesn't give magnificent performance, ZFS with compression enables gives slightly better performance but looks great on for DD aficionados.

    There are plenty of guides on building ZFS toasters but needless to say, lots of SAS ports, don't share ports or use extenders, don't use RAID controllers, use decent SSD for ZIL and so on.

    Certainly Mark, I need to figure out a way to do it, software RAID for testing then hardware RAID 10, if it helps in IO even with compression enabled, then I shall try it. I'll see what I can do here.

  • No RAID for ZFS

    ZFS provides its own software 'RAID'

    Thanked by 1MikePT
  • MikePTMikePT Moderator, Patron Provider, Veteran

    @MarkTurner said:
    No RAID for ZFS

    ZFS provides its own software 'RAID'

    Wasn't aware about it, I shall investigate further.

  • Wasn't aware about it, I shall investigate further.

    For ZFS you want the raw disk presented to ZFS. Don't skimp on the SSDs for ZIL, use decent ones only.

    Make sure the whole system is SATA3 and all devices are connecting at 6Gbps. Make sure you don't have any SATA expanders in there, many of the larger Supermicro boxes use single or dual connections to a SATA expander so it heavily contends capacity from the HBAs to the disks.

    Don't have all your ZILs on one controller, split them around. So if you have 20 disks and 4 ZILs then split the 4 ZILs onto different controllers.

    Don't use 16 port controllers, performance is crap under load you start getting bus contention when using x8 slots. 8 Ports perform better in my 6+ month test on building these systems.

    Thanked by 1MikePT
  • One other thing, if you are going to use them as a backend for KVM then you need to decide whether you'll use NFS or ISCSI. If you're going to have more than a few VMs using the storage make sure you have multiple 10GE's between the VM server and the ZFS server

    Thanked by 1MikePT
  • raindog308raindog308 Administrator, Veteran

    MarkTurner said: One other thing, if you are going to use them as a backend for KVM then you need to decide whether you'll use NFS or ISCSI. If you're going to have more than a few VMs using the storage make sure you have multiple 10GE's between the VM server and the ZFS server

    Even with multiple 10GE between the KVM server and the ZFS NAS appliance/server, is that performance really going to be in the same category as RAID-on-SSD? I'm asking out of curiosity...

    Thanked by 1MikePT
  • MikePTMikePT Moderator, Patron Provider, Veteran

    Thanks for the posts folks, I've been trying to get ZFS working, it misses the kernel headers, although I've installed another version of the kernel, this is simply confusing and I might give up :P

  • @MrGeneral - If you are wanting to try out VM's on a ZFS backed storage, try FreeNAS, and mess around with the jails. It's probably the easiest way to accomplish what you are wanting to accomplish. They will not be KVM containers however. Someday I personally hope to start running VM's on FreeBSD nodes.. but that is another topic.

    Thanked by 1MikePT
  • MikePTMikePT Moderator, Patron Provider, Veteran

    @cncking2000 said:
    MrGeneral - If you are wanting to try out VM's on a ZFS backed storage, try FreeNAS, and mess around with the jails. It's probably the easiest way to accomplish what you are wanting to accomplish. They will not be KVM containers however. Someday I personally hope to start running VM's on FreeBSD nodes.. but that is another topic.

    Sure thing! I'd want to try to install SolusVM, KVM containers using ZFS, not sure if it's achievable, I lack the knowledge to achieve it though.

  • MikePTMikePT Moderator, Patron Provider, Veteran

    Yep, tried it, kernel headers are missing for the current kernel, I've reinstalled those but it's not linking to the right kernel. Oh well, I guess I should indeed learn this :P

Sign In or Register to comment.