Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Missing CPU features on KVM is now a critical performance issue
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Missing CPU features on KVM is now a critical performance issue

Many KVM providers hesitate to passthrough host CPU features. Missing SSSE3, SSE4, AVX2, and AES-NI incur significant performance penalties only under specialized workload (e.g., encryption, although there is a hack to force it).

Now there is an argument that the Meltdown patch makes PCID a much more important performance feature. Hopefully this will nudge providers to do CPU passthrough, but I am not holding my breath...

https://groups.google.com/forum/m/#!topic/mechanical-sympathy/L9mHTbeQLNU

Thanked by 2caracal rm_

Comments

  • What are some of the reasons providers hesitate to do so? Does it increase the amount of resources per VM?

  • caracal said: What are some of the reasons providers hesitate to do so? Does it increase the amount of resources per VM?

    Incompetence, perhaps? It doesn't increase the amount of resources one iota for most CPU features, although I am not sure if circumstances like this would happen if every VM is using AVX-512 like crazy.

  • Possibly a way to prohibit nested virtualization inside the vm.

    Of course passthough is preferred these days. In most cases, exposing cpu features like AES-NI reduces CPU loads.

    Thanked by 1Falzo
  • Enjoy openvz. Mmm

  • FranciscoFrancisco Top Host, Host Rep, Veteran

    I've had issues with some things not being bootable when in Intel mode.

    We simply have a drop down to allow users to adjust.

    Francisco

  • cause said: Possibly a way to prohibit nested virtualization inside the vm.

    Well, providers can at least choose which features to passthrough, no?

  • @psb777 said:
    Well, providers can at least choose which features to passthrough, no?

    s/way/wrong way/
    Yes, but one needs to specify correct set of cpu features to passthrough manually, iirc.

  • ClouviderClouvider Member, Patron Provider

    What features would you like to see exposed?

  • AnthonySmithAnthonySmith Member, Patron Provider
    edited January 2018

    psb777 said: Incompetence, perhaps?

    No, the reason is lack of automation 9 times out of 10, I assume as this is so much of a concern to you I can safely say you don't use shitty out of date (most of the time) pre-created templates to install your OS.

    SolusVM for example only allows you to set the CPU passthrough options on a per operating system template bases only, I put that in bold so you would understand just how completely insane that is from a logic perspective, but as hosts that is what a lot of us are working with.

    So being the caring customer you are, you install your OS from ISO (in my experience about 80% of customers use ISO's over templates) you get stuck with the ide driver and qemu emulation for your CPU because solusvm does not afford you the same options on a per ISO basis (not that the template/ISO is by any way shape or form the right place for this to be set anyway) so you must specifically request itvia a ticket.

    SolusVM have been pushed very hard on this, they confirm without compromise that this is by design and they wont change it, so we can expect more of this nonsense in v2 when it comes out in 2019.

    We as hosts cant watch what 10,000 customers are doing on that much of a granular level so I am afraid you do need to request it via ticket.

    Obviously, if a host says no then that is for them to justify on a case by case bases, having passthrough does not automatically enable nested virt either so that is not really a valid reason.

    hope that clarifies/

  • cause said: In most cases, exposing cpu features like AES-NI reduces CPU loads.

    +1 , not using builtin features will cause an overall performance hit - I never understood why providers, esp. those that push nodes to the limit often don't care about that.
    most likely what @AnthonySmith said - no automation, no passthru...

    Thanked by 1lion
  • Wasn't aware of this at all. Both my KVM's are custom installs (Arch & FreeBSD).

    Question, if you use a template to install say Ubuntu 16.04 and then upgrade to 18.xx, do you lose whatever options where configured in the base iso?

  • @DediServe don't allow CPU passthrough and reject my request for allowing it

  • @danninov said:
    @DediServe don't allow CPU passthrough and reject my request for allowing it

    Not sure what you mean, CPU is already pass-through, you can see the model from VM, if you're talking about specific feature pass-through, OnApp does not support this.

    Thanked by 1vimalware
  • Clouvider said: What features would you like to see exposed?

    I think here we were talking about "Process-Context Identifiers" aka pcid, which happens to be a performance feature since the Meltdown patch. According to benchmarks, to some extent it does save performance from degrading caused by kpti.

    Me? Honestly I don't really care. Basically I've got two classes of VM's: solid instances from professional providers that already support CPU passthrough; and cheap toys that privilege escalation or info leakage vulnerabilities like Meltdown is simply a non-issue.

    AnthonySmith said: SolusVM have been pushed very hard on this, they confirm without compromise that this is by design and they wont change it, so we can expect more of this nonsense in v2 when it comes out in 2019.

    Thanks for the explanation. I agree that given the current status of SolusVM it makes sense to open a ticket to ask for CPU passthrough. But hey we've got a drop down to choose from e1000 and virtio, why not adding another one for CPU passthrough. Without it, people might be unaware of the this option and its performance implications. Well, that's the whole point of this post..

  • dediserve said: Not sure what you mean, CPU is already pass-through, you can see the model from VM, if you're talking about specific feature pass-through, OnApp does not support this.

    Nope you don't, at least not on my VPS. Here my VPS (fixed instance)

    cat /proc/cpuinfo
    processor   : 0
    vendor_id   : GenuineIntel
    cpu family  : 6
    model       : 13
    model name  : QEMU Virtual CPU version (cpu64-rhel6)
    stepping    : 3
    microcode   : 0x1
    cpu MHz     : 2399.998
    cache size  : 4096 KB
    physical id : 0
    siblings    : 1
    core id     : 0
    cpu cores   : 1
    apicid      : 0
    initial apicid  : 0
    fpu     : yes
    fpu_exception   : yes
    cpuid level : 4
    wp      : yes
    flags       : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush
     mmx fxsr sse sse2 syscall nx lm nopl pni cx16 hypervisor lahf_lm
    bugs        :
    bogomips    : 4799.99
    clflush size    : 64
    cache_alignment : 64
    address sizes   : 46 bits physical, 48 bits virtual
    power management:
    

    There is no AES-NI flag as well.

  • vimalwarevimalware Member
    edited January 2018

    @danninov said:

    dediserve said: Not sure what you mean, CPU is already pass-through, you can see the model from VM, if you're talking about specific feature pass-through, OnApp does not support this.

    Nope you don't, at least not on my VPS. Here my VPS (fixed instance)

    There is no AES-NI flag as well.

    I've had it out with dediserve over this nitpick, over a year ago. They fixed it one-time too IIRC in Amsterdam pool for my instance.

    But it's a manual process and doesn't carry over when you redeploy fresh VM in your resource pool.

    But they're reliable, so I stopped complaining and carry on.

    It doesn't really affect speed of deploying new VMs. I can live with that.

    If dediserve has no worrisome load on their hypervisors, no need to worry about this:

    openssl speed -evp aes128
    Doing aes-128-cbc for 3s on 16 size blocks: 17116173 aes-128-cbc's in 2.99s
    Doing aes-128-cbc for 3s on 64 size blocks: 4971035 aes-128-cbc's in 2.99s
    Doing aes-128-cbc for 3s on 256 size blocks: 1293078 aes-128-cbc's in 3.00s
    Doing aes-128-cbc for 3s on 1024 size blocks: 693329 aes-128-cbc's in 3.00s
    Doing aes-128-cbc for 3s on 8192 size blocks: 88834 aes-128-cbc's in 2.99s
    Doing aes-128-cbc for 3s on 16384 size blocks: 44312 aes-128-cbc's in 3.00s
    OpenSSL 1.1.0g  2 Nov 2017
    built on: reproducible build, date unspecified
    options:bn(64,64) rc4(16x,int) des(int) aes(partial) blowfish(ptr) 
    compiler: gcc -DDSO_DLFCN -DHAVE_DLFCN_H -DNDEBUG -DOPENSSL_THREADS -DOPENSSL_NO_STATIC_ENGINE -DOPENSSL_PIC -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DRC4_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPADLOCK_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/usr/lib/ssl\"" -DENGINESDIR="\"/usr/lib/x86_64-linux-gnu/engines-1.1\"" 
    The 'numbers' are in 1000s of bytes per second processed.
    type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
    aes-128-cbc      91591.56k   106403.42k   110342.66k   236656.30k   243387.33k   242002.60k
    

    Practically speaking, you're not going to be writing to diskdoing disk/network IO, at over 240MB/sec on a sustained basis.

  • KuJoeKuJoe Member, Host Rep

    I'd like it if somebody much more knowledgeable than myself could explain the pros and cons of this in more detail from a provider perspective. I'd definitely consider enabling this (or giving clients the ability to enable this) if I had more information.

  • RhysRhys Member, Host Rep
    edited January 2018

    @KuJoe said:
    I'd like it if somebody much more knowledgeable than myself could explain the pros and cons of this in more detail from a provider perspective. I'd definitely consider enabling this (or giving clients the ability to enable this) if I had more information.

    pros:
    speed
    speed
    speed
    speed

    cons:

    extreme edge case like this can happen but you can consider that abuse and ban hammer.

    Thanked by 1KuJoe
  • @psb777 said:
    Many KVM providers hesitate to passthrough host CPU features. Missing SSSE3, SSE4, AVX2, and AES-NI incur significant performance penalties only under specialized workload (e.g., encryption, although there is a hack to force it).

    Now there is an argument that the Meltdown patch makes PCID a much more important performance feature. Hopefully this will nudge providers to do CPU passthrough, but I am not holding my breath...

    https://groups.google.com/forum/m/#!topic/mechanical-sympathy/L9mHTbeQLNU

    Question is how easy is it for a SolusVM based host to enable that?

    Is it just a matter of editing the OS template for example and check a box and that enables it to all customers or does it require reboots etc?

  • ClouviderClouvider Member, Patron Provider
    edited January 2018

    @dediserve said:

    @danninov said:
    @DediServe don't allow CPU passthrough and reject my request for allowing it

    Not sure what you mean, CPU is already pass-through, you can see the model from VM, if you're talking about specific feature pass-through, OnApp does not support this.

    What version are you on ? Our supports it ?

Sign In or Register to comment.