Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


NetCup Performance Issue - Page 5
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

NetCup Performance Issue

1235»

Comments

  • edited June 1

    @itsdeadjim said:
    I dont remember if qemu can have access to the executed instructions when vmx are used, but maybe it's better this way, as you could monitor kvm process itself and stay out from qemu GPL codebase, if you are seriously thinking about it.

    Since the host has access to the guests memory it's all a matter of figuring out RIP.

    If you open the door to something like that, nothing stops you to scan memory in general and access unencrypted network traffic, too. But then again all this opens many questions about privacy.

    I wouldn't think so. Network traffic (encrypted or not) is essentially public knowledge (any hop on the way to the destination sees it) anyways and scanners are not a new thing at all. Any mail setup running a virus scanner against the incoming attachments or really any kind of spam filter are doing essentially the same thing just in different surroundings.

    @itsdeadjim said:

    @siemens said:

    @totally_not_banned said: swimming in cash part could be a little problematic due to Qemu's GPL status though

    There is nothing preventing you from selling GPL code, you just have to give the source to the people you sold it to, afaik.

    I guess the problem with selling GPL code is that you can sell it only once :)

    Exactly. GPL code is usually monetized through support (or by fitting into the various loopholes) and i (like many programmers) don't fancy being a support person.

  • emghemgh Member

    m y i m love support person (m y i a m support person)

    Thanked by 1totally_not_banned
  • edited June 1

    @emgh said:
    m y i m love support person (m y i a m support person)

    Nothing wrong with that. I guess it's somewhat related to the fact of programmers (at least historically) often times being antisocial assholes and having no clue how marketing or support actually works ;)

    Thanked by 1emgh
  • siemenssiemens Member

    @itsdeadjim said:

    @siemens said:

    @totally_not_banned said: swimming in cash part could be a little problematic due to Qemu's GPL status though

    There is nothing preventing you from selling GPL code, you just have to give the source to the people you sold it to, afaik.

    I guess the problem with selling GPL code is that you can sell it only once :)

    You can still have terms. Look at what RedHat is doing with their Enterprise Linux.

  • edited June 1

    @siemens said:

    @itsdeadjim said:

    @siemens said:

    @totally_not_banned said: swimming in cash part could be a little problematic due to Qemu's GPL status though

    There is nothing preventing you from selling GPL code, you just have to give the source to the people you sold it to, afaik.

    I guess the problem with selling GPL code is that you can sell it only once :)

    You can still have terms. Look at what RedHat is doing with their Enterprise Linux.

    Totally different ball game. I don't really know what they are doing but you can't compare a huge aggregate, which actually has the liberty to keep parts of its project out of the GPL's reach (besides that i'm pretty sure the bulk of Redhat's revenue still comes from selling support contracts) with a single application project for which everyone can just request source and upload it to Github for the whole world to build their own binaries. Once you trigger the GPL clauses your code is basically public property. The only way around that is either not triggering those clauses (impossible if you are directly basing off of GPL'd code at least not without putting in quite a bit of extra work and probably accepting a good amount of legal uncertainty) or not release anything at all and just provide remote access (the most glaring loophole at least for non-AGPL code).

    TLDR: GPL is nice for users. For programmers... not so much.

  • emghemgh Member
    edited June 1

    @totally_not_banned said:

    @emgh said:
    m y i m love support person (m y i a m support person)

    Nothing wrong with that. I guess it's somewhat related to the fact of programmers (at least historically) often times being antisocial assholes and having no clue how marketing or support actually works ;)

    I’m not😉

    Although sometimes it feels like it with internal questions…

  • edited June 1

    @itsdeadjim https://github.com/JulesWang/qemu-deterministic-replay I didn't test the project but it seems Qemu is able to actually record the full instruction flow even in a KVM environment. If that's actually possible there should be quite some room for detecting VMs hogging memory bandwidth.

  • @emgh said:

    @totally_not_banned said:

    @emgh said:
    m y i m love support person (m y i a m support person)

    Nothing wrong with that. I guess it's somewhat related to the fact of programmers (at least historically) often times being antisocial assholes and having no clue how marketing or support actually works ;)

    I’m not😉

    Although sometimes it feels like it with internal questions…

    Well, i've become a bit more tolerable over the years too but support is essentially a skillset of it's own. Just because you theoretically know the answer doesn't mean you'll be any good at supporting people.

  • emghemgh Member

    @totally_not_banned said: Just because you theoretically know the answer doesn't mean you'll be any good at supporting people.

    Or even want to B)

    Thanked by 1totally_not_banned
  • @totally_not_banned said:
    @itsdeadjim https://github.com/JulesWang/qemu-deterministic-replay I didn't test the project but it seems Qemu is able to actually record the full instruction flow even in a KVM environment. If that's actually possible there should be quite some room for detecting VMs hogging memory bandwidth.

    Interesting, but have a look at 722 and 406 https://github.com/qemu/qemu/commit/bf3c6a0fe0543143834a9dd0f1c5f9429975e545 the main logic is around setting VM in debugging mode and exiting after executed instruction.

  • edited June 1

    @itsdeadjim said:

    @totally_not_banned said:
    @itsdeadjim https://github.com/JulesWang/qemu-deterministic-replay I didn't test the project but it seems Qemu is able to actually record the full instruction flow even in a KVM environment. If that's actually possible there should be quite some room for detecting VMs hogging memory bandwidth.

    Interesting, but have a look at 722 and 406 https://github.com/qemu/qemu/commit/bf3c6a0fe0543143834a9dd0f1c5f9429975e545 the main logic is around setting VM in debugging mode and exiting after executed instruction.

    Ouch, so chances are it'll probably have about the performance of software emulation, even if i don't see why anyone would put in the effort for this and not just run software emulation to begin with.

    Still good to know that this can be dynamically switched on and off. Even if the the performance is not sufficient with it enabled a short break should be enough to fetch RIP/register contents after which one could pretty much go by memory contents. In theory one could even just copy the memory and simultaneously run it through software emulation to figure out what's going on.

  • vikiahm3dvikiahm3d Member

    Guess the summary of this entire conversion is.

    FU**K Miners. Don't let the twats ruin your day and rest of the customers.

    Thanked by 1Mastodont
  • vikiahm3dvikiahm3d Member

    Thankfully the performance issue seems to have resolved still some woocommerce shop pages and checkout is slow but not the same as it used to be.

    Someone in netcup community mentioned they have some templates which over time throttle the traffic and it's normal not to see the same performance. Don't know what's that about but if netcup force you to sign 12 month contract then the customers must be given the service which netcup is advertising.

  • dev_vpsdev_vps Member
    edited June 1

    @vikiahm3d
    https://www.netcup.eu/bestellen/agb.php

    Terms and Conditions
    A document very few actually care to read.

    Thanked by 1vikiahm3d
  • itsdeadjimitsdeadjim Member
    edited June 1

    @totally_not_banned said: Still good to know that this can be dynamically switched on and off. Even if the the performance is not sufficient with it enabled a short break should be enough to fetch RIP/register contents after which one could pretty much go by memory contents. In theory one could even just copy the memory and simultaneously run it through software emulation to figure out what's going on.

    You can also start qemu with -s parameter. This will open a gdb listening port. Then you can use gdb directly or you can write your "scanner" to use gdb protocol to gather memory contents and VM registers for analysis. The hiccup will be only during dumping. This way you can have your scanner run on a dedicated machine for this purpose and you will completely avoid qemu source code.

    Thanked by 1totally_not_banned
  • @itsdeadjim said:

    @totally_not_banned said: Still good to know that this can be dynamically switched on and off. Even if the the performance is not sufficient with it enabled a short break should be enough to fetch RIP/register contents after which one could pretty much go by memory contents. In theory one could even just copy the memory and simultaneously run it through software emulation to figure out what's going on.

    You can also start qemu with -s parameter. This will open a gdb listening port. Then you can use gdb directly or you can write your "scanner" to use gdb protocol to gather memory contents and VM registers for analysis. The hiccup will be only during dumping. This way you can have your scanner run on a dedicated machine for this purpose and you will completely avoid qemu source code.

    I wonder how that would look in the field though. GDB is already quite annoying with mere threads but at least there's a bunch of options.

  • @totally_not_banned said:

    @itsdeadjim said:

    @totally_not_banned said: Still good to know that this can be dynamically switched on and off. Even if the the performance is not sufficient with it enabled a short break should be enough to fetch RIP/register contents after which one could pretty much go by memory contents. In theory one could even just copy the memory and simultaneously run it through software emulation to figure out what's going on.

    You can also start qemu with -s parameter. This will open a gdb listening port. Then you can use gdb directly or you can write your "scanner" to use gdb protocol to gather memory contents and VM registers for analysis. The hiccup will be only during dumping. This way you can have your scanner run on a dedicated machine for this purpose and you will completely avoid qemu source code.

    I wonder how that would look in the field though. GDB is already quite annoying with mere threads but at least there's a bunch of options.

    chatgpt seems able to write a gdb client in C that gets instruction pointer and dump a memory range from qemu through TCP in less than 50-100 lines. Give a try

  • edited June 2

    @itsdeadjim said:

    @totally_not_banned said:

    @itsdeadjim said:

    @totally_not_banned said: Still good to know that this can be dynamically switched on and off. Even if the the performance is not sufficient with it enabled a short break should be enough to fetch RIP/register contents after which one could pretty much go by memory contents. In theory one could even just copy the memory and simultaneously run it through software emulation to figure out what's going on.

    You can also start qemu with -s parameter. This will open a gdb listening port. Then you can use gdb directly or you can write your "scanner" to use gdb protocol to gather memory contents and VM registers for analysis. The hiccup will be only during dumping. This way you can have your scanner run on a dedicated machine for this purpose and you will completely avoid qemu source code.

    I wonder how that would look in the field though. GDB is already quite annoying with mere threads but at least there's a bunch of options.

    chatgpt seems able to write a gdb client in C that gets instruction pointer and dump a memory range from qemu through TCP in less than 50-100 lines. Give a try

    Nah, i have more than enough no-money-projects already and i kind of hate code written by chatgpt. I can write my own C if needed instead of trying to figure out what kind of subtle errors it burried somewhere. C is not a language where you want to gamble on correctness or code quality and i have seen what it did to something as simple as shell script.

    I've been doing C for around 20 years now, programming in general pretty much 30. As much as i hate it, the excitement is long gone. Earlier i saw such stuff as a challenge but these days i pretty much know that if it's possible i can do it. It's all just a question of how much time you throw at it (and shit, this stuff takes time...), so unless it's something that benefits myself or makes me money i tend to hardly go beyond the theoretical level now. Life's too short to pump weeks and months into random stuff and it's really just the same crosswords puzzle over and over again, only with a different layout.

  • jsgjsg Member, Resident Benchmarker
    edited June 2

    @Falzo said:
    @TimboJones vs @jsg - I just can't tell who is who 🤷🏻‍♂️

    Huh? I wasn't even here for quite a while, let alone in this thread. Plus, there is no "[xyz] vs jsg" because I've given up any hope that that asshole ever will reach homo sapiens status ~ meeting minimal standards for a discussion.

    As for the topic: I didn't notice any significant problems with my netcup root server but that might be due to it being somewhat older (than gen. 11).

    From what I read here the problem seems to be miners, which makes sense because not only do they fully use the processor but, to make it worse, their workload is extremely one-sided (processor-heavy), i.e. not what most virtual dedis are designed for.
    IMO the solution is very, very simple: Put something to the effect of "mining not acceptable use; doing it leads to immediate termination without refund" into TOS/AUP and be done.

    Btw, I'm generally for prohibiting mining on virtual servers, no matter whether dedicated or shared.

    Thanked by 1Falzo
  • AdvinAdvin Member, Patron Provider

    As a side note, I've noticed that the load average is far higher with nodes that have Quilibrium VMs on them. I'm still not sure as to why this is.

  • FranciscoFrancisco Top Host, Host Rep, Veteran

    @Advin said:
    As a side note, I've noticed that the load average is far higher with nodes that have Quilibrium VMs on them. I'm still not sure as to why this is.

    How cache hungry is Quilbrium? Maybe it's murdering the L2/L3 cache making other users suffer.

    Or AVX?

    Francisco

    Thanked by 1totally_not_banned
  • AdvinAdvin Member, Patron Provider

    @Francisco said:

    @Advin said:
    As a side note, I've noticed that the load average is far higher with nodes that have Quilibrium VMs on them. I'm still not sure as to why this is.

    How cache hungry is Quilbrium? Maybe it's murdering the L2/L3 cache making other users suffer.

    Or AVX?

    Francisco

    It really does seem like it's something like that, but I'm not sure.

    The CPU usage was never a problem for us, even on full tilt we can handle the load. Problem was that for some reason it was causing load average to spike heavily and other VMs were seeing a lot of CPU steal despite CPU resources remaining on each node.

  • dev_vpsdev_vps Member
    edited June 4

    chatgpt also says that heavy cache usage can cause cpu steal on other VPS on the host server

    This Quilbrium is disturbing the equilibrium.

  • cybertechcybertech Member
    edited June 4

    from this the conclusion is very clear: only the CPU thread can be dedicated, whereas the cache cant. its kinda like RAM speed and IOPS.

    so just advertise accordingly.

    2 vCores (dedicated)
    16kb CPU cache dedicated or 2M shared fair use
    2GB ram dedicated (but bus speed is shared)
    20GB NVMe dedicated (but IOPS shared fair use)

    Thanked by 1totally_not_banned
  • labzelabze Member, Patron Provider

    @Advin said:

    @Francisco said:

    @Advin said:
    As a side note, I've noticed that the load average is far higher with nodes that have Quilibrium VMs on them. I'm still not sure as to why this is.

    How cache hungry is Quilbrium? Maybe it's murdering the L2/L3 cache making other users suffer.

    Or AVX?

    Francisco

    It really does seem like it's something like that, but I'm not sure.

    The CPU usage was never a problem for us, even on full tilt we can handle the load. Problem was that for some reason it was causing load average to spike heavily and other VMs were seeing a lot of CPU steal despite CPU resources remaining on each node.

    I've had the same issue on EPYC nodes. Had CPU usage of around 25%, low IO usage, but load went above 700. I suspended a handful of a clients VMs and it returned to normal.

    Thanked by 1emgh
  • emghemgh Member

    @labze said:

    @Advin said:

    @Francisco said:

    @Advin said:
    As a side note, I've noticed that the load average is far higher with nodes that have Quilibrium VMs on them. I'm still not sure as to why this is.

    How cache hungry is Quilbrium? Maybe it's murdering the L2/L3 cache making other users suffer.

    Or AVX?

    Francisco

    It really does seem like it's something like that, but I'm not sure.

    The CPU usage was never a problem for us, even on full tilt we can handle the load. Problem was that for some reason it was causing load average to spike heavily and other VMs were seeing a lot of CPU steal despite CPU resources remaining on each node.

    I've had the same issue on EPYC nodes. Had CPU usage of around 25%, low IO usage, but load went above 700. I suspended a handful of a clients VMs and it returned to normal.

    @davide So not just an issue of CPU allocation

  • CalinCalin Member, Patron Provider
    edited June 4

    @labze said: I've had the same issue on EPYC nodes. Had CPU usage of around 25%, low IO usage, but load went above 700. I suspended a handful of a clients VMs and it returned to normal.

    >

    Same thing here , need lot of cache

    (Idk why but at 400-450 GBs KSM Sharing memory) node go crash , we still investigate and make more tests :)

    Regards

  • stxshstxsh Member

    This seems to have gone back to normal for me.

    Original GB6: 2034/6607
    May 31st GB6: 920/2146
    Today's GB6: 2004/6442

    Thanked by 1maverick
  • MoopahMoopah Member

    @stxsh said:
    This seems to have gone back to normal for me.

    Original GB6: 2034/6607
    May 31st GB6: 920/2146
    Today's GB6: 2004/6442

    Let it be known on this day that Netcup has returned to Equilibrium

Sign In or Register to comment.