New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
CPU fair use
ColonelPanic
Member
in General
I'm running a prohosting24 deal I picked up (4 cores/14Gb/100Gb) for personal projects - Keyhelp for hosting some websites and docker containers with various apps - Trelium. portainer, Jenkins, Redmine, etc. Logging into my netdata account, I see it's using a constant 75-85% of CPU. Am I likely to be booted?
I really like the deal I have and the service has been solid, so I can delete a few of the containers if it appears that I am abusing the service. I'm kinda surprised that these idling containers are making it work so hard TBH but htop says docker is the cpu hog. Using 10 of 14Gb of RAM too.
Comments
Why are you asking here?
Every provider have different policy. Some of them have that in TOS/AUP, some not, some have vague things like "If we deem it too much we kick you" with no hard numbers.
Ask them or if you too scared to ask them because they will boot you - take a lot of backups ( ͡° ͜ʖ ͡°) ( ͡° ͜ʖ ͡°)( ͡° ͜ʖ ͡°)( ͡° ͜ʖ ͡°)( ͡° ͜ʖ ͡°)
also because something is working "now" it does not mean it will work in 2 weeks - maybe your neighbor gonna go crazy on CPU, they will notice in monitoring, they will boot you both.
And fuck me, 85% CPU of 4 cores on a shared? People on your node must love you.
I mean, if you're hitting 75-85% at all times then yeah, that's excessive.
Most providers aren't selling you dedicated cores unless it's specifically stated somewhere.
Definitely contact your provider. Maybe they already throttle you to your fair share allocation after this long period of high CPU use and you can "burst 100%" of your use without worry.
or.... use
CPUQuotawith systemdThanks for the responses, most of you. I will start shutting down containers looking for the culprit - I was expecting the node to be idling
@JabJab why do you go out of your way to post vitriol? Are you OK?
I'm not sure the prohosting stats are accurate tbh. Considering my vps with them would regularly lose connectivity or just shutdown randomly, I'd avoid and move to a better host. Their BF deal was great but too good to be true imo. I've moved all of my stuff off.
I went with a dedi, from reliable site plus storage at servarica instead
is the 'constant 75-85% of CPU' what htop is reporting?
is this on all cpus? on a particular cpu? or an average of all cpus?
I ask because htop has lots of different cpu reporting modes.
I suggest you try playing with them a bunch. (? or h for help, F2, C, or S to setup top chart displays.)
you might also want to try the 'merged' view (m), the tree view (F5), or viewing user or kernel threads (H / K) to really drill down on what's tanking your performance.
obviously, sorting by cpu is key here.
also, what does the load average look like?
also, try using ctop. it gives easy it interpret container stats, similar to htop.
( try docker run --rm -ti --name ctop -v /var/run/docker.sock:/var/run/docker.sock quay.io/vektorlab/ctop:latest )
( also, google ctop first, and check the repo source. don't run random things people tell you to without knowing what they are.
that should give you a LOT more info, and some insight.
Isn't scheduler supposed to take care of "balancing cores" across vm's?
It wont be visible for user perspective(at least without benchamarking) but the hypervisor will be taking care of that.
I never understood "FUP" with VPS'es.
/shill hat on
Funny enough, not too long ago I went through the same thought process as we kept getting asked our core usage policy and related types of questions in private. I never understood how people were 'abusing' CPU cores on KVM. I kept thinking it was tied to some sort of exploit where they were somehow able to get access to 6 cores while paying for 2 or something along those lines.
Not it--it's simply down to overselling. My confusion was basically I couldn't understand the same re: FUP with a VPS. It turns out that because our plans are all allocated their resources which are based around CPU cores available that it just wouldn't be an issue. Of course this means there is a limit as to 'how cheap' we can go and if you were to split the cores further and give a 'fair use' AUP you could then achieve that tier of pricing.
/shill hat off
This isn't to say that overselling is bad. It absolutely isn't. I can say based on usage metrics: we could probably oversell every node by a factor of 3-4 and not a single person would be impacted. It just comes down (at least for me) on how we want to manage things going forward, and at scale the human management aspect becomes unrealistic for us so I'd rather just make it impossible to matter. If you look at certain well-liked/proven providers here you'll see how they allocate cores (in a very realistic manner) which backs this up. The flip-side to that is its impossible to offer $15/yr plans and stick to that sort of deployment of resources. Given modern processors and efficiencies, most use cases don't even need a dedicated thread/vcpu so I can see why there is a very strong market for shared resources and having a layer of management/monitoring to ensure people are handling fair use policies.
@grooveuser thanks for the ctop advice. Looking closer, it seems like netdata.cloud is wildly off. Checking htop again, I'm running at idle as I would expect - I must have checked it during a spike. Netdata says it's constantly running at 79%
I get odd usage stats for prohosting24 too, mine is completely idle and the control panel says it's using 8gb memory and 30% CPU, which it definitely isn't. Haven't really used it so not sure of the stability but it was down for 24 hours on one of the rare occasions I did try to login. Does indeed seem like the BF offer was too good to be true, will probably not continue it once my credit runs out
Even with that setup, unless you're locking each user to specific cores, it's possible for some users to give other users a terrible experience.
If you have 8 cores, 16 threads, and you sell a single vCPU to 16 users, if 9 of them max out "their dedicated vCPU" then you'll start to see steal between them, so even then you need to have some kind of FUP. In the worst case, if you had a user with 8 vCPUs maxing them out, they'd probably get scheduled to a different core each and potentially nobody else on that host could do anything useful at all as their mostly idle cores would be scheduled on the threads with the busy ones.
Max at server-factory had the fairest setup, although I think he's now moved away from it. If you bought a 2 vCPU VPS, you got two threads locked to one core, so you could do what you wanted with it and it wouldn't affect anyone else (apart from memory and IO bandwidth, I guess). If you just ordered 1 vCPU, you got a single thread locked to one core and timesliced to maximum 50%. This also meant that you could max it out and you "probably" wouldn't affect anyone else.
Obviously, the reason people oversell is that most of the time, most people's VPS are pretty idle.
If I was a VPS provider, personally I'd consider a hybrid where maybe half the cores are in a common pool, and the other half are dedicated to each client, so you might have e.g. 2 threads locked to a dedicated core and 4 threads that could float around the other 4 cores and everyone got to share them. Effectively a 8 core, 16 thread machine could support 4 clients with 6 vCPU each and each user would be guaranteed at least 1 core / 2 threads no matter what anyone else was doing, but could potentially get up to 5 cores of throughput if the rest of the machine was idle.
if this host is allowing single tenant using 75-85% CPU constantly, then its time avoid such host. thanks for starting this thread.
You're making some incorrect assumptions here. The first being that every single vCPU on a hypervisor is sold. We do actually set them up so there is a 'common pool' (which remains unassigned/unsold) to cover float, if necessary. You can also get creative with how you place different users(resources) on hypervisors to further help this situation as well--such as not placing an 8 vCPU user on an 8 core CPU. The short of it is that you can still assign VPSes in a manner that gives you an extremely high degree of certainty that nobody will run into your theoretical limits, or other resource/abuse issues would show first. There is still a general AUP around abuse, but that is mostly for disk IO as you mentioned.
The rest of what you're talking about gets into VDS versus VPS territory which generally isn't popular in the LET world from what I've seen, probably due to the costs of truly locking those resources up. I also thought locking threads to cores is usually avoided in case you need to migrate a user to a different machine (maybe even different physical hardware like Intel to AMD hypervisor). You end up losing a lot of the functionality of VPS (at least on the provider's end) by doing so without gaining the benefits of bare metal.
Your hypothetical VPS provider setup would actually be interesting to see some results on. I would normally have assumed locking threads/vcpu to physical cores would be better but with modern CPUs having wider base/boost clocks and now the P&E cores with Intel I wonder if the linux scheduler/cpu is more efficient at deciding what runs where and when.
FAOD, I wasn't making assumptions about your services as I've never used them, just what most of the lowest of the low end seem to do.
That's true. VDS is generally more expensive because can't be oversold.
Yeah, I've never operated as a provider, so I don't know about that. I use libvirt directly on my dedis, and I have proxmox on my home router and so as I've only got the one instance, I've not tried a live migration.
It's essentially how I have my dedis set up, although it's a bit more hybrid still as I don't want my haproxy to ever run on a shared pool, so that's just 2 dedicated threads on the same core. My two webapps are set up in this hybrid way, but everything else just runs on the pool as I don't really care about latency on those as they're only used for CI in the background.
Yeah, I wondered about those too. Linux always used to try to use cores in numerical order, hence why 0...n-1 and n...2n-1 are pairs. It'd be interesting to know whether on linux the performance cores are always lower numbers and so the efficiency cores are only used under high load, or the opposite, which might mean if you only have 4 threads at max CPU they might never run on performance cores, or whether they are dynamically renumbered depending on load. And if the latter, what happens to pinned cores. Or of course, maybe they just changed the scheduling strategy and left the numbers alone.
Shared CPU plan is such a thing of the past. Just use VM's with dedicated vCores (aka threads) or just rent dedicated servers.
Can't imagine being worried about CPU utilization. . .
Such issues shouldn't be a problem from the customer side to be solved. It should be handled by the provider by capping / setting a limit in the hypervisor.
Provider sells shared CPU for cheap and expects user to follow fair use, and refuses to define what's the acceptable usage.
Provider can then suspend a user whenever they want for CPU abuse.
User is living in fear under the ban hammer.
@yoursunny exactly.
Just buy a shared hosting from a provider that is known for quality. Make sure they use cloudflinux and cagefs. Ask first what the LVE limits are. Minimum CPU 100%. 200% even better. Thank me later.
My services depend on:
repreproNone of them would work in shared hosting.
When I received @visualwebtechnologies DirectAdmin account in a raffle, I gave it away.
@Francisco 's tally script will punish you.(or the other script in the buyshared directory)