Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


How to limit the resource to satisfy High Usage Policy?
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

How to limit the resource to satisfy High Usage Policy?

xiyanxiyan Member
edited December 2019 in General

As you all know , virmach has really strict High Usage Policy. I want to limit the resource positively by myself so that I won't worry about my vps being banned by virmach when unexpected things happen. First, I fully understand, we should not abuse any resource on purpose, i want to know how to limit the resource that I use automatically to avoid high usage and getting banned as a VPS user. Second, I use "virmach" as an example(sorry for any bad influence to you) and I only want to talk about things based on facts.

1 To limit High CPU, " cpulimit -l" can limit a process to use the CPU, but I want to burst to 90% within 5 five minutes when compile something. And I want to make sure CPU average usage lower than than 50% and I need the limitation still take effect after reboot. How can I manage to do that?

2 To limit High Load, usleep causes the calling thread to be suspended from execution to lower the cpu load.

3 To limit High Mail Volume, I will not send a mail.

4 To limit High I/O, I can use iostat to get IO status. I don't know how to limit it.

5 To limit High Network Usage, wondershaper limits the Bandwidth, but I want to burst to 90% within a short time. To limit conntrack sessions, set " /sbin/sysctl -w net.netfilter.nf_conntrack_max=xxxx".

I have been searching some key word and reading articles for hours, but don't have a good solution to it except changing a provider. Do you have other ways to accomplish that?

For those who are not familiar with linux, please think twice about the meaning of High Usage Policy before buying a vps with a strict High Usage Policy.

To @VirMach , you make such a strict usage policy , and you should offer a solution to make your custom feel free to enjoy your service under High Usage Policy. Not all customers want to abuse your resource, some are new to linux system and they feel fear to get banned when unexpected things lead high usage. They want an automatic tool to limit the resource they are using to get away being banned from you. You should offer a tool to let your customers to limit resource by themselves or deprioritize and limit the resource on your end to meet your requirements. Just simply power down or suspend is a kind of ridiculous. Maybe, this is a new method to earn money by claiming high usage , cancel the service to release the resource and offer a special offer to attract another victim.

Thanked by 1pluush

Comments

  • xiyanxiyan Member
    edited December 2019

    High CPU: Customer’s Service cannot burst to 95-100% usage for more than five (5) minutes and cannot average higher than 50% usage within any two (2) hour period. Packages advertised to include dedicated CPU, Services with the high CPU option, and any customized Service plans that include high CPU option may burst to 100% at all times.
    High Load: Customer’s Service cannot have a 15-minute load average higher than the number of full logical cores assigned and cannot have a 1-day load average higher than 70% of the number of full logical cores assigned.
    High Mail Volume: VirMach reserves the right to block port 25 on Customer’s Service. Customer cannot send more than 100 maximum e-mails per hour, and must maintain a similar average volume of mail on a week-to-week basis—no bursting permitted. VirMach reserves the right to waive this requirement for the purpose of a customized Service plan.
    High I/O: Customer’s Service cannot average more than 80 IOPS within any two (2) hour period, cannot burst above 300MB/s disk write average for more than ten (10) minutes, cannot average more than 300 write operations per second for more than 1 hour, and cannot be above 20% average utilization within any six (6) hour period.
    High Network Usage: Customer’s Service cannot have more than 50,000 conntrack sessions at any given time, and cannot use more than the allocated bandwidth. Customer understands that the network is shared and utilizing maximum network speed will not always be possible.

    VirMach reserves the right to deprioritize, power down, or suspend Customer’s Service(s) at any time in order to avoid congestion and/or disrupting the performance of other customers, which may directly affect the performance of Customer’s Service.

  • By the way, virmach doesn't offer alpine linux while they have a special plan with small ram, I mean the plan with 64MB RAM. Oh, they charge you extra fee if you want cuntom image.

    To those who have small ram vps, you can install alpine under rescue mode.

  • Had a problem with them about that too, server suspended 5 times over it
    The best thing is to change the service, I am really happy with the nexusbytes

    Thanked by 4seriesn tgl pluush kkrajk
  • seriesnseriesn Member
    edited December 2019

    Boss,
    The key is, don't be a dick. Keep your usage to a point where it won't impact others and don't be a noisy neighbor. You can throw parties every Friday night but don't start blasting music at 3 am on a Monday morning.

    @DeepBoom007 said:
    Had a problem with them about that too, server suspended 5 times over it
    The best thing is to change the service, I am really happy with the nexusbytes

    Hey Stranger, glad to hear you are happy :)

  • xiyanxiyan Member
    edited December 2019

    @DeepBoom007 said:
    Had a problem with them about that too, server suspended 5 times over it

    I didn't say I had a problem already, I just worry about that, I want to obey rules in a easy way. I prefer that they deprioritize and limit the resource, thus the service on server is still available.

  • jarjar Patron Provider, Top Host, Veteran
    edited December 2019

    xiyan said: To @VirMach , you make such a strict usage policy , and you should offer a solution to make your custom feel free to enjoy your service under High Usage Policy. Not all customers want to abuse your resource, some are new to linux system and they feel fear to get banned when unexpected things lead high usage. They want an automatic tool to limit the resource they are using to get away being banned from you. You should offer a tool to let your customers to limit resource by themselves or deprioritize and limit the resource on your end to meet your requirements. Just simply power down or suspend is a kind of ridiculous. Maybe, this is a new method to earn money by claiming high usage , cancel the service to release the resource and offer a special offer to attract another victim.

    This entire part of your post stems from a misunderstanding of servers, the industry, and economics as they relate. It's also a natural reaction because you're new to it, but I want to go ahead and prepare you for what's going to happen. You're going to change your perspective or stay frustrated, the industry is not going to change to accommodate your fear. You're not the first and you certainly won't be the last. If you're smart, you'll come to appreciate that the industry isn't going to change on this for it's technical and economic value.

    You see, 99 out of 100 (I made this number up) people are going to do what they want with their servers and never cause resource issues. But the only way to give you this kind of access to do what you want without charging you an incredibly higher price is to give you the ability to be the 1 out of 100 who causes resource issues. Perhaps along the way you learn that what you want to do doesn't work well with shared resources and you move to dedicated, but that doesn't mean the industry is at fault for not charging 100 out of 100 people 100x the price just because you're afraid and don't know what causes what under average scenarios.

    If they did charge 100x the price so as to not require such a strict policy, someone else would charge 100x less and include the policy, and you'd be buying from them and sharing this fear all over again because you don't want to pay 100x the price or you would have already to alleviate your fears. You should be paying 100x the price in this example with these made up numbers, by purchasing a dedicated server, if you don't know what causes what to do what on a linux system and you're afraid that server suspension/termination is going to harm important data. Us seasoned admins can do just fine under the given restrictions, and we are able to purchase a cheaper product because we're capable of doing so. You bought our cheaper product, so learn how to use it or go buy a more expensive one that is better suited to your current understanding.

  • I would say (i) know what processes are running on your VM and how they behave, no tool is a substitute for this. If you have low memory there should not be many of them! (ii) monitor yourself and stay ahead of the curve; typically you can detect an issue before VirMach does since they publish their limits fairly clearly. (iii) one of the biggest culprits is swap - this hits you on both the iops and cpu category (VirMach seem to count cpu wait as cpu usage), so either disable it or set swappiness to 0/1. (iv) check stuff like cron jobs to avoid spikes; (v) the most difficult ones are services that the public access, so you need to tune things like maximum connections and the number of workers.

    Thanked by 1xiyan
  • xiyanxiyan Member
    edited December 2019

    @jar said:

    xiyan said: To @VirMach , you make such a strict usage policy , and you should offer a solution to make your custom feel free to enjoy your service under High Usage Policy. Not all customers want to abuse your resource, some are new to linux system and they feel fear to get banned when unexpected things lead high usage. They want an automatic tool to limit the resource they are using to get away being banned from you. You should offer a tool to let your customers to limit resource by themselves or deprioritize and limit the resource on your end to meet your requirements. Just simply power down or suspend is a kind of ridiculous. Maybe, this is a new method to earn money by claiming high usage , cancel the service to release the resource and offer a special offer to attract another victim.

    This entire part of your post stems from a misunderstanding of servers, the industry, and economics as they relate. It's also a natural reaction because you're new to it, but I want to go ahead and prepare you for what's going to happen. You're going to change your perspective or stay frustrated, the industry is not going to change to accommodate your fear. You're not the first and you certainly won't be the last. If you're smart, you'll come to appreciate that the industry isn't going to change on this for it's technical and economic value.

    You see, 99 out of 100 (I made this number up) people are going to do what they want with their servers and never cause resource issues. But the only way to give you this kind of access to do what you want without charging you an incredibly higher price is to give you the ability to be the 1 out of 100 who causes resource issues. Perhaps along the way you learn that what you want to do doesn't work well with shared resources and you move to dedicated, but that doesn't mean the industry is at fault for not charging 100 out of 100 people 100x the price just because you're afraid and don't know what causes what under average scenarios.

    If they did charge 100x the price so as to not require such a strict policy, someone else would charge 100x less and include the policy, and you'd be buying from them and sharing this fear all over again because you don't want to pay 100x the price or you would have already to alleviate your fears. You should be paying 100x the price in this example with these made up numbers, by purchasing a dedicated server, if you don't know what causes what to do what on a linux system and you're afraid that server suspension/termination is going to harm important data. Us seasoned admins can do just fine under the given restrictions, and we are able to purchase a cheaper product because we're capable of doing so. You bought our cheaper product, so learn how to use it or go buy a more expensive one that is better suited to your current understanding.

    I understand your meaning as a provider and I already said the best choice is changing a provider (with dedicated service).

    My point is , the providers aim to supply stable shared service thus they have to limit abuse. Depriorityize or limit the resource is technically doable.
    CPU https://support.solusvm.com/hc/en-us/articles/360020444811--Is-there-any-option-to-limit-CPU-usage-for-KVM-virtual-server-
    Network https://doc.ispsystem.com/index.php/Limiting_VM_network_speed

    I know I can monitor the status of the server and receive an alert automatically. Just imagine there are suddenly large amount of page views during middle night, the web server will consume more resource that might result in abuse. So the monitor sends me an alert and I get up to get access to the server and locate the root cause. What I want is the monitor detect that it's about to abuse the resource , save logs and apply resource limitation command to avoid resource abuse or set a load balancer to transfer some traffic to another server. I can have a good sleep and check it without being disturbed.

    If it failed to accomplish to limit resource abuse inside the vitual server, solusvm can do it. The providers and the customers are both to managed to limit resource abuse, which will make the node even more stable.

    it seems that you might not feel happy when you know your customer wants to limit resource usage which will cause less problem to your node.

  • jarjar Patron Provider, Top Host, Veteran
    edited December 2019

    xiyan said: If it failed to accomplish to limit resource abuse inside the vitual server, solusvm can do it. The providers and the customers are both to managed to limit resource abuse, which will make the node even more stable.

    It just doesn't work that way in reality. Removing limits can actually improve performance for everyone. A process that bursts and finishes quickly is open for less time, for example. The only way you can limit it so that a single VPS can't cause an issue is to sell dedicated cores which, again, you could just buy if that's what you wanted. You don't have to tell the rest of us that this product you purchased shouldn't exist. You don't have to buy shared CPU cores.

  • @tetech said:
    I would say (i) know what processes are running on your VM and how they behave, no tool is a substitute for this. If you have low memory there should not be many of them! (ii) monitor yourself and stay ahead of the curve; typically you can detect an issue before VirMach does since they publish their limits fairly clearly. (iii) one of the biggest culprits is swap - this hits you on both the iops and cpu category (VirMach seem to count cpu wait as cpu usage), so either disable it or set swappiness to 0/1. (iv) check stuff like cron jobs to avoid spikes; (v) the most difficult ones are services that the public access, so you need to tune things like maximum connections and the number of workers.

    I am really appreciating your suggestions, thanks a lot.

  • @jar said:

    xiyan said: If it failed to accomplish to limit resource abuse inside the vitual server, solusvm can do it. The providers and the customers are both to managed to limit resource abuse, which will make the node even more stable.

    It just doesn't work that way in reality. Removing limits can actually improve performance for everyone. A process that bursts and finishes quickly is open for less time, for example. The only way you can limit it so that a single VPS can't cause an issue is to sell dedicated cores which, again, you could just buy if that's what you wanted. You don't have to tell the rest of us that this product you purchased shouldn't exist. You don't have to buy shared CPU cores.

    Got that . Thanks for your explanation.

    Thanked by 1jar
  • I should express it in a better way. It's understandable for the providers set strict High Usage Policy.

    It just doesn't work that way in reality. Removing limits can actually improve >performance for everyone. A process that bursts and finishes quickly is open for less >time, for example.

    The real question is how to avoid high resource usage.

    The reply of @tetech is what I expected.

    Thanked by 2jar Ouji
  • @xiyan said:
    I should express it in a better way. It's understandable for the providers set strict High Usage Policy.

    It just doesn't work that way in reality. Removing limits can actually improve >performance for everyone. A process that bursts and finishes quickly is open for less >time, for example.

    The real question is how to avoid high resource usage.

    The reply of @tetech is what I expected.

    You can't avoid it if you need it. There is no simple solution boss. You need to know your software, your usage and set your expectations that you.

    There is no "here is the script, do what ever you want, run this and nothing will ever happen to you" solution.

    There is a reason Facebook and google runs out of Multiple Datacenter and I run my personal blog on a vps with 128MB ram, 1 tiny cpu core and 10 GB of storage.

  • xiyanxiyan Member
    edited December 2019

    seriesn said: You can't avoid it if you need it.

    The situation is I don't need that much for average, it might burst and result in resource abuse.

    Thanked by 1pluush
  • xiyanxiyan Member
    edited December 2019

    seriesn said: I run my personal blog on a vps with 128MB ram, 1 tiny cpu core and 10 GB of storage.

    [ corner case ]
    When you are sleeping, 1000 people(or even more) are visiting your personal blog at the same time, which will result in resource abuse. You will receive the notification that says your service is terminated according to High Usage Policy.

    This is the situation that I am trying to express.

    Thanked by 2yoursunny pluush
  • Move away from virmach or shut down your blog.

  • yoursunnyyoursunny Member, IPv6 Advocate

    When my Nextcloud is at Virmach, I'm observing high CPU usage whenever my phone uploads hundreds of photos from a trip. It goes above 90% for more than 5 minutes.
    I never got hit with a High Usage Policy enforcement though.

  • @xiyan said:

    seriesn said: I run my personal blog on a vps with 128MB ram, 1 tiny cpu core and 10 GB of storage.

    [ corner case ]
    When you are sleeping, 1000 people(or even more) are visiting your personal blog at the same time, which will result in resource abuse. You will receive the notification that says your service is terminated according to High Usage Policy.

    This is the situation that I am trying to express.

    Pretty sure burst is okay.

    But then again, if you pay peanuts, something something.

    Don't overstress yourself. Do your best, don't be a dick, if it doesn't work, you can always divorce her and find another one.

    Thanked by 1Ouji
  • jarjar Patron Provider, Top Host, Veteran
    edited December 2019

    @xiyan said:

    seriesn said: I run my personal blog on a vps with 128MB ram, 1 tiny cpu core and 10 GB of storage.

    [ corner case ]
    When you are sleeping, 1000 people(or even more) are visiting your personal blog at the same time, which will result in resource abuse. You will receive the notification that says your service is terminated according to High Usage Policy.

    This is the situation that I am trying to express.

    It just doesn't really happen unless you've built the worst blog ever that can't handle a visitor without spiking a core to 100%. Also known as installing 80 WordPress plugins.

    Thanked by 1seriesn
  • make your Lowend box idle, you will passing the High Usage Policy

    don't take my words

    Thanked by 2yoursunny pluush
  • NoCommentNoComment Member
    edited December 2019

    I imagine providers are able to tell if someone is deliberately abusing resources or not especially after having many years of experience in the business. So you'll just have to hope that virmach suspends your service instead of outright terminating it when you unintentionally abuse the resources. (Which is probably what they are doing)

    Thanked by 1jar
  • @jar said:

    @xiyan said:

    seriesn said: I run my personal blog on a vps with 128MB ram, 1 tiny cpu core and 10 GB of storage.

    [ corner case ]
    When you are sleeping, 1000 people(or even more) are visiting your personal blog at the same time, which will result in resource abuse. You will receive the notification that says your service is terminated according to High Usage Policy.

    This is the situation that I am trying to express.

    It just doesn't really happen unless you've built the worst blog ever that can't handle a visitor without spiking a core to 100%. Also known as installing 80 WordPress plugins.

    Bro! What is wordpress without plugins?

    Thanked by 1jar
  • @seriesn said:

    @jar said:

    @xiyan said:

    seriesn said: I run my personal blog on a vps with 128MB ram, 1 tiny cpu core and 10 GB of storage.

    [ corner case ]
    When you are sleeping, 1000 people(or even more) are visiting your personal blog at the same time, which will result in resource abuse. You will receive the notification that says your service is terminated according to High Usage Policy.

    This is the situation that I am trying to express.

    It just doesn't really happen unless you've built the worst blog ever that can't handle a visitor without spiking a core to 100%. Also known as installing 80 WordPress plugins.

    Bro! What is wordpress without plugins?

    Word-

    Thanked by 1Kiwi83
  • jarjar Patron Provider, Top Host, Veteran

    seriesn said: Bro! What is wordpress without plugins?

    Fast and lightweight :joy:

    Thanked by 1bdl
  • @xiyan said:
    As you all know , virmach has really strict High Usage Policy. I want to limit the resource positively by myself so that I won't worry about my vps being banned by virmach when unexpected things happen. First, I fully understand, we should not abuse any resource on purpose, i want to know how to limit the resource that I use automatically to avoid high usage and getting banned as a VPS user. Second, I use "virmach" as an example(sorry for any bad influence to you) and I only want to talk about things based on facts.

    1 To limit High CPU, " cpulimit -l" can limit a process to use the CPU, but I want to burst to 90% within 5 five minutes when compile something. And I want to make sure CPU average usage lower than than 50% and I need the limitation still take effect after reboot. How can I manage to do that?

    2 To limit High Load, usleep causes the calling thread to be suspended from execution to lower the cpu load.

    3 To limit High Mail Volume, I will not send a mail.

    4 To limit High I/O, I can use iostat to get IO status. I don't know how to limit it.

    5 To limit High Network Usage, wondershaper limits the Bandwidth, but I want to burst to 90% within a short time. To limit conntrack sessions, set " /sbin/sysctl -w net.netfilter.nf_conntrack_max=xxxx".

    I have been searching some key word and reading articles for hours, but don't have a good solution to it except changing a provider. Do you have other ways to accomplish that?

    For those who are not familiar with linux, please think twice about the meaning of High Usage Policy before buying a vps with a strict High Usage Policy.

    To @VirMach , you make such a strict usage policy , and you should offer a solution to make your custom feel free to enjoy your service under High Usage Policy. Not all customers want to abuse your resource, some are new to linux system and they feel fear to get banned when unexpected things lead high usage. They want an automatic tool to limit the resource they are using to get away being banned from you. You should offer a tool to let your customers to limit resource by themselves or deprioritize and limit the resource on your end to meet your requirements. Just simply power down or suspend is a kind of ridiculous. Maybe, this is a new method to earn money by claiming high usage , cancel the service to release the resource and offer a special offer to attract another victim.

    You should not be using @VirMach if you are going to complain about/advice them, they are known for this crap, not just for 5$ per year VPS, and also for 80 USD per year VPS too. Move your VPS to some reliable providers, pay extra for quality services.

  • yoursunny said: When my Nextcloud is at Virmach, I'm observing high CPU usage whenever my phone uploads hundreds of photos from a trip. It goes above 90% for more than 5 minutes.

    Are you on a regular price plan or the xiyan shitweasel plan?

  • xiyanxiyan Member
    edited December 2019

    @gks said:
    You should not be using @VirMach if you are going to complain about/advice them, they are known for this crap, not just for 5$ per year VPS, and also for 80 USD per year VPS too. Move your VPS to some reliable providers, pay extra for quality services.

    My real question should be how to monitor a vps and locate the root cause of unexpected resource burst. I know I should not expect that much with a crazy low price.

    I made a mistake and complain about virmach since they said they cauld deprioritize and limit the resources on their policy. I thought they were too lazy to power down the vps directly when they could deprioritize it. Thanks to @jar 's explanation, I know I I had a misunderstanding. Virmach does well with such low price.

    Actually, I haven't been notified that I used too much resources. I just want to know how to avoid resource burst so that the service on vps can running more smoothly.

  • vimalwarevimalware Member
    edited December 2019

    Or just use Wishosting.
    (Automatic management of cpu.)

    Honestly, I'm happy to see more considerate users like OP. Good on you.

    Thanked by 1skorous
  • @xiyan said:
    By the way, virmach doesn't offer alpine linux while they have a special plan with small ram, I mean the plan with 64MB RAM.

    Updated. They offer alpine linux iso now.

Sign In or Register to comment.