Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Suspended again for high CPU usage

2

Comments

  • MurvMurv Member, Megathread Squad

    @sillycat said:
    suspended for gooning

    it's so over

    image

  • OP, without knowing your use case, recommendations don't mean much.

  • @zed said:
    have you considered just not doing that

    It was an accident, as I said

  • You can get dedicated CPU at
    https://my.hosteons.com/store/hybrid-dedicated-server-special

    2 cpus, 8GB ram, 100GB disk, 30TB bandwidth for $8.99
    3 cpus, 12GB ram, 150GB disk, 30TB bandwidth for $11.99

    It's Ryzen 3950x cpus, dedicated so you should be fine.

  • Even a VDS can be suspended for cpu abuse. Be careful. For a detailed explanation of the differences between a VDS and a VPS, please ask @YassineB https://lowendtalk.com/discussion/comment/4891797/#Comment_4891797

    Thanked by 1itachikonoha
  • jadenjaden Member

    If you consider $10/month a big jump, maybe you shouldn't be running a free API with billions of requests. What if you limited requests to keep riffraff like that averagedatahoarder user from costing you too much ;)

  • @jaden said: free API with billions of requests

    Steward

    Thanked by 1averagedatahoarder
  • TangeTange Member

    you have a billion API calls a month, and you are not willing to rent a dedicated server for it?

    Thanked by 1HostBilby
  • netcup root server?

  • sumosumo Member

    @SKRIME said:

    @sumo no, the 200% was just an example value for capping that one logging service, it has nothing to do with our rules. The fair use is about a server running at 100% permanently, and in practice that's the only case where we step in. The cap is only there so a single buggy process can't take the whole server with it, so pick whatever value fits your setup.

    Thanks, good to know. Just a comment, at the right price point, 50% sustained isn’t horrible

  • @Tange said:
    you have a billion API calls a month, and you are not willing to rent a dedicated server for it?

    No, my API powers many popular free consumer apps. They link back to me, but most users donate to the app instead of me (the one with infrastructure costs) - many of the developers of the apps take credit. I’m in the process of updating my policies to make attribution required so this can be funded but of course it will take a while for all the apps to implement this

    Thanked by 1rpqu
  • @averagedatahoarder said:

    @Tange said:
    you have a billion API calls a month, and you are not willing to rent a dedicated server for it?

    No, my API powers many popular free consumer apps. They link back to me, but most users donate to the app instead of me (the one with infrastructure costs) - many of the developers of the apps take credit. I’m in the process of updating my policies to make attribution required so this can be funded but of course it will take a while for all the apps to implement this

    I want to be developer on one of those apps. Where do I apply.

  • How does the docker ram limit flag differ from the shim (ram) flag?

  • @itachikonoha said:

    @averagedatahoarder said:

    @Tange said:
    you have a billion API calls a month, and you are not willing to rent a dedicated server for it?

    No, my API powers many popular free consumer apps. They link back to me, but most users donate to the app instead of me (the one with infrastructure costs) - many of the developers of the apps take credit. I’m in the process of updating my policies to make attribution required so this can be funded but of course it will take a while for all the apps to implement this

    I want to be developer on one of those apps. Where do I apply.

    Not another vibecoded API wrapper 😭

  • @averagedatahoarder said:

    @Tange said:
    you have a billion API calls a month, and you are not willing to rent a dedicated server for it?

    No, my API powers many popular free consumer apps. They link back to me, but most users donate to the app instead of me (the one with infrastructure costs) - many of the developers of the apps take credit. I’m in the process of updating my policies to make attribution required so this can be funded but of course it will take a while for all the apps to implement this

    A bit more on this.

    I started the API as a small hobby project, I mainly intended for users to access the service via my webapp. Then, the developer of a popular app found my API and began using it in their app -- the first app developer was very nice, credited me, etc. But then all the other players found my API and began hammering me with millions of requests under spoofed useragents, etc.

    And finally, the main alternative to my service (which was already quite popular) "merged" with me - in other words, they shut down and redirected all traffic to me.

    I don't think this is a bad thing, I'm happy to provide this service and enjoy maintaining it, I just never set this up properly to sustain this level of traffic. And it only seems like traffic is going to increase from now on, every day I check the analytics and new apps are wrapping my API...

  • LEBUserJoeLEBUserJoe Member
    edited October 20

    @averagedatahoarder said:

    @yoursunny said:

    @averagedatahoarder said:
    This is a free API I host with over a billion requests a month

    Please show API documentation, source code, and deployment instructions (exact command to deploy the software).
    We can then audit them and give suggestions on how to prevent excessive CPU usage again.

    You really willing to do this?

    For record, fulltime developer here need any help just PM me happy too.

    Sounds less like you need solution than either a more solid application, or a firm CPU limit % set so you can't go over. During peak requests slow down a little but you dont get suspended.

    • I'm currently stuck with family in another country so will happily do some free work especially if you also provide the service for free.
    Thanked by 1averagedatahoarder
  • @LEBUserJoe said:

    @averagedatahoarder said:

    @yoursunny said:

    @averagedatahoarder said:
    This is a free API I host with over a billion requests a month

    Please show API documentation, source code, and deployment instructions (exact command to deploy the software).
    We can then audit them and give suggestions on how to prevent excessive CPU usage again.

    You really willing to do this?

    For record, fulltime developer here need any help just PM me happy too.

    Sounds less like you need solution than either a more solid application, or a firm CPU limit % set so you can't go over. During peak requests slow down a little but you dont get suspended.

    Thanks, I'll message you in a bit.

    I've already done everything possible I know of to optimize my endpoint .. issue is at 500 reqs/sec my VPS simply can't handle the load. At one point I even started dropping requests to keep the service running for web users.

    At this point, I think what's really needed is a better server. I suppose I'll just cough up the money, or work out a solution with some of the downstream apps.

  • @averagedatahoarder said:

    @LEBUserJoe said:

    @averagedatahoarder said:

    @yoursunny said:

    @averagedatahoarder said:
    This is a free API I host with over a billion requests a month

    Please show API documentation, source code, and deployment instructions (exact command to deploy the software).
    We can then audit them and give suggestions on how to prevent excessive CPU usage again.

    You really willing to do this?

    For record, fulltime developer here need any help just PM me happy too.

    Sounds less like you need solution than either a more solid application, or a firm CPU limit % set so you can't go over. During peak requests slow down a little but you dont get suspended.

    Thanks, I'll message you in a bit.

    I've already done everything possible I know of to optimize my endpoint .. issue is at 500 reqs/sec my VPS simply can't handle the load. At one point I even started dropping requests to keep the service running for web users.

    At this point, I think what's really needed is a better server. I suppose I'll just cough up the money, or work out a solution with some of the downstream apps.

    I'd be very curious if I could help so if you get time please do, I also a big fan of running heavy-used apps on the lowest cheapest hardware wouldnt want you to give up the dream :)

    Thanked by 1averagedatahoarder
  • rpqurpqu Member

    @averagedatahoarder said:

    @LEBUserJoe said:

    @averagedatahoarder said:

    @yoursunny said:

    @averagedatahoarder said:
    This is a free API I host with over a billion requests a month

    Please show API documentation, source code, and deployment instructions (exact command to deploy the software).
    We can then audit them and give suggestions on how to prevent excessive CPU usage again.

    You really willing to do this?

    For record, fulltime developer here need any help just PM me happy too.

    Sounds less like you need solution than either a more solid application, or a firm CPU limit % set so you can't go over. During peak requests slow down a little but you dont get suspended.

    Thanks, I'll message you in a bit.

    I've already done everything possible I know of to optimize my endpoint .. issue is at 500 reqs/sec my VPS simply can't handle the load. At one point I even started dropping requests to keep the service running for web users.

    At this point, I think what's really needed is a better server. I suppose I'll just cough up the money, or work out a solution with some of the downstream apps.

    Whether it's doable under a single host depends on what kind of stuff you do. Is it stateless?

  • @rpqu said:

    @averagedatahoarder said:

    @LEBUserJoe said:

    @averagedatahoarder said:

    @yoursunny said:

    @averagedatahoarder said:
    This is a free API I host with over a billion requests a month

    Please show API documentation, source code, and deployment instructions (exact command to deploy the software).
    We can then audit them and give suggestions on how to prevent excessive CPU usage again.

    You really willing to do this?

    For record, fulltime developer here need any help just PM me happy too.

    Sounds less like you need solution than either a more solid application, or a firm CPU limit % set so you can't go over. During peak requests slow down a little but you dont get suspended.

    Thanks, I'll message you in a bit.

    I've already done everything possible I know of to optimize my endpoint .. issue is at 500 reqs/sec my VPS simply can't handle the load. At one point I even started dropping requests to keep the service running for web users.

    At this point, I think what's really needed is a better server. I suppose I'll just cough up the money, or work out a solution with some of the downstream apps.

    Whether it's doable under a single host depends on what kind of stuff you do. Is it stateless?

    Mostly stateless

  • rpqurpqu Member

    @averagedatahoarder said:

    @rpqu said:

    @averagedatahoarder said:

    @LEBUserJoe said:

    @averagedatahoarder said:

    @yoursunny said:

    @averagedatahoarder said:
    This is a free API I host with over a billion requests a month

    Please show API documentation, source code, and deployment instructions (exact command to deploy the software).
    We can then audit them and give suggestions on how to prevent excessive CPU usage again.

    You really willing to do this?

    For record, fulltime developer here need any help just PM me happy too.

    Sounds less like you need solution than either a more solid application, or a firm CPU limit % set so you can't go over. During peak requests slow down a little but you dont get suspended.

    Thanks, I'll message you in a bit.

    I've already done everything possible I know of to optimize my endpoint .. issue is at 500 reqs/sec my VPS simply can't handle the load. At one point I even started dropping requests to keep the service running for web users.

    At this point, I think what's really needed is a better server. I suppose I'll just cough up the money, or work out a solution with some of the downstream apps.

    Whether it's doable under a single host depends on what kind of stuff you do. Is it stateless?

    Mostly stateless

    Well, that means you can't just add A records from $7/y vps. Gotta do proper cluster

    Thanked by 2tentor yoursunny
  • @rpqu said:

    @averagedatahoarder said:

    @rpqu said:

    @averagedatahoarder said:

    @LEBUserJoe said:

    @averagedatahoarder said:

    @yoursunny said:

    @averagedatahoarder said:
    This is a free API I host with over a billion requests a month

    Please show API documentation, source code, and deployment instructions (exact command to deploy the software).
    We can then audit them and give suggestions on how to prevent excessive CPU usage again.

    You really willing to do this?

    For record, fulltime developer here need any help just PM me happy too.

    Sounds less like you need solution than either a more solid application, or a firm CPU limit % set so you can't go over. During peak requests slow down a little but you dont get suspended.

    Thanks, I'll message you in a bit.

    I've already done everything possible I know of to optimize my endpoint .. issue is at 500 reqs/sec my VPS simply can't handle the load. At one point I even started dropping requests to keep the service running for web users.

    At this point, I think what's really needed is a better server. I suppose I'll just cough up the money, or work out a solution with some of the downstream apps.

    Whether it's doable under a single host depends on what kind of stuff you do. Is it stateless?

    Mostly stateless

    Well, that means you can't just add A records from $7/y vps. Gotta do proper cluster

  • rpqurpqu Member

    @averagedatahoarder said:

    @rpqu said:

    @averagedatahoarder said:

    @rpqu said:

    @averagedatahoarder said:

    @LEBUserJoe said:

    @averagedatahoarder said:

    @yoursunny said:

    @averagedatahoarder said:
    This is a free API I host with over a billion requests a month

    Please show API documentation, source code, and deployment instructions (exact command to deploy the software).
    We can then audit them and give suggestions on how to prevent excessive CPU usage again.

    You really willing to do this?

    For record, fulltime developer here need any help just PM me happy too.

    Sounds less like you need solution than either a more solid application, or a firm CPU limit % set so you can't go over. During peak requests slow down a little but you dont get suspended.

    Thanks, I'll message you in a bit.

    I've already done everything possible I know of to optimize my endpoint .. issue is at 500 reqs/sec my VPS simply can't handle the load. At one point I even started dropping requests to keep the service running for web users.

    At this point, I think what's really needed is a better server. I suppose I'll just cough up the money, or work out a solution with some of the downstream apps.

    Whether it's doable under a single host depends on what kind of stuff you do. Is it stateless?

    Mostly stateless

    Well, that means you can't just add A records from $7/y vps. Gotta do proper cluster

    Dude, I've been there. At uni, me and my friend ran a $15/m cluster. Our competitor rack up a $200-300/m GCP bills.

    Thanked by 1averagedatahoarder
  • @rpqu said:

    @averagedatahoarder said:

    @rpqu said:

    @averagedatahoarder said:

    @rpqu said:

    @averagedatahoarder said:

    @LEBUserJoe said:

    @averagedatahoarder said:

    @yoursunny said:

    @averagedatahoarder said:
    This is a free API I host with over a billion requests a month

    Please show API documentation, source code, and deployment instructions (exact command to deploy the software).
    We can then audit them and give suggestions on how to prevent excessive CPU usage again.

    You really willing to do this?

    For record, fulltime developer here need any help just PM me happy too.

    Sounds less like you need solution than either a more solid application, or a firm CPU limit % set so you can't go over. During peak requests slow down a little but you dont get suspended.

    Thanks, I'll message you in a bit.

    I've already done everything possible I know of to optimize my endpoint .. issue is at 500 reqs/sec my VPS simply can't handle the load. At one point I even started dropping requests to keep the service running for web users.

    At this point, I think what's really needed is a better server. I suppose I'll just cough up the money, or work out a solution with some of the downstream apps.

    Whether it's doable under a single host depends on what kind of stuff you do. Is it stateless?

    Mostly stateless

    Well, that means you can't just add A records from $7/y vps. Gotta do proper cluster

    Dude, I've been there. At uni, me and my friend ran a $15/m cluster. Our competitor rack up a $200-300/m GCP bills.

    I can't even begin to fathom what my spend on GCP would be for this service

  • rpqurpqu Member

    @averagedatahoarder said:
    I can't even begin to fathom what my spend on GCP would be for this service

    For your convenience bro

    Assuming Linux, on-demand, running 24/7 (730 hours/month), US regions, and a 100 GB standard SSD disk, budget roughly:

    • AWS: about $129/month — t3.xlarge compute is $121.47/month, plus about $8 for 100 GB of gp3 storage. T3 is burstable; sustained CPU use can add credit charges.

      • Google Cloud: about $108/month — e2-standard-4 compute is about $97.84/month in us-central1, plus about $10 for 100 GB of balanced persistent disk. An n2-standard-4 instead would bring the estimate to about $152/month with that disk.
      • Azure: about $135/month — D4as v5 compute at the listed $0.172/hour is about $125.56/month, plus about $9.60 for a 128 GiB Standard SSD disk (the closest standard disk size to 100 GB).
      • These are estimates, not exact like-for-like quotes: region and VM family affect prices, and Azure disk sizes are offered in set tiers. They exclude outbound data transfer, backups, taxes, and other services, which can increase the bill.
    Thanked by 1averagedatahoarder
  • JabJabJabJab Veteran

    @rpqu said: They exclude outbound data transfer, backups, taxes, and other services, which can increase the bill.

    Very helpful, thanks.

  • @rpqu said:

    @averagedatahoarder said:
    I can't even begin to fathom what my spend on GCP would be for this service

    For your convenience bro

    Assuming Linux, on-demand, running 24/7 (730 hours/month), US regions, and a 100 GB standard SSD disk, budget roughly:

    • AWS: about $129/month — t3.xlarge compute is $121.47/month, plus about $8 for 100 GB of gp3 storage. T3 is burstable; sustained CPU use can add credit charges.

      • Google Cloud: about $108/month — e2-standard-4 compute is about $97.84/month in us-central1, plus about $10 for 100 GB of balanced persistent disk. An n2-standard-4 instead would bring the estimate to about $152/month with that disk.
      • Azure: about $135/month — D4as v5 compute at the listed $0.172/hour is about $125.56/month, plus about $9.60 for a 128 GiB Standard SSD disk (the closest standard disk size to 100 GB).
      • These are estimates, not exact like-for-like quotes: region and VM family affect prices, and Azure disk sizes are offered in set tiers. They exclude outbound data transfer, backups, taxes, and other services, which can increase the bill.

    And 10 Gbps (3.2PB) is only $150k a month extra (if I counted the zeros right).

  • @JabJab said:

    @rpqu said: They exclude outbound data transfer, backups, taxes, and other services, which can increase the bill.

    Very helpful, thanks.

    Outbound data transfer 10TB/mo @ $11/TB egress = $1100/mo.

    Backups - assuming 100GB/day = 3TB/mo = $33/mo egress, $60/mo storage for 3TB

    Taxes - who needs to pay those these days?

  • Petition to start a GoFundMe for @averagedatahoarder so they can get a dedicated server and leave all these poor VPS hosts alone.

  • @Dupondt said:
    Petition to start a GoFundMe for @averagedatahoarder so they can get a dedicated server and leave all these poor VPS hosts alone.

    Petition to start a GoFundMe for @averagedatahoarder so they can keep buying the cheapest posible "10Gbps unmetered" VPS's and complaining on LowEndTalk so the MJJs can have their drama.

    Thanked by 3rpqu edideaur fishypop
Sign In or Register to comment.