Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Looking for a GPU Server for Tensorflow Inferencing
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Looking for a GPU Server for Tensorflow Inferencing

Hi,

I am looking to rent a GPU server long term for REST inferencing using flask.

My current setup is using CPU only and takes 15/20 seconds per computation. So a GPU is needed.

Any advise?

Comments

  • Thanked by 1lentro
  • AlbaHostAlbaHost Member, Host Rep

    @millzee said:
    Hi,

    I am looking to rent a GPU server long term for REST inferencing using flask.

    My current setup is using CPU only and takes 15/20 seconds per computation. So a GPU is needed.

    Any advise?

    What GPU are you looking for?

  • Suspect a k40 or k80 will do but need to do some testing on which will be more suitable.

  • lentrolentro Member, Host Rep

    Thanks!

    @millzee

    For ML, I would recommend these instant deploys:
    https://fluidstack.consoledock.com/deploy

    Prices start at $0.42/hour, and you only pay compute costs when the instant deploy server is on. So, if you turn it off, you only pay the storage cost. Calculations are done on a per-minute basis.

    E.g. if you get a Quadro 4000 server, every hour it is on costs $0.42/hour, but every hour it is off costs only $0.01/hour

    They aren't the cheapest but are certainly better than AWS/Azure.

    If you're interested in dedicated servers rented on a monthly basis, I can do them for like $100-$200/month for a GeForce 2070 GPU.

  • @lantro thanks for the info. The problem i have is that I need the server to listen for requests anytime, so 1 day there might be no requests but the next day there might be 100. So I suspect the server needs to be on 24/7. In relation to the 2070GPU, is it good for tensorflow/ml inferencing? as our model is already trained.

  • stefemanstefeman Member
    edited August 2021

    @lentro said:

    Thanks!

    @millzee

    For ML, I would recommend these instant deploys:
    https://fluidstack.consoledock.com/deploy

    Prices start at $0.42/hour, and you only pay compute costs when the instant deploy server is on. So, if you turn it off, you only pay the storage cost. Calculations are done on a per-minute basis.

    E.g. if you get a Quadro 4000 server, every hour it is on costs $0.42/hour, but every hour it is off costs only $0.01/hour

    They aren't the cheapest but are certainly better than AWS/Azure.

    If you're interested in dedicated servers rented on a monthly basis, I can do them for like $100-$200/month for a GeForce 2070 GPU.

    How much currently for 2070 or 2070 super monthly?

  • lentrolentro Member, Host Rep

    @stefeman said: 2070 super

    For 2070 Supers / 2080 / 2080 Super GPUs, my pricing depends on specifics (RAM, CPU, disk, bandwidth), I generally sell for $150 - $200/month.

    Lower end might be:
    $150/month
    14 GB RAM
    8 vCPU of AMD EPYC 7401P
    100 mbps fair share

    Higher end might be:
    $200/month
    62 GB RAM
    AMD Ryzen 5 3600
    1 gbit fair share

    If you're remotely interested in these, shoot an email to [email protected], would be happy to work something out, we've got over half a dozen GPU models so there's something for every customer :)

  • awesome, i'm just doing some testing with rtx atm so will be in touch!

  • lentrolentro Member, Host Rep

    @millzee said:
    @lentro thanks for the info. The problem i have is that I need the server to listen for requests anytime, so 1 day there might be no requests but the next day there might be 100. So I suspect the server needs to be on 24/7. In relation to the 2070GPU, is it good for tensorflow/ml inferencing? as our model is already trained.

    Ok, so you need a monthly GPU, not an hourly.

    I think GeForce cards are pretty suitable for inference. Compared to V100s, A10s or other "enterprise" GPUs (which we also have if you need :) ), the FP16 and FP32 performance-to-cost is usually much better on GeForce.

    Main reason to go for V100s or A10s would be because of the error-correcting RAM, NVIDIA-certified apps, vGPU functionality, or FP64 performance.

    I doubt you need good FP64 performance or other "enterprise" features so GeForce cards are definitely good enough.

    I don't know your exact computing needs but we (Dash Cloud) have the following GeForce GPUs:
    1070
    2070 Super
    2080
    2080 Super
    2080 Ti
    3080

    It might be helpful to start off with a 2070 Super or 2080. If you need more usage, we can migrate you over to a 3080, if it's sufficient, a 1070 might suffice. If you're interested, shoot an email to [email protected]

Sign In or Register to comment.