Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Searching for a GPU rental, need about 48 GB vram

So i started training LLM's but because my training script is incredibly heavy (im not kidding i have 45 hours of 2xT4 on kaggle which i burned all of them) i neeed about 48 GB vram with a good teraflop rate(should be better than T4 because my script on those T4's took around 12 hours)

Comments

  • What’s the budget? Cloudify has that for 450€/mo

  • onesinrenonesinren Member

    If it's a one-off training run rather than a month-long commitment, hourly marketplaces are usually much cheaper than a monthly dedicated box. Vast.ai and RunPod both list A6000 / L40S 48GB cards at roughly $0.4-0.9/hr, so a 12-hour run costs a few dollars. An A6000 or L40S should also be well over twice as fast as a T4. If the script is heavy on memory, try QLoRA, gradient checkpointing and bf16 first, since you might fit it on a single 24GB card for even less.

  • WilliamWilliam Veteran

    what about an unlocked 80GB A100 (CMP/Crypto Miner 8GB card voltage glitched to full on chip memory)? Fast enough in compute?

  • dataforestdataforest Member, Host Rep
    edited 2:05PM

    If you need a little more power (hourly billed, no setup required): https://cloud.dataforest.net/en/products/gpu-seeds

    Thanked by 1oloke
Sign In or Register to comment.