New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Searching for a GPU rental, need about 48 GB vram
So i started training LLM's but because my training script is incredibly heavy (im not kidding i have 45 hours of 2xT4 on kaggle which i burned all of them) i neeed about 48 GB vram with a good teraflop rate(should be better than T4 because my script on those T4's took around 12 hours)

Comments
What’s the budget? Cloudify has that for 450€/mo
If it's a one-off training run rather than a month-long commitment, hourly marketplaces are usually much cheaper than a monthly dedicated box. Vast.ai and RunPod both list A6000 / L40S 48GB cards at roughly $0.4-0.9/hr, so a 12-hour run costs a few dollars. An A6000 or L40S should also be well over twice as fast as a T4. If the script is heavy on memory, try QLoRA, gradient checkpointing and bf16 first, since you might fit it on a single 24GB card for even less.
what about an unlocked 80GB A100 (CMP/Crypto Miner 8GB card voltage glitched to full on chip memory)? Fast enough in compute?
If you need a little more power (hourly billed, no setup required): https://cloud.dataforest.net/en/products/gpu-seeds