New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Searching for a GPU rental, need about 48 GB vram
So i started training LLM's but because my training script is incredibly heavy (im not kidding i have 45 hours of 2xT4 on kaggle which i burned all of them) i neeed about 48 GB vram with a good teraflop rate(should be better than T4 because my script on those T4's took around 12 hours)

Comments
What’s the budget? Cloudify has that for 450€/mo
If it's a one-off training run rather than a month-long commitment, hourly marketplaces are usually much cheaper than a monthly dedicated box. Vast.ai and RunPod both list A6000 / L40S 48GB cards at roughly $0.4-0.9/hr, so a 12-hour run costs a few dollars. An A6000 or L40S should also be well over twice as fast as a T4. If the script is heavy on memory, try QLoRA, gradient checkpointing and bf16 first, since you might fit it on a single 24GB card for even less.
what about an unlocked 80GB A100 (CMP/Crypto Miner 8GB card voltage glitched to full on chip memory)? Fast enough in compute?
If you need a little more power (hourly billed, no setup required): https://cloud.dataforest.net/en/products/gpu-seeds
im not renting them monthly because they are very expensive
QLoRA degrades the knowledge so im not seeking that way also theres an STL pipeline which will cost a lot of gpu compute
Noted thanks for the information
Hi!
We have an NVIDIA A6000 with 48 GB VRAM available for hourly rental. It could be an option for your LLM training workload and has more compute performance than a T4, although the actual training time will depend on your setup.
One configuration comes with 12 vCPUs, 64 GB RAM and 256 GB storage in the US.
The regular price is $0.74/hour. With our LowEndTalk promo code LOWENDTALK_15, it’s approximately $0.63/hour. The 15% discount also applies to renewals and is available until December 31, 2026.
https://xgpu.store/en/?promo_code=LOWENDTALK_15
By the way, what model are you training?
They do hourly.
I'm currently training Qwen 3.5 9b base model with a big dataset for coding and doing something called SFT which is a fancier and better way to feed an dataset then then the second part then theres reinforced learning which is a part where the model recieves 2500 tasks about coding and cybersecurity if it fails it learns from the mistake and does this until it gets everything correctly without overfitting after coding it will learn how to understand quantized inputs so when people use the model with quantization open on input it wont generate degenerate token soup and dinally it will be quantized to q8, q6,Q5,q4,q3,q2, ternary
All i see is monthly plans are you sure they do rent out hourly
The website i did go Into is: https://www.google.com/url?sa=t&source=web&rct=j&opi=89978449&url=https://cloudify.ro/en/ai-servers&ved=2ahUKEwjLu7Pyto-YAxXmif0HHeidEJgQFnoECCIQAQ&usg=AOvVaw0c99qtUqwksWhy-zu-CiWd
Terrible deal, also a reseller
They don't seem to have 48GB available right now but they definitely do hourly.
https://i.imgur.com/Tvx729M.png
Edit: Actually the 2X RTX 4000 package has 48GB VRAM
@nullwqh If you can work with multiple GPUs (e.g. DeepSpeed/FSDP/etc.) you can rent 2x3090/4090/5090 (depending on what ou need), look for them on Runpod, Vast.ai, or Clore.
You can also rent modded 4090-48GB on Vast iirc.
If you need true 48GB, RTX PRO 6000 or L40S.
You can rent by the second etc. but honestly not worth coming to lowendtalk for GPU offers, LET has some insane VPS offers but GPUs are overpriced and annoying to use. just use a well known gpu provider
also... if you want some free rtx pro 6000s... molab from coreweave has free rtx pro 6000s https://molab.marimo.io/
https://vast.ai/
http://salad.com/
https://runpod.io/
https://www.primeintellect.ai/
https://lium.io/
If you can go with AMD (trust me it will be a pain but maybe your LLM can figure it out):
https://hotaisle.xyz/
I'll review all offers and options, thanks for the response
I have experience with AMD GPUs thanks to my homelab so i believe i can do AMD too
if you say so ¯\_(ツ)_/¯
all things are possible with fable i suppose
but "i have experience with amd gpus thanks to my homelab" dude c'mon
All i can say about training or inference on AMD is: hell because i never had this much of issue in Nvidia GPUs even the old ones but AMD GPUs especially 6000 series are just pure pain to ass
yah thats my point
someone should host a competition who can install flash-attn on mi300x the fastest i bet nobody can do it in <30mins
@nullwqh oh you are from drama.gg 😭
Thanks! That sounds like an interesting project
If you decide to try the A6000, we have a PyTorch environment option in the configurator, so you can select it when setting up the instance. The exact memory requirements and training time will depend on your configuration, especially for the later RL stage.
If you run into any questions about the instance or setup while comparing options, feel free to ask.
xgpu = reseller, borderline scam
at least 5 accounts because my accounts keep dying due to server errors
Verda used to have very cheap spot instances a year ago, but I think the ship has sailed. Vast is the only other cheap one I know of.