New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
looking for AI LLM (Private deployment) ready hosting with competitive price
chrisanderson
Member
in Requests
like LLaMA 3, Gemma 2 , Falcon 2 , deepseek or others popular open source LLMs ready.
The API price need very competitive compared to the official .
Location , US ,EU or Japan.
Thanked by 1oloke
Comments
and Allow fine-tuning better
You should rent a dedicated server with gpu for this. There are many providers that offer GPU cloud and dedicated servers with GPU.
https://www.hetzner.com/dedicated-rootserver/matrix-gpu/
https://crunchbits.com/gpu/cloud
https://www.netcup.com/en/server/vgpu
https://www.ovhcloud.com/en/public-cloud/gpu/
Hi. Explore our options. We can help you set up tools for working with LLMs. For example, we can deploy Open WebUI + Ollama with access via web and API endpoints.
We can offer several inexpensive GPU server configurations:
Conf#1
Location: Netherlands
CPU: AMD Ryzen 9 3900X | 12 Core 24 Threads 3.8/4.6GHz
RAM: 64GB REG ECC DDR4 \ 2x 500GB NVMe
GPU: Gigabyte GeForce RTX 2080 Ti 11GB VRAM
Port/Traffic: 1 Gbps - 50TB Traffic
Link to order
Conf#2
Location: United Kingdom
CPU: Intel Core i3 7100 | 4 Core 8 Threads 1.2/4.4GHz
32GB RAM \ 1x 500GB SSD, 3 x 6TB SATA
GPU: 1x Nvidia GeForce GTX 1080 Ti 11GB (Up to 2 Nvidia GeForce GTX 1080 Ti 11GB)
Port/Traffic: 1 Gbps - Unmetered
Link to order
Very unlikely. We are talking about hardware investment in the range of 5-6 digits of USD if you want to run the big models, say, Deepseek 671B. It would be orders of magnitudes more expensive than the official API, even if you run the official API 7/24/365.
If you are only interested in the smaller models, say, Deepseek 14B, then you can use any modern CPU/GPU to do that. But at that point the free GPT/Grok/gemini will beat all of those self hosted small models.
Hey, if a server with a GPU would be an option to self-host a smaller model, I have the following configurations in stock:
With GPU:
AMD Ryzen 5 3600, 64GB RAM, 512GB NVMe, RTX 3060 12GB, €135/mo
AMD Ryzen 7 5700X, 128GB RAM, 512GB NVMe, GTX 1080 ti 11GB, €180/mo
Intel Core i7-9750H, 16GB RAM, 256GB NVMe, GTX 1650m, €32,50/mo
With iGPU:
Intel Core i5-9500T, 16GB RAM, 256GB NVMe, €28/mo (discount on larger amount)
Intel Core i5-8525U, 20GB RAM, 512GB NVMe, 1TB HDD, €20/mo
Additional HDD storage is possible via a storage box.
Custom OS templates are possible.
6-month or yearly commitment can introduce an additional discount.