All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Looking for 1000+ Cloud/VPS servers with 2-4GB, 1-2 cores, 20-40GB
anbelevebel
Member
Hi,
We're currently running around 1500 Hetzner Cloud instances and about half of them are CX11 (2GB, 1C, 20GB) and the other half is CPX11 (2GB, 2C, 40GB main reason is being 2 cores).
They cost about $4-4.5 a month. We also have about 20 of CPX31 (8GB, 4C, 80GB) which cost about $15 a month.
The main reason we're using them is because they used to be cheap (not anymore), they have API (which we definitely need since this is a SaaS business) and the ability to rescale the servers (for instance all of our servers were used to CX11 but we had to rescale half of them to CPX11 last year due to 1 core not being enough for the application that these half of the servers were running.).
Location doesn't matter. Could be anywhere.
CPU doesn't matter. They're running idle 90% of the time. However initial setup process may use 100% for a day or two and that's why we had to use CPX11 for some.
Disks have to be SSD and the amount is important. Usage will gradually increase so we'll have to increase them as well. These 1500 instances are using about 20TB disk space in total.
RAM is very important. CPX11s are using about 1600MB constantly and CPX11s are also using around 1500MB.
Half of the instances use about 70GB bandwidth a month for each. Other half is using about 5GB. So they're like using about 50TB a month in total and Hetzner gives 20TB bandwidth limit to each instance so we're literally using nothing when compared to what we're given.
All of the instances are running Ubuntu 20.04 which were updated from Ubuntu 16.04 last year. We can also work with Linux containers/Docker etc. We don't necessarily need root access, KVM or whatever.
We're doing this with Hetzner since 2018. So you can see that we're looking for a long term relationship.
We're looking for a provider that would worth to move all of our servers from Hetzner. We basically have zero-downtime with Hetzner and that is the most crucial part. Our needs are also very dynamic and can change drastically over time and the ability to upscale/downscale resources is very important.
Recently I discovered about Advin (https://advinservers.com/micro) and they seemed to have what we need for less than half the price but the reviews were really bad so we decided not to go with them.
If we can find what we're looking for this price point, we can expect to triple the number of servers we have in the next 6 to 12 months (about 5000 servers I'd say.).
We'd appreciate your offers/suggestions. We're open to any custom solutions as well.
Thanks!
Comments
that's some volume, I think many providers here won't be able to deliver that much
have you considered "private cloud" aka for example managed Proxmox clusters? Many providers here would be able to deliver many dedis, but you would either have someone to manage them or ask for management from the provider
There is large risk as a smaller-mediumish company deploying tens of thousands of dollars worth of hardware for one customer for virtual machines, so I'd expect to sign a long term contract for this. What if you order services that end up costing a provider $60,000 in hardware to setup, then you cancel a few months later? They are sitting on tons of hardware. In the case of some super cheap high ram VM providers, the prices are clear that memory ballooning and other overselling methods are used so even if you got those you wouldn't be able to use 100% of the resources across dozens/hundreds of VMs at once.
Imo something like Hetzner, who has massive scale, and a large employee base with devs for full featured APIs that you like is your best option.
Why not use dedicated servers and setup your own VM platform? That way you get the resources at no markup. I assume you just don't want to deal with setting it up?
Actually we would deal with this, not that we don't want to but it seems it'd be more expensive for us plus we're not sure about upscaling if/when necessary. We're no experts in virtualization or anything tbh. But as I said we're open to any solutions really. It doesn't have to be necessarily Cloud/VPS.
I would just buy a couple dedicated server from a reliable host and setup Proxmox or XCP-ng.
bulk dedis, good support and nice prices
https://www.reliablesite.net/dedicated-servers/
you can also get management and i bet they do virtualization
https://www.reliablesite.net/dedicated-servers/managed-dedicated-servers.aspx
for this volume, you can most certainly nego the price on both of these things.
@MrRadic
I would also recommend renting IP Subnets.
You can get /24 for ~$100 while you would have to pay $500+
also "private cloud as a service"
https://www.worldstream.com/en/cloud/private-cloud
@treesmokah @MikeA @concept the thing is, if we go with dedicated servers and over time our instances may start to use like more RAM or SSD. While we may basically increase SSD space drastically, I think we that's not the case for RAM. What if suddenly the applications running on one dedicated starts to use double the RAM and it all fills up? What can we do in such case? Right now when this happens we're basically able to rescale the instances through Hetzner's API. I'm just trying to wrap my head around the concept of building our own platform. Please allow me to understand.
You can tell the provider to for example put more ram sticks in to the server and then re-scale vm's.
With Worldstream IaaS they will do it for sure.
Hetzner has 512GB EPYC systems for under €250 EUR on auction right now or €338 for the AX161. You could run on 10 of those as a Proxmox cluster for half of what you're paying now by my calculations.
Alternatively, this sounds like a great fit for Kubernetes
You can live migrate VMs between your hypervisors to optimise for that
Then move that particular VM to another empty node within the cluster??
You can do that easily using Proxmox.
@Peppery9 @BingoBongo the applications we're running have to have their own dedicated public IPv4 addresses. As far as I know Hetzner doesn't support BYOIP and they're selling the IPs for 1.7 Euros a month per IP which is quite expensive for what we're trying to achieve here.
Also wouldn't moving the VM to empty node or whatever change the IP address? We cannot accept any change in that specific VM's/application's public IPv4 address by any means.
We have enough capacity to handle this across our 25+ location platform.
1500 VPS is easy for us to handle, we have client's with 10,000+ in production, and had one hit 20,000 at peak ;-)
We have several possible solutions for this that might be suitable for your needs.
Contact us at [email protected]
~ SMARTHOST
No it will not. Am not talking about hetzner.
Just get few dedicated servers from providers like reliablesite and install proxmox on all of them and join them on cluster. That's it
And for BYOI just rent/lease a /24 subnet from company like ipxo.com
It will cost you only $150/m (maximum) for all 256 IPs
And no, your IP will not changed after migration in proxmox.
(However I'm not sure if proxmox has live scalability feature)
In that case other software can help like VirtFusion or solusvm instead of proxmox.
We have the hardware/infrastructure on hand to handle something like this, and given the quantity could likely work with you on sizing appropriately to the physical node hardware to scale costs accordingly. We just don't have the API functionality built out so that is a miss.
However I have to agree with most here and I think a few dedicated servers and BYOIP/BGP would make a lot more sense for you financially. I believe proxmox can live migrate, and VirtFusion definitely can (no changing of IPs, etc--it's been very seamless)
@treesmokah @MikeA @Peppery9 @BingoBongo @crunchbits so you guys are basically saying that with a few dedicated servers, a few IPv4 subnets and something like Proxmos, VirtFusion etc I can basically build my own personal-use small little Hetzner Cloud like solution for cheaper? This sounds like a great business opportunity if anyone wants to help. 😄 And I'm ready to pay for the hardware.
Yes it might be cheaper in terms of machine cost for you to build your own private cloud, but you are adding on significant responsibility for maintenance, uptime, etc. As a SaaS provider, can your users deal with downtime if you are at fault? Maybe not a good idea, especially if you do not have the expertise.
Does every machine need a dedicated IPv4? May be worth considering doing IPv6 only network and then some IPv6 over IPv4 tunelling.
Netcup.eu has something similar for 3.25EUR/mo, but I am unsure if they have the API capabilities you need.
In a round about way, yes. Not much different than colo'ing the hardware or just renting some dedicated servers from someone else and doing it. It all depends how much you want to offload responsibility/tasks on a provider and how much to take on yourself. The easiest would be to rent VPSes in bulk as you are requesting. Next, renting dedicated servers (maintained by provider) and setting up your own private cloud (most providers can probably give you a private VLAN/BGP at that scale). Third would be renting rackspace/colo and building/buying servers to send and then it is all on you--hardware swaps, remote hands, complete management.
Remembers me of some Chinese client who had 4k VPS with us, arround same specs running some blockchain based CDN.
That being said, I might be able to do that. But deployement might take a month or two to purchase/ship and setup the hardware etc (out of stock currently on dedicated servers)
Yes you are right.
Just get some dedicated server on rent and few IPs as well and you are good to go.
If you are not familiar with Proxmox or any other virtualization software to maintain your own cloud then just hire someone to do that for you.
You will save a lot $$$ doing this
Just make sure you are not violating the TOS of your server provider and you are fine.
Proxmox has APIs for live migration but I don't think you can do that directly from the Proxmox panel. You may need some external setup like WHMCS or Hostbill
Can anyone confirm it please?
We already have our own billing system integrated with Hetzner’s API. If Proxmox or Virtfusion has a client API for server deployments, we can integrate it as well.
I've PMed you about our services as well. We're hosting multiple big clusters for clients, we can certainly help out!
Then go for it.. good luck 👍
We might be able to do that in Lithuania. We have many unused IPs, own a hardware warehouse, etc.
Do they all need a dedicated IP?
@anbelevebel what you are asking is definitely possible with few large dedis and cluster. Proxmox comes with full featured API if that is what you need. At readydedis we have been handling large amount of clients with multitudes of VMs, and if there is very large client we create their very own cluster with our In-house virtualization panel - Hypervisor.io.
Have you ever checked the reviews of Hetzner? Happy customers don't leave reviews very often. Unhappy customers make a point to leave reviews, often times skewing what actually happened.
I think OP requires either tons of IPs or burstable I/O or CPU resources.
I run a vector search service, on top of about 100 VMs. The QPS is low, but whenever an incoming request arrives, every node eats up all CPU for 1-5 seconds at the I/O speed of 1GB/s. In this situation, only by deploying VMs on massive nodes, could I achieve a good performance. Not sure if the OP shares the same requirement.
I don't share the same situation with you to be frank. Yes, I also have burst situations as you mentioned but when it happens, it is for 10-15 nodes max out of 1600 that we currently have, not for all of them unlike you. So it's not really a concern for our use case.
You're absolutely right. As a business owner I really understand this situation and I totally agree.
However, when it comes to my case, from my understanding (correct me if I'm wrong on this), I did some research and I didn't just came to this conclusion about Advin because of bad reviews. I came to this conclusion because all of those bad reviews were about server down times and they're not being responsive to this for up to a week in some cases.
Whereas if you check Hetzner's reviews almost 99% of the bad reviews are about account related problems like Hetzner closing newly created accounts etc. We didn't have any downtime problems with Hetzner neither most likely 99% of the people that are using them. Since I never had any technical issues with Hetzner, I don't care about their customer service either.
In my 5 years of using Hetzner Cloud with 1000+ instances, I never have contacted their support for a technical issue, not even once because everything works without any issues. I only contacted their billing department a few times.
But if all your bad reviews are about downtimes (which seems to be the case for Advin) then I would hesitate to use your service because that is the one thing that we -and probably most businesses- can't tolerate.
I hope this clears it up.