New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Looking for VPS, NYC Area, 2c 2G
W1zzardTPU
Member
in Requests
I'm looking for several VPS providers in the NYC/East Coast area, for etcd cluster nodes
- 2 cores
- 2 GB RAM
- SSD or NVMe storage (10 GB should be enough)
- 1x IPv4
Budget is around $6/month
Already using: Fourplex, Ouiheberg, Interserver, Purevoltage, Cyclone Servers
Comments
https://my.kuroit.com/store/sale-offers/flash-sale-ashburn-2vcore8gb-ram40gb-space
@AlexPads
You can go with @Clouvider . They sometimes do 10% off promotion.
Good suggestions, ordered three VMs so far, keep them coming please
Racknerd
2 vCPU Cores
30 GB PURE SSD RAID-10 Storage
2 GB RAM
NY
$17.66 / year
aff
no-aff
We've a double core promotion running: https://lowendtalk.com/discussion/206617/kvm-vps-eu-usa-free-upgrades-ipv4-64-ipv6-virtfusion-instant-setup/p1
80 GB SSD Space
2 GB RAM
2 CPU Cores (when commenting your order number at the link above)
4 TB Bandwidth
10 Gbps Port
1 IPv4 Address
/64 IPv6 Space
Available in New Jersey for $6/mo. This is powered by KVM and VirtFusion
Note to self: https://lowendtalk.com/discussion/208256/dedirock-kvm-vps-instant-deployment#latest
Just curious.
How many nodes does your etcd cluster consist of? In how many data centers is it deployed?
5 nodes is the target. Ideally 5 different DCs, just a few ms apart, but finding VMs with good storage is pretty hard, so right now I'm running two nodes in the same DC, which of course is workable, considering 5 nodes can tolerate two failures
Is 2 cores strict requirement?
I can definitely do that
✨VPS Plan : 2G-2C-NY-KVM-VPS
🔹 2GB DDR5 RAM
🔹 2 CPU Cores (AMD Ryzen 7950X)
🔹 30GB NVMe Storage
🔹 4TB Bandwidth
🔹 10Gbps Port Speed
🏷️ Standard Price : $7/month
💰 Discounted Price : $4.90/month (Use code : H4F-30%-J0625)
https://www.host4fun.com/new-york-vps
As you may know well, I don't recommend to use more than 3 data center for etcd, which would result in network partitioning or split-brain. For HA, 2(DC1) - 2(DC2) - 1 (DC3) would be better most of cases especially if over the wan.
Well I came to suggest us but looks like you're already using us
With that setup you could just run 3 nodes, because putting two eggs in one basket doesn't give you extra HA
I wonder if @MikeA has something to offer you.
Will Valley Forge, PA work for you? I need to update our Looking Glass (previously was "Allentown, PA"), but we'll have full restock within 24 hours.
Happy to do a custom plan for you that matches your budget. Shoot me a PM if you're interested.
For example, this is the ping from us to Purevoltage's NYC POP:
For the technically curious, here's what I'm checking on the VMs.
https://imgur.com/X9Lqx0W
This chart plots disk apply latency of etcd. Note the purple line, pretty high, so it got replaced. Also note how spiky the green line is, despite being lower overall, so this one's getting replaced next.
The second part of how happy etcd will chug along is network latency. While average ping looks fine, sub 10 ms for all our current nodes, the 99th percentile looks MUCH worse for some, sometimes reaching as high at 300+ ms
https://imgur.com/ub2p1PL
I mean 5 nodes; 2 nodes (on different hypervisor in DC1 as known anti affinity group), 2 nodes (on different hypervisor in DC2), 1 node (DC3)
This would be way better than 5 nodes each different DC. There is hardly two data center off at the same time.(you should consider physical address of data center in case of power outages).
I'm sure you will see infinite leader election messages from each other. The worst case scenario is that the cluster stops working due to split brain or network partitioning.
Good luck.
Not really, that's with a 9-node etcd cluster in 9 DCs (under load, including "bad" DCs)
https://imgur.com/sqEGmzW
Also a bunch of these are when I changed the cluster members
Just curious, would you show me Raft term numbers that when Leader changes add 1?
I don't think etcd exports the current raft term to Prometheus?
This cluster is currently on index 7470 with 1269984583 applied, so about 200k transactions per term, in case that's what you're asking
I found metric name is etcd_server_leader_changes_seen_total.
Yeah, that's a good proxy. It's not exactly the raft term though, it's a counter that starts at zero when the etcd process is started.
I'm alerting on it with
expr: 'increase(etcd_server_leader_changes_seen_total[10m]) > 2'
for: 0m
labels:
severity: warning
annotations:
summary: Etcd high number of leader changes (instance {{ $labels.instance }})
description: "Etcd leader changed more than 2 times during 10 minutes\n VALUE = {{ $value }}\n LABELS = {{ $labels }}"
Done
Some findings from the last few days:
We can help out with this:
2 vCore
2gb Ram
20gb Storage
2tb Bandwidth
$5/month or $30/year
https://store.soutarmc.com/index.php?rp=/store/nyc-vps-hosting/earl-vps
almost zero leader changes now, with 6 cluster nodes in 6 DCs (yes I know even number is not optimal)
https://imgur.com/oWem97f
Now I just need to find which one is the weakest and drop it
How to pick? disk latency vs network latency vs how often noisy neighbors cause issues vs cost, no clear winner right now
Network Latency isn't that important. It can be easily overcome with configuration (and depending on your use case) such as election timeout , heartbeat interval. If you need high TPS, latency is also important.
With the Raft consensus algorithm, disks or neighbors may not be a major problem for a small number of nodes.(but if your leader has a problem on a disk latency or bad neighbors, it could be loose a leader, resulting in making the cluster slow even stop)
However, packet transmission stability between data centers is far more important. The above metrics indicate when each node receives a leader election message. I don't know if I understand that metric correctly because I don't use Prometheus.
I see this in the logs about once a month or nothing.(I'm currently running ETCD Cluster in 3 Availibility Zone in AWS)
For me It seems not to be stable but acceptable.
It's your choice.
About Disk Latency when I install via the provider template, I saw a lot of warning message about Disk Latency dispite YABS Disk result is excellent (4K~1M). It could be a coincidence. I'm not sure.
etcd relies on fsync, which YABS doesn't test for at all. There's decent docs out there how to test fsync performance with fio. lmk if you need more info on that, I found that it's still better to just add the new node into the cluster and look at the metrics.
3 nodes or 5 with 2/2/1 as you mentioned before? Which AZs? What's the regular ping like?
What your monthly AWS bill like (just for the etcd cluster)?
Yeah 99th latency is what matters, monitoring it helped me a lot diagnosing problems.
Noisy neighbors can eat up all CPU time, NIC bandwidth or disk IO. That also depends a lot on how the hosts VM limits are configured