New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
It's OpenVZ, so there's more room in there to work :P
Honestly though, love the Anycast.
Same thing happened to me, after adding an anycast IP it was already on on all other hosts (and it didn't work on other hosts, just on the one added). KVM if it matters.
I have a ticket open - 384112, thanks.
Noted the 'active' thing
The 'status unknown' thing should also be addressed. Our propagation scripts weren't applying the correct chmod/chown settings.
I just got up after being up way too late this morning so I'll start chunking away at tickets & whatever bug reports come in.
Actually, nLayer took a dive this morning according to reports on WHT which caused our BGP session with Staminus to go dark in at least Jersey. It's fine once again but yeah, was out of our hands and Staminus'.
That's how we operate
I hate competing on pricing.
Francisco
Does anyone have this working with openVZ ?
I'm working out a routing derp that causes issues on OpenVZ's right now
I wish I could've spotted a lot of this crap while it was in beta but I guess not.
The issue is due to what interface it tries to send packets as.
Francisco
Alright so KVM users should be set
You'll need to 'save' your settings on each VPS one more time but this time it'll store the entries properly.
The problem with OpenVZ's is that since venet0 is still used, it's stopping the reply packets from being sent as they're technically flagged as being spoofed. The solution is to add source route entries.
Instead of forcing people to do this manually, I'm currently plotting & coding it so it automatically adds the entries for people at boot time & whenever you 'save' your IP entries.
We won't be opening stock until these last few things are addressed.
Francisco
I need some feedback from everyone doing anycast on OpenVZ
Currently we're trying to make both venet0 & ha0 work together at the same time. Originally in our test beds this worked perfectly but I guess I didn't think about testing tons of different IP addresses all at once.
This is proving to be challenging and requiring source routes to work, as an example:
auto ha0:1 iface ha0:1 inet static address 209.141.xx.200 netmask 255.255.255.0 up ip rule add from 209.141.xx.200 table 10000 up ip route add default via 209.141.xx.1 table 10000 down ip rule del from 209.141.32.200 table 10000 down ip route del default via 209.141.xx.1 table 10000Is what an entry would look like in debian/ubuntu's /etc/network/interfaces.tail
Are you OK with this or would you rather we just make it so venet0 isn't even used if you have failover/anycast IP's available to this VPS, and shove everything through the bridged ha0/eth0?
I think the bridge way is a lot easier since we don't have to explain routing tables and all that.
Francisco
IMO if your clients have to enter routes it may become a nightmare for you. (think MCHPhil from a year ago)
I would run everything through ha0/eth0 if the client has anycast.
That was our taken on it. We'll likely revert back from ha0->eth0 and introduce an eth1 that is LAN IP's only. We can then have a single unified support article that explains how to bring up IP's.
For users that don't have any floating IP's/etc enabled, we'll just have it automatically put them back on venet and remove their eth0/eth1 interfaces.
Francisco
I agree. Requiring source routes is a big hurdle. Anything you can do to simplify that will make everyones lives easier in the long run.
My OpenVZ systems seem to have an even lower layer problem. I've enabled the anycast IP on OpenVZ systems in NJ and LV, but I don't see any traffic for the anycast IP on either venet0 or ha0 using tcpdump.
My KVM system in LU works just fine (using eth0).
throw in a ticket just so I don't forget you
I'll be working on the eth0/venet0 changes in the next little bit. I'm thinking i'll have it in play by the end of tonight.
Francisco
@MCHPhil , And as I said at the time, it was completely BS that OVH required your clients to do a bunch of fancy routing for their IP addresses to work. (Hence why we discontinued this nonsense in our Montreal location (OVH) and just made OVH fix their broken system for our IP range...)
I also recommend against any fancy routing being required, as if your client needs to do anything special (like bridging of their own, or anything to do with VPNs or other sort of adapter controlled stuff...) they won't have to come back to you asking why it broke.
Albeit I am aware that OVH's routing issue is a completely different issue than what yours poses, but it requires a similar amount of end-user intervention for an otherwise normal feature to "work".
That's where our concern is. I don't want to have to write a long guide explaining policy-baced-routing and explaining that 'No, you have to set a unique routing table ID', etc. We already do that for GRE tunnels but since it's only a single routing table addition over there, I was able to make it copy/paste friendly.
Right now the guys & I have pretty well decided that if a customer has floating/anycast IP's enabled on a VPS, it'll disable venet0 all together and give them a pair of interfaces, eth0 & eth1, and they use them like they would on KVM.
They would have to (for now) manually configure them for all of their addresses which is kinda lame, but they would have to do that for their anycast/failover IP's anyway.
Francisco
Watch out on systems that have that "Predictable Interface Naming" crap from RHEL/CentOS/Fedora after versions 7, 7 and 16 respectively; with systemd. This will give the interfaces in the machines running these distributions unusual interfaces names, and render documentation near impossible to follow.
This is a bad lesson we learned with CentOS 7 being added to our offerings, and our documentation not being updated for all the ridiculous names interfaces and adapters obtain in the new naming scheme.
Yep, already noted. I'm thinking it won't be able an issue on OVZ, though, since those naming changes are usually based around whatever the NIC's driver is (BSD style).
I'm going to go sit down and get these things touched up. We've not had sales open or done an official email blast letting people know yet since we wanted to get a much bigger testing base in place first. My own testing was saying the 'veth along side venet' method was going to work fine but I most likely had the default route going out eth0 anyway.
Francisco
Best of luck on the development then! I wouldn't mind hearing how it turned out, we had a very similar issue when using OpenVPN server on a UCARP controlled device-set, through bonding. It turned into a large mess, and clients of the respective OpenVPN adapters (32 in total) were getting all sorts of fun routing.
Well, with going with the 'venet0 is cancer' route, it makes OVZ networking exactly like KVM's. KVM's are working perfectly with anycast & IP failover so at least I got that going for us :P
Fran
Not sure why I was tagged. I have nothing to add to this. :P
Congrats on the new features though
I decided to wait until you had worked on the eth0/eth1 changes for OpenVZ.
I noticed just now that everything is working fine just now. I had to open a console and manually configure my primary IP and my anycast IP on to the eth0 interfaces, but it works now at least. Yay!
One odd thing I've noticed is that ping times are quite a bit higher on the anycast IP than on the primary IP.
Primary:
64 bytes from 199.195.250.92: icmp_seq=109 ttl=52 time=29.6 ms
Anycast:
64 bytes from 198.251.86.45: icmp_seq=35037 ttl=53 time=44.9 ms
Looking at the traceroute output, they take two completely different routes from my test host (primary goes he.net to choopa, anycast goes cogent to gtt to staminus).
I'm guessing this is for network engineering reasons, but figured I'd point it out.
Great
Staminus has a more limited amount of providers so that makes sense.
I'll be updating the 'save' popup in a few minutes to make it warn OVZ users that they have to manually configure their addresses when using failover/anycast IP addresses.
I'll put together a wiki article a little later as well about this all.
Francisco
Yeah! it's working in openVZ for me too.
Time to do the happy dance
Thanks Francisco
Is the manual is available somewhere @Francisco, I managed to get your OpenVZ VPS in al location but having problem/have no idea on how to setup the IP manually there ...
@godong - You can try the KVM setup instructions here for your main (not anycast) ips as eth0 and then add the anycast ips to each vps as an aliases. (eth0:0 eth0:1 etc)
@godong @ElChile If you have more than one node in a location, remember that only one VPS can have the Anycast IP. To get a serious setup up and running, atleast 2 nodes in each location are recommended, to use failover with keepalived. Remember that they don't backhaul. Feel free to contact me if necessary, got a working setup up and running.
Put in a ticket and we can help
I'm still writing documents for it all.
OpenVZ's been a bit of a pain in the ass due to having to deal with veth's. OpenVZ doesn't really provide a list of interfaces in vzlist or anything like that, so i'd have to parse the configuration files to find out if a virtual server has its interfaces bound.
Francisco
LU seems to be down @Francisco
LU's fine over here? I know Staminus had a burp in NL but it seems fine.
PM me an IP and I can check your settings?
Francisco
@Francisco - any chance of 128mb's ovz's being restocked in NJ?
I'm going to likely send over another node soonish.
The 256MB+ plans are a nice pick up since they're pure ssd's, snapshots, and all those fun things.
Francisco
@Francisco - yeah we'll see I'm interested in testing out anycast with my dns server you currently host in las vegas. I need to do some more reading because I'm unsure if it can do what I want it to do, as I'm wondering if it's possible to make my main IP address in las vegas into the anycast IP If I buy servers in your other 2 locations?