All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
OVH/SYS failover IP bridging
Has anyone with an ovh/sys server managed to get VMs working on bridged failover IPs using a normal linux distribution (Ubuntu in my case) on the host? I need the host to run a standard distro (not proxmox).
I've registered a virtual mac, assigned it to the VM etc etc. Looked at all the documentation but traffic just won't flow. It sounds simple in theory, I've got everything bridged like the documentation says, set up static routes on the guests but the gateway refuses to reveal its MAC to ARP requests which I can see going out eth0. Is there some magic step I'm missing?
Any pointers more than gratefully received, really need to do some actual work on this machine tomorrow
Comments
sounds like you need arp_proxy enabled?
@william thanks, I have tried that - also rp_filter=2 and 0 which a lot of people seem to claim works...
Welcome to OVH.
Also, beware the claim in their marketing / website that they "We are the only Service Provider to support your Virtual Machines!" as they don't even know what KVM is, and even their "SWAT" team doesn't know what "Bridging" is and why you need it.
I was disgusted with the level of service we have been shown, but I am very happy with their network otherwise, as well as their hardware selection and what they do support.
Also worth noting, you can't use the Primary IP [ eth0 /32 ] as well as a vSwitch KVM Bridge [ eth1 /whatever ] simultaneously. Their routers will kill your server for broadcasting on the wrong nets. Also, bridging doesn't [currently] respect custom routing [routes-eth0] under CentOS 6.5 latest.
You're going to have to invent some pretty nasty ARP rules, back-filtration and proxying, just to get their network to play nice with you. You're going to have a huge problem if you want to use their IPv6 (stuck on eth0) with your normal failover IP ranges (stuck on eth1) due to the aforementioned bridging / arping hell.
Good luck!
Thanks - the SYS servers don't have eth1 connected, so assume this is for their non-budget servers? I've got my VMs bridged to eth0, and I can see broadcast traffic on the VM but their router never speaks directly to the VM, even if I enable proxy arp and explicitly create static arp entries set to 'publish'.
Did know what I was getting into with OVH, but you'd think they'd at least write a simple document somewhere that explained their network weirdness such that someone who knows the difference between bridged and routed networking can actually set it up.
KVM is not panacea. VMWare is correct solution for their case, because they know how to work with it. OpenVZ is the only way for Linux-only cheap machines and it's cool, if you know what is cgroups.
They have correct business model and i am sure they'll provide better service than you ever can.
Rather than blame good provider, go to read manuals.
Try to read French documentation. They usually provide much more detailed information in their native language.
My comment was not baseless, I've had servers with them for well over three years now, and have learned how to work "around" the staff to get the answers I need. OVH constantly gives me ranges that aren't even routed to my rack, ranges with IP addresses containing pre-existing firewall / mitigation rules, ranges on the wrong VLAN / vSwitch, .... lots of issues.
As per the documentation, the only documentation they have on bridging or KVM at all, as well as how to use their silly /32 IPv4 ranges is located on this rather old link:
http://help.ovh.co.uk/BridgeClient
However, their own staff do not recommend the use of this guide / manual anymore, as it is outdated and only works "sometimes". It was what I needed to get the /32 working in the first place, so I could download mod_bridging, then I did the bridging, and the /32 never worked again. [ Oh well, I don't need 1x IPv4 that badly. ]
Their marketing / website clearly states that they support "Virtual Machines". It does not say that they specifically only support "VMWare" , else I might agree with this bit of your statement [ as VMWare is what they use. ] They should update their site / advertisements if they intend to only support VMWare. I managed to get 30% off my services for life because of this marketing blunder on their part, and they still didn't bother correcting it [ so I guess they don't care that much. ]
Oh, and to top it all off; here's my bandwidth graph: two points if you can figure out what's wrong with it, OVH staff couldn't figure it out!
Hint: Those values are shown in bytes per second.
@GoodHosting KVM can be easily set up on OVH. I have been doing it for a few years for non-profit use & Phil from mycustomhosting has been doing it for his company.Only thing you need to understand is how bridging/networking works on OVH servers through their manuals & rest is easy just like on other hosts' networks.
If you want I can write a tutorial so that anyone who is having problems can benefit from it.
Could not agree more. MAC addresses are big. Make sure they match and you have a proper bridge setup.
OVH provides an unmanaged service so yes I expect them to tell me I'm on my own if I asked a question concerning software or etc.
I'd be curious to see if I was missing anything obvious.
Here's the fun part about KVM bridging in a smart environment like OpenNebula: OpenNebula is smart enough to have the ability to tag each Virtual Machine's traffic with an individualized vLan identification [ QinQ ] . OVH's network does not support this.
OpenNebula falls back to MAC-based IP allocation when vLan identification and smart switching is not supported [ i.e.: when the host network does not support Vyetta, or does not have Open vSwitch listening, etcetera. ]
OVH's vSwitch will happily listen to any MAC address on any IP address, on any vLan [ a huge security issue I've brought up with SWAT, and they are currently looking into how to solve. ] However, their public network [ eth0 ] does not support this whatsoever.
So what happens when [for example] 00:02:4b:6a:ce:9f tries to speak for 2607:5300:60:2517::1 on the public network [ eth0 ] ? OVH's network will drop all incoming packets, but allow outgoing packets just fine [???] . This is also something that the SWAT is still looking into, as they are sure that it should either allow all or block all; not work half and half.
This would all easily be solved if:
At least one of the four above qualifying factors [ or full Vyetta support ] can be found with most other hosts [ even ColoCrossing's network supports QinQ... somehow. ] I'm not sure why OVH's network isn't able to support one of these.
SWAT's responses on the matter so far are humorous:
Nobody has commented on the broken bandwidth graph yet.
Their sales & tech support is handled by same guys with just basic knowledge only but I cannot blame them for it because they provide decent quality server with high specs with lower pricing as compared to competitors.You helped me when I needed help with setting up SolusVM & IPV6 properly on OVH's network there are lot of guys out there who do have problems & OVH support would do nothing except providing you with links to their guides.
Sorry @Goodhosting I do not use the vRack. I assumed you are talking about vRack as you mention VLAN's as that would be the only way I know OVH to support this. If you'd like to send me your ifcfg-eth0 and ifcfg-br0 or likewise, interfaces if debian etc etc etc. I will compare things with my setup and provide any assistance I may :P PM me. Response time is not guaranteed but neither is OVH As we both know.
Likely the reason for the troubles is the classless routing involved with everything, they do this to avoid having to spare X IP's per block etc. It makes sense to me at least.
Also please note the OVH support staff are NOT the ones creating the scripts. As a programmer I can tell you this will cause a bit of an issue translating to bugreports.
From what I have heard the most popular cloud platforms like VMware & Parallels work very well with OVH, don't know what is wrong with OpenNebula though.
Please, please do! Extracts from a working config would be more than enough.
Alright will work on it very soon, may be tonight or tomorrow.
Awesome, thanks.
I'm beginning to suspect something is broken on OVH's side. I resorted to assigning the FO IP's to the host, and NATing them through to VMs. This works fine, but only on IPs that have never had a virtual mac assigned. The ones I've tried to bridge, and removed the virtual mac from are all failing to route to the host, even after a few hours.
Any tips on getting your problem in front of someone competent at OVH?
Typically - just as I get my NAT setup mostly working, got an email from OVH saying 'the switch is fixed' (which it actually is) and my bridged IPs sprang to life. Not sure if this is a win or not.
@K2Bytes would still be interested to see your tutorial. Since I am apparently capable of basic networking let me know if I can do anything to help. There really is no non proxmox/vmware based tutorial out there that isn't largely wrong, so I think this would be a good thing to contribute to the community...
Yea...unlikely, it just looks like that in a L2 design or is a misconfiguration.
@K2Bytes
@MCHPhil
@William
@tehdan
This is our current setup, which magically works; but requires our customers to do some really fancy overrides in their Guest Operating System; and only works in Some Operating Systems [ doesn't work in Windows at all. ]
It makes absolutely no sense, but it's the only way we've managed to have both the guest and host machine's routing with the Primary IPv4 and Primary IPv6 working.
ifcfg-eth0
ifcfg-br0
Guest ifcfg-eth0
Note that this gateway makes absolutely no sense, and watch this fun:
Guest route-eth0 [MUST CREATE THIS FILE]
This makes absolutely no sense, but was the only way I could actually get it working.
I'm waiting for OVH to comment on this, so far they've taken five ticket replies to even realize I was using KVM, and which files were on the host and which were on the guest.
Workaround amalgamation taken from: http://help.ovh.co.uk/BridgeClient
Huge caveat: DHCP can't force that route-eth0 file onto guest machines, therefore it can't be done through DHCP. DHCPd also requires you give a subnet, gateway, network, etcetera; which breaks the configuration of the guest as well.... so this all has to be done manually, have fun.
If you want an IPV6 VPS to work, replace a few steps below:
Guest ifcfg-eth0:
Guest route-eth0:
So be it IPv4 or IPv6 , your host machine turns into a router forwarding packets, due to how their network is set up to only allow certain MAC addresses to speak for each individual IP address. It's bloody ludicrous.
In both cases, the host machine should be set up to forward packets.
I don't think you are using your gateway IP correctly. The IP's are routed to the host nodes gateway. If you use any other IP for your gateway OVH will block your IP after XX hours or days.
Please look back over the config I showed you, as that configuration will work if you place your values into the spots. All IP's route to your host nodes gateway. Using another gateway will cause very bad ARP issues. OVH blocks for this.
Setting gateway in the network configure while also setting up static routes? That is not right. Debian will automatically fail the network if you do so. Comment them out. Set the gateway manually via routes. Do not your your primary node IP. Use the gateway IP and you will have to setup vMAC addresses in OVH manager. Use type OVH and generic other information. If MAC on VM is not correct with MAC in manager. Bad day.
@MCHPhil
What gets me:
This means that your guest machines:
" Route " to your host machine BUT use the /24 gateway of your host machine. They're also REQUIRED to ARP themselves, which breaks a lot of things [ both on OVH's network, and on mine. ]
The above steps are literally what I was told to do during my 2 hour call with them today.
Don't worry about what they say, I have multiple nodes and 100's of KVM's online in OVH that use this setup and it's just classless static routing. It's a pita but it's nothing wrong or invalid. Please be familiar with http://tools.ietf.org/html/rfc3442
I'm rather curious, what would an exmaple dhcpd ruleset look like for that?
We've got the following at the moment, which worked on their vSwitch , but doesn't work on the public network [ obviously. ]
Please look at the link I pasted and it provides all you want to know about configuring DHCP with classless static routes.
Found it:
Absolutely confusing.
Sorry, even worse; is that you must specify it in special speak:
option 121 18:C0:A8:7B:0A:22:48:2A ;
That is IT sir. It's pretty straightforward in my eyes. No magic needed
You should not have any hex characters. Notice the examples they gave has no hex values?