New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
BGP software for anycasting?
Need a BGP software suitable for anycasting. New to this, but I assume that means it needs to be able to (1) maintain a BGP session and announce through it, and (2) can be configured not to accept routes from peers and (3) does not disturb the ordinary routing of the Linux kernel.
Ideally, I'd like something that can also let me play around with path prepends and community attributes.
What would be easiest to get started with and configure? I'd prefer to use a "swiss army knife" type package rather than a bloatware. Can ExaBGP do most of what I need? Is there anything even lighter out there that could do the trick?
Thanked by 1aglodek
Comments
quagga
If you don't run the zebra daemon and only run the bgpd daemon it will not inject any routes in the linux kernel / routing tables. And even if you decide to run the zebra daemon, you can simply negotiate with your peers to not send you any prefixes, or you can filter them at your end.
That. Quagga.
Well, since it's unanimous, I'll get digging on the Quagga documentation, then!
Quagga (bgpd) or exabgp
http://vyos.net/wiki/Main_Page
Perfect solution, will run on small KVM of 512mb
Exabgp or Bird are other options.
bird!
Seems like you spoke too soon
Indeed. But I think I will go with Quagga's bgpd after all.
Go with Quagga!
Quagga. Configuration is cisco like.
I use bird, it's excellent!
Has anyone here used (1) Quagga, (2) Bird or (3) ExaBGP in multiple locations to setup and run an anycast network specifically? I understand that anycast - as opposed to standard BGP session - requires additional manual setup. Hence my question which software is better suited for this particular use...
A few practical questions that come to mind:
have you been running the software on VM's (KVM or Xen)?
what kind of traffic is such a lightweight setup able to handle? (what's your VM configuration (RAM, vCPU's, HDD or SSD)?
how much RAM needed to hold the whole routing table in memory? (which, I understand, should increase traffic handling capacity considerably?)
any special configuration requirements on the part of the VM provider?
any other caveats I should keep in mind going in for the first time?
Quagga, Yes.
Yes.
10Gbit+ with the right NIC
Around 256MB RAM per peer is fine
You need an unfiltered (spoofing enabled) KVM (no ebtables, ebtables disabled or your anycast network added to ebtables).
You will get massive routing latency from various countries depending on the upstream of the enduser network (i.e. UPC -> Cogent/Level3 -> US node instead of something in EU) - And there is zero you can do besides adding more locations, which in turn will again cause this problem with new ISPs. Only way to solve this is scaling more and getting your own IX ports.
have you been running the software on VM's (KVM or Xen)?
Yes
what kind of traffic is such a lightweight setup able to handle? (what's your VM configuration (RAM, vCPU's, HDD or SSD)?
Depends from setup, if you need this to act as router (distribute traffic to second, internal subnet) better use small dedi
_
how much RAM needed to hold the whole routing table in memory? (which, I understand, should increase traffic handling capacity considerably?)_
~ 1G, for most of the purposes you don't need full table
any special configuration requirements on the part of the VM provider?
As @William mention
any other caveats I should keep in mind going in for the first time?
haha, route optimization
we had to write custom AI solution which constantly monitors our setup and adjusts the PoP (but we have > 100 servers)
I think this is wee bit beyond my little, private anycast "school project"
This said, can you elaborate a little on how getting your own IX ports helps exactly?
If you are going to use a VM you are more less limited by provider's port speed (shared 10/100/1000M ?). For physical interconnection (fiber to dedicated IX port etc.) you will need small dedi (second NIC etc.).
Shorter AS path and many EU carriers prefer peering traffic over transit. Only getting IX won't help either - many ISPs on i.e. DECIX or AMSIX backhaul from very far away (Moscow, US, Africa...) - so you need to optimize for every single peer on each IX.
Quagga + SNMP monitoring with LibreNMS!
Bleargh, I was hoping that playing with prepends will do something for this (very coarse control, but at least something).
Some peers also have communities to control the routes.
Just remember that prepends affect all upstream peers, you can use bgp communities but not all providers support it. More PoPs more problems. The best option is to pick providers whith more or less the same upstreams
I see. Well, good thing one of the main objectives for the project is just to gain some experience with BGP, building a solid low-latency global anycast isn't 100% necessary ^_^
Anycast is about low latency
In some cases it's about failover and availability. Low latency is nice, but not the only reason why people could want to do anycast.
You can do HA on unicast and anycast without low latency makes no sense. Anyway as long as your provider offers bgp communities and good upstream blend it should be easy to optimize.
Well, after a few hours of troubleshooting, I finally got two Quagga instances on VMs to exchange routes. Actually the main problem was figuring out that I had to set up QEMU vlans rather than using tap devices with ip forwarding. Though I'm still not sure why the latter doesn't work for BGP when HTTP works fine that way o_O.
BGP works on tcp packets sent with ttl 1. So it only works on directly connected routers (i.e. on the same VLAN, not going through any intermediate routers).
For BGP between routers with different ASn you could use ebgp-multihop, then there can be intermediate routers (i.e. ttl of the BGP packets is more than 1). But this is used only in special cases.
Oh I see now. Now that I think of it, I remember reading something along these lines -- it's a security measure.
Not only security. You technically cannot add routes to a gateway with which you are not in the same network*
*You can do that, but you need an IGP like OSPF which will add the route for this gateway. BGP will then resolve the route and add the correct next hop.