All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
mapping the internet (routing table collection)
Hi everyone - I apologize in advance for my lack of knowledge on this subject. I'm interested in monitoring as many complete routing tables as I can possibly get access to in an effort to gain a comprehensive view into peering relationships. I'm aware of RouteViews and similar projects that aggregate public routing tables but I'm interested in getting access to new routing tables beyond what these projects offer.
I also understand there are public route servers I can connect to but I'm not clear on how I would go about downloading the full bgp routing table from them (my understanding is the table is huge and would crush the server's CPU etc.).
Lastly, I've been reading about buying a /24, registering an ASN, and using BIRD / Quagga / exabgp as a BGP daemon on a VPS.
Would this BGP daemon have access to a complete routing table? If I spin up a bunch of VPSes around the world running Quagga would this give me access to many routing tables?
I found this post which seems to indicate this is possible but my BGP daemon would need to peer with another ASN? How would I pull that off? Is that the challenging part in all this? Finding peers?
Thanks for schooling me on the topic and pointing me in the right direction
Comments
Not really. You need a few gigs of RAM but most Cisco's ASA are just Intel Atoms (the better performing ones though) and they need to handle routing real time, all the time.
You need to learn it before you go ahead. Trust me, this will be the best things ever.
https://dn42.eu/
Simulate it there, run your own AS, announce your IPs, peer with IX or just some other random guys and learn.
and realize in the real world, many ISPs are dicks about peering. "buy my transit instead hahahaha"
Ok super helpful! dns24.eu looks awesome, will definitely practice there.
So the real world is all I care about. It seems the real challenge is finding peers? (please correct me if that's not the case). A few questions on this front:
Lastly, if I spin up a VPS w/ Vultr or DigitalOcean, how would I identify the list of possible peers available to me?
Thanks for your help!
just why do you need to map all bgp tables now? what is with everyone wanting to use bgp all of a sudden to "map" everything...just why do they need "map" anything that other services provide better?
nevermind super suspicious the person signed up just to ask this today:
Well, I did specifically register here to ask this question
I've been interested in this topic for quite some time but never had the resources (time/money) to actually attempt it.
Are there others pursuing a similar direction that are considered "bad actors"? Thanks for educating me so I can avoid heading down a similar path.
And to answer your question re: why. I find the relationship information in routing tables to be very interesting and at times indicative of how markets change (say Netflix begins peering w/ Apple or something to that affect). I have a data science background and I'm interested in performing analysis on these relationships in an attempt to understand what's happening in the "real world".
Hope this helps!
you should know registering today and this being your first discussion/comment is very suspicious as this has been seen in recent times too often someone wants to "map" bgp tables which in itself is very suspicious when someone first joins and this is the first you ask about.
hopefully you understand it now since the important parts are now bold and some italicized.
Yow! Zippy wants th' BIG BGP BURGER, but is afraid th' OSPF might go right back out!
lol! What is suspicious about aggregating / mapping bgp tables? More bold and italics please!!
That depends on your intentions.
I'm just trying to learn here. Can you give an example of bad intentions?
I'll defer to @William. I don't like to masturbate over routing, because I don't get paid for it- and he knows more than I've forgotten.
http://www.routeviews.org
Thanks @clouvider! Yes, I'm familar w/ RouteViews. I'd like to add to their dataset (create something more comprehensive in nature).
I was nodding along with your posts until I read this.
Oh well, good luck.
Thanks @vimalware. Could you explain why?
I recall, not 100% positive, that with Vultr you can either bring in your own AS or use a private one from them, and you can announce your own subnet. AFAIK this has to be done in a ticket, and there is some human touch involved; it is not automated.
Edit: https://www.vultr.com/docs/configuring-bgp-on-vultr
To be honest I don't really see the problem, seems like a cool project to do. Best of luck @cloudmapper!
I don't see what can you possibly add to their dataset heaving a handful of full tables from a few ISPs?
what's wrong with asking a question??? not like he's promoting some shit here. chill out!
@cloudmapper, I think you are doing a really neat project to learn and expand your horizons. Don't give a rats ass what @cubedata says, he has no idea wtf he is talking about just spoutting a bunch of dumb stuff his words don't mean shit. Frankly speaking, all he is saying is "suspicious", "suspicious", "your first day" without furthering articulating how are your intentions "suspicious"
He lacks ambitions to do cool projects like this one you are doing, and doesn't have the voracious appetite to learn like you. So he labels everything above his level of understanding(or what he considered normal learning of understanding) "very suspicious", because it took him quite a while to understand BGP routing.
@cloudmapper I currently do this sort of thing with Vultr, and even run my own modified Bird daemon to get data out programmatically easier.
The big issue you have is you will not see all routes from your direct BGP peers, you can see the full routing table but this is default/best routes, you will not see every route held without using BGP extensions like bgp add path.
Feel free to send me a PM maybe we can pull some resources together.
No idea, it is fairly worthless data in malicious ways, and even then it can be replaced by something as simple as traceroute (which will give the ASN... sometimes...).
If we stay with that example, this likely means nothing: Apple is not a content provider for Netflix (and even then it would be mirrored only once and is then in netflix), neither reverse, and none does end-user connectivity that would matter as transit/transit-peer.
Anyway from the tech side, you generally do not see what you have no path to - unless you have access to something via Apple or Netflix and record the route to the other side you have, simply, no data.
Peer does ultimately not mean free. Many gladly sell you transit via BGP, without IP blocks or customers to sell said transit to (you do not need IPs for that, generally) this is... fairly useless though.
No, as generally you only use a free (thus exchange or direct) peer for what they send you - you send only your and your customer routes; why would you send your private peering to ISP Z to IX A users for free usage. Would be dumb.
A transit provider will generally have a path to nearly any prefix and send you all (in case of a full table) which you can use with another ISP that does this and your IX/private peerings to use the shortest (or other regex like known reliability of path). BGP 1x1.
Well, what do you offer?
etc etc
Depends where. Depends on definition of "transit". Full transit from basically 10c to 150$ depending on location (eg. low = Amsterdam, LA or New York, med = Tokyo or Hong Kong, high = Brazil or South Africa + China direct at the top).
EU peering only (so a blend of multiple exchanges and backhaul through the ISP network basically, but no transit routes they pay for) can be had for few cent per Mbit.
No, month to month is rare for anything requiring a crossconnect to colo, VPS/dedicated with BGP for just a month yields the provider likely hating you also (setting up session for just 1mo is damn annoyance especially if done free as value service for customers).
Transit contracts range from 6mo to 10yr with clauses from very, very simple (eg. Cogent) to complex as hell (transit eg. Telmex and Claro, peering UPC and Comcast).