Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


How much memory is needed for the Clearnet BGP table?
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

How much memory is needed for the Clearnet BGP table?

jerry_mejerry_me Member
edited June 2021 in Help

I know that on the Internet, some service providers offer different options, full table, partial table and no route, what is the difference between these?

I know it takes a lot of memory to use a full table, but how much?

I tried to collect the full table from BuyVM and it shows that only 70 MB of memory is needed, while many blog on the web say that 512M minimum, why is this?

R0qMH1.png

What features will I lose if I choose No route?

This is the first time I've asked for help here, so if there are any irregularities, please let me know, thank you.

Comments

  • cybmp3cybmp3 Member
    edited June 2021

    2.5GB in theory but it makes the router very unstable, In practice, 4GB is required.
    you just create a tuunel from buyvm to your home, and then use your own high-performance router/computer to receive BGP routing information.

  • tomletomle Member, LIR

    I've run full table on a 1GB Debian VM for both v4 and v6, it works but you'll probably not want to run other services on it.

  • jerry_mejerry_me Member
    edited June 2021

    @cybmp3 said:
    2.5GB in theory but it makes the router very unstable, In practice, 4GB is required.
    you just create a tuunel from buyvm to your home, and then use your own high-performance router/computer to receive BGP routing information.

    Thank you very much for your reply, I only care about the IPv6 table, and from what I can see when I run birdc show memory, he shows that it only takes 70MB, even if I take the IPv6 table in at the same time, it only takes 241MB, where do I get the idea that he needs 2GB of memory?

    @tomle said:
    I've run full table on a 1GB Debian VM for both v4 and v6, it works but you'll probably not want to run other services on it.

    I could consider it as an entry point and other high performance machines to establish VXLAN/WireGuard tunnels with him, using BGP Confederation or iBGP internally.

    It would be more memory efficient to choose No Route, but what features would lose?

  • vm_jevm_je Member

    Size of the table complete depends on how the bgp table data structure is implemented.

    Bird runs fine with 512MB RAM (table use less than 100MB) bgpd in frr requires at least 4GB RAM because it use data structures to speed up the calculation process but which needs additional ram for bgpd and zebra. Mikrotik can mange it with 200MB.

    So, memory usage depends on used bgp daemon.

    But this has nothing to do with fully, partial or default / no route table. Full table or default route having impact of your routing structure. You do not need full table with one upstream. Thus, default route is enough. If you have multiple upstreams full table makes sense because you can do routing calculation based on as-path lengths.

    Partial is used to reduce the costs. Mostly it doesn't contains expensive routes to eyeballs or are restricted to local routes.

  • @vm_je said:
    Size of the table complete depends on how the bgp table data structure is implemented.

    Bird runs fine with 512MB RAM (table use less than 100MB) bgpd in frr requires at least 4GB RAM because it use data structures to speed up the calculation process but which needs additional ram for bgpd and zebra. Mikrotik can mange it with 200MB.

    So, memory usage depends on used bgp daemon.

    But this has nothing to do with fully, partial or default / no route table. Full table or default route having impact of your routing structure. You do not need full table with one upstream. Thus, default route is enough. If you have multiple upstreams full table makes sense because you can do routing calculation based on as-path lengths.

    Partial is used to reduce the costs. Mostly it doesn't contains expensive routes to eyeballs or are restricted to local routes.

    If it's a single upstream, then the full table doesn't make sense, if it's multiple upstreams, then I have the need to pick a path and the full table makes sense.

    Thanked by 3cybmp3 dfroe tomazu
  • cybmp3cybmp3 Member
    edited June 2021

    @jerry_me said:

    @cybmp3 said:
    2.5GB in theory but it makes the router very unstable, In practice, 4GB is required.
    you just create a tuunel from buyvm to your home, and then use your own high-performance router/computer to receive BGP routing information.

    Thank you very much for your reply, I only care about the IPv6 table, and from what I can see when I run birdc show memory, he shows that it only takes 70MB, even if I take the IPv6 table in at the same time, it only takes 241MB, where do I get the idea that he needs 2GB of memory?

    @tomle said:
    I've run full table on a 1GB Debian VM for both v4 and v6, it works but you'll probably not want to run other services on it.

    I could consider it as an entry point and other high performance machines to establish VXLAN/WireGuard tunnels with him, using BGP Confederation or iBGP internally.

    It would be more memory efficient to choose No Route, but what features would lose?

    Actually BGP table size is not memory usage.In 2008 cisco said 1gb is require(https://www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/12512-41.html).The BGP table size in 2021 is now 4 times larger than in 2008.(https://bgp.potaroo.net/)
    And clearnet is a FAKENEWS.(because china telcom is very very very busy with 1.3billion people that there is no additional capacity to attract overseas networks. And the most important is BGP has RPKI and other security measures. )(a comment form dirtynet).

  • tomazutomazu Member, Host Rep

    You need to read about FIB, RIB and possibly TCAM (Ternary Content Addressable Memory) and BGP to have the full picture.

    Full BGP is needed only for certain aspects and if you need visibility up to /24 etc.

    Probably a midrange VPS would be sufficient to keep the BGP table & updates, but the difficult part is the routing/forwarding of packets itself with a huge "lookup"-table and this is were TCAM comes/came into play.

  • vm_jevm_je Member

    @jerry_me said: If it's a single upstream, then the full table doesn't make sense, if it's multiple upstreams, then I have the need to pick a path and the full table makes sense.

    That's what i have written. Do you like to repeat my words?

    @tomazu said: Probably a midrange VPS would be sufficient to keep the BGP table & updates, but the difficult part is the routing/forwarding of packets itself with a huge "lookup"-table and this is were TCAM comes/came into play.

    TCAM isn't important for a vps. TCAM is mostly implemented in hardware. VPP can handle a lot of packets but does not need any TCAM.

    Thanked by 1tomazu
  • tomazutomazu Member, Host Rep

    @tomazu said: Probably a midrange VPS would be sufficient to keep the BGP table & updates, but the difficult part is the routing/forwarding of packets itself with a huge "lookup"-table and this is were TCAM comes/came into play.

    TCAM isn't important for a vps. TCAM is mostly implemented in hardware. VPP can handle a lot of packets but does not need any TCAM.

    yes, that is true - but why would you want to to run full BGP if not for routing and/or pushing/forwarding a lot of packets with optimized routing? Of course you can have your route reflector and/or route-server as a carrier or IXP (or even for research purposes), but the OP seemed to want to know about what features and not full BGP would bring.

  • vm_jevm_je Member

    VPP can handle a lot of traffic on top of commodity hardware. It does not need any special ASICs and is far enough for handling multiple 10Gs of bandwidth and dozens of mpps.

    https://www.ciscolive.com/c/dam/r/ciscolive/us/docs/2017/pdf/DEVNET-1221.pdf
    https://wiki.fd.io/view/VPP/What_is_VPP?

    And even without VPP you can easily push 10G(s) through the Linux Kernel.

    But all of that has nothing do with amount of bandwidth or the question. You can run default route with 100Gs of traffic but you can need full table with a few Mbps.

    Maybe you haven't ready my first post.

    For you:

    @vm_je said: But this has nothing to do with fully, partial or default / no route table. Full table or default route having impact of your routing structure. You do not need full table with one upstream. Thus, default route is enough. If you have multiple upstreams full table makes sense because you can do routing calculation based on as-path lengths.

  • tomazutomazu Member, Host Rep
    edited June 2021

    VPP is extremely fast at packet forwarding and pushing 10G on Linux is possible even without kernel bypass (CloudFlare was doing this already a long time ago, mitigating SYN-floods etc), but is there a robust BGP (Bird, quagga/zebra etc.) implemenation with VPP that can be deployed in production with multiple 10GE, 40GE or even 100GE ports and at least two BGP peers? Last time I checked the results with Microtik etc. were nice, but still not on par with Cisco (hardware), Arista & Co.

  • @cybmp3 said:
    2.5GB in theory but it makes the router very unstable, In practice, 4GB is required.
    you just create a tuunel from buyvm to your home, and then use your own high-performance router/computer to receive BGP routing information.

    I'm planning this too, but I don't know what kind of tunnels would be better, VXLAN or whether to set up sessions on the VPS?

  • stratagemstratagem Member, Host Rep

    @tomazu said:
    VPP is extremely fast at packet forwarding and pushing 10G on Linux is possible even without kernel bypass (CloudFlare was doing this already a long time ago, mitigating SYN-floods etc), but is there a robust BGP (Bird, quagga/zebra etc.) implemenation with VPP that can be deployed in production with multiple 10GE, 40GE or even 100GE ports and at least two BGP peers? Last time I checked the results with Microtik etc. were nice, but still not on par with Cisco (hardware), Arista & Co.

    Not free, but TNSR on the correct hardware will see multiple 10Gbps/100Gbps with multiple peers.

    Thanked by 1tomazu
  • vm_jevm_je Member

    @tomazu said: VPP is extremely fast at packet forwarding and pushing 10G on Linux is possible even without kernel bypass (CloudFlare was doing this already a long time ago, mitigating SYN-floods etc), but is there a robust BGP (Bird, quagga/zebra etc.) implemenation with VPP that can be deployed in production with multiple 10GE, 40GE or even 100GE ports and at least two BGP peers? Last time I checked the results with Microtik etc. were nice, but still not on par with Cisco (hardware), Arista & Co.

    DANOS developed and used by AT&T

  • tomazutomazu Member, Host Rep

    @vm_je said:

    @tomazu said: VPP is extremely fast at packet forwarding and pushing 10G on Linux is possible even without kernel bypass (CloudFlare was doing this already a long time ago, mitigating SYN-floods etc), but is there a robust BGP (Bird, quagga/zebra etc.) implemenation with VPP that can be deployed in production with multiple 10GE, 40GE or even 100GE ports and at least two BGP peers? Last time I checked the results with Microtik etc. were nice, but still not on par with Cisco (hardware), Arista & Co.

    DANOS developed and used by AT&T

    you mean AT&T's version of Vyatta :-D !?

  • vm_jevm_je Member

    Yes, sure. Has integrated support for vpp/dpdk.

  • cybmp3cybmp3 Member

    @jerry_me said:

    @cybmp3 said:
    2.5GB in theory but it makes the router very unstable, In practice, 4GB is required.
    you just create a tuunel from buyvm to your home, and then use your own high-performance router/computer to receive BGP routing information.

    I'm planning this too, but I don't know what kind of tunnels would be better, VXLAN or whether to set up sessions on the VPS?

    GRE tunnel

  • jsgjsg Member, Resident Benchmarker
    edited July 2021

    @vm_je said:
    TCAM isn't important for a vps. TCAM is mostly implemented in hardware. VPP can handle a lot of packets but does not need any TCAM.

    No, TCAM is hardware. Yes, one could implement it in software but to do so would be idiotic because it would basically loose the advantage of TCAM.

    @jerry_me (and others)

    Back to the topic question: About 64 MB; that's what is needed - BUT:

    • one virtually always doesn't just want the IP range-AS table but rather additional data too
    • just to store the table isn't enough for effective operation, one also wants fast "indexing"
    • one usually wants/needs additional complexity such as e.g. evaluating different options against a set of criteria.

    So, the problem isn't size, it's how to store and access the data and structures.
    Plus there is "never change a working engine" as well as and hand in hand with an aversion against changes in critical infrastructure, so both the manufacturers and their customers tend to prefer/stick to what works.
    Example: tries are an elegant, effective, and fast structure for routing data, but they are also relatively new. So companies like Cisco are not likely to throw away what they have and maintained and know to work well. linux on the other hand is kind of a living experiment and quite ready, sometimes even eager, to implement and play with new things, e.g. in the firewall.
    Btw, even the the most expensive TCAM and ASICs is much cheaper than engineering hours or, God forbid, a faulty system (which might kill the company).

    That's why the answers range from '100 MB' to '4 GB'. Rule of thumb: 1 - 4 GB if you need a professional solution and reliability, 100 MB - 500 MB for a home, hobby, play solution.

    Thanked by 1tomazu
  • MaouniqueMaounique Host Rep, Veteran
    edited July 2021

    @tomazu said: Microtik

    "Cloud" ones are very weak, one out of two failed after a little more than a year and the other one started to have same symptoms, losing config and reboot loop. A reflash works, but you may never know when it will fail again. Cheap but poor quality, I only had 2 links 10 gbps...
    This is a shame because my RB ones I used at home through many years never failed.

    Thanked by 1tomazu
  • CristianDCristianD Member, Host Rep

    Hello,

    For full table from 1 provider 1M routes you need 2GB RAM but you cant run nothink else on it (stats, fastnetmon...etc), with 2 providers, 1M each, 2M Total 4GB minimum recomanded.

    If you use same router as firewall, with firewall rules you need 8GB, the much firewall rules you have on the router the usage will be bigger.

    The cheapest solution is server runing vyos on it with grafana, fastnetmon ...etc is working fantastic, if you plan to buy a mikrotik for full tables, i dont recomand, it will dye fast or freez, cuz there ar way to manny bugs on it witch mikrotik dose not fix even if they sell new products, they came broken (OS) unable to use them as they mention on the product description, much better Vyos instead of mikrotik and you dont have limits :) you can add as much Ram yo uwhant and make your way.

    Regarding the default table and full table, if you use full table you will have better routes ( Customer > Your Router (Server)... and you can play with.

    GO from the beginning with full table, you will see why in the future and you will avoind wasting money by changing routers.

    Dont make same mistakes i made, wasted more then 3-5$k because some opinions come from people who have not played with bgp or they did but "home use".

Sign In or Register to comment.