Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Migrating from Mikrotik to 6wind. First impressions
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Migrating from Mikrotik to 6wind. First impressions

jmginerjmginer Member, Patron Provider
edited February 2022 in General

Hi all,

we're migrating our Mikrotik routing core to 6WIND Virtual Border Router.

Initially we see that the command nomenclature is totally different and the learning curve is slightly more complex in 6wind compared to Mikrotik.

However, the documentation is very good: https://doc.6wind.com/turbo-router-3/latest/turbo-router/

We have started a basic configuration:

  • VLAN
  • GRE
  • Static routes
  • BGP
  • Filters / prefix-lists
  • sflow
  • Fast-path

These days we are going to be fighting with BGP communities....

Our initial impressions, we see an amazing stability.
With a stress test 100% UDP, we have filled the 2 x 10 Gbps transits we have connected (GTT + Telia) for 20 minutes with an impressive result of 0 packet loss and less than 10% CPU consumption (E3-2690 v3).

The installation was performed on a Proxmox VM (CPU Host mode) and an Intel-XL710-BM2 network card (40 Gbps) configured in PCI-Passthrought.

We will probably migrate to production in the next weeks.

Comments

  • dfroedfroe Member, Host Rep

    Thanks for sharing your test drive with 6wind.

    Just out of curiousity, did you run MikroTik RouterOS v6 before or did you already try the rather new v7?

    The core of RouterOS v6 is decades old and many features like IPv6 or BGP are more added in a dirty way than smothly built-in. Resulting in several well-known quirks.

    After many years of development, RouterOS v7 promises huge improvements over the ancient v6 as the core os was re-written with many important modern features designed in. However I personally was not brave enough to give v7 a try in the field, yet. :)

  • jmginerjmginer Member, Patron Provider

    @dfroe said:
    ...

    Yes, I was testing ROS v7, but when I saw that they had changed the command nomenclature of the BGP filters (I have about 60-70 BGP filters in ROS v6), I thought that if I had to learn to use a new product from 0, it was worth making the switch to 6wind which apparently is a far superior product.

    Thanked by 2dfroe pan_ia0_net
  • dfroedfroe Member, Host Rep

    Feature-wise it looks like 6wind includes all well-known and typically used features incl. those you already mentioned, IPsec etc.

    Have you already seen whether there is some kind of scripting engine in 6wind as well?

    I don't love the internal scripting in RouterOS but it does its job.
    For example to periodically ping various destinations over different paths, meassuring packet loss over time and for example to automatically adjust bgp attributes or increase routing costs if loss within last hour exceeds a certain threshold.

    Don't know if a stable and api/scripting-friendly routing system might be a niche but would be very helpful in the field.

    Thanked by 1pan_ia0_net
  • jmginerjmginer Member, Patron Provider

    6wind uses NETCONF> @dfroe said:

    Have you already seen whether there is some kind of scripting engine in 6wind as well?

    6wind uses NETCONF, which we have to integrate to automate the re-routing when we receive DDoS attacks.

    I don't know if we will use flowspec or communities... we are still learning how to use it.

    Thanked by 2dfroe pan_ia0_net
  • What's the benefit of 6wind vs. something like FRRouting?

    Thanked by 1pan_ia0_net
  • FRR is just the control plane (runs BGP, OSPF, etc.) while the actual routing of the data packets is done by something else (for example linux kernel, *bsd kernel, DPDK, hardware). For example VyOS and DANOS both use FRR as their control plane daemon (I think 6WIND now uses their own implementation for BGP, etc.). While VyOS just uses the linux kernel to route the packets, DANOS and 6WIND both use DPDK to route the packets.
    With DPDK you can easily route more than 10G line-rate on a rather modest system while routing using the normal routines in the linux kernel will struggle routing 10G of 64-byte packets (but 10G of more normal sized packets is not really an issue these days).

    Thanked by 2pan_ia0_net jmginer
  • sezingsezing Member, Host Rep

    @jmginer ,

    We have been using Mikrotik RouterOS for almost 3 years to prevent DDoS. Mikrotik with a lot of firewall rules causes CPU spikes during PPS attacks. How does 6wind Handels DDoS?

    I would like to test it 6wind too.

    Thanked by 1pan_ia0_net
  • jmginerjmginer Member, Patron Provider

    @sezing said:
    @jmginer ,

    We have been using Mikrotik RouterOS for almost 3 years to prevent DDoS. Mikrotik with a lot of firewall rules causes CPU spikes during PPS attacks. How does 6wind Handels DDoS?

    I would like to test it 6wind too.

    OVH Uses 6WIND For DDoS Protection
    https://www.6wind.com/ovh-6wind-ddos-protection/

    Thanked by 1pan_ia0_net
  • stratagemstratagem Member, Host Rep

    Looks interesting, like a much more mature TNSR. Care to share what the approximate pricing was? TNSR was/is $500/year but the product isn't production ready imho.

    Thanked by 1pan_ia0_net
  • jmginerjmginer Member, Patron Provider

    @stratagem said:
    Looks interesting, like a much more mature TNSR. Care to share what the approximate pricing was? TNSR was/is $500/year but the product isn't production ready imho.

    The 6wind licenses price depends on the amount of traffic you move, it is better to talk to them directly. They are not expensive...

  • MaouniqueMaounique Host Rep, Veteran

    @jmginer said: With a stress test 100% UDP, we have filled the 2 x 10 Gbps transits we have connected (GTT + Telia) for 20 minutes with an impressive result of 0 packet loss and less than 10% CPU consumption (E3-2690 v3).

    So I would figure that 40-60 Gbps would not exceed the CPU capacity to handle it.
    UDP, though, would not test the CPU much, could you test with a syn flood?

    Thanked by 1pan_ia0_net
  • rm_rm_ IPv6 Advocate, Veteran

    This sounds like something even more proprietary than Mikrotik, i.e. migrating in a wrong direction.

  • @Maounique said:

    @jmginer said: With a stress test 100% UDP, we have filled the 2 x 10 Gbps transits we have connected (GTT + Telia) for 20 minutes with an impressive result of 0 packet loss and less than 10% CPU consumption (E3-2690 v3).

    So I would figure that 40-60 Gbps would not exceed the CPU capacity to handle it.
    UDP, though, would not test the CPU much, could you test with a syn flood?

    For just routing it doesn't really matter if it's UDP or TCP. And ddos-mitigation should probably be on-par, since you are essentially just evaluating a few stateless rules to drop the bad traffic (actually in most XDP/DPDK showcases I have seen that was were it really shined).

  • They have any free versions for small home networks?

    Thanked by 1crilla
  • stratagemstratagem Member, Host Rep

    @doughnet said:
    They have any free versions for small home networks?

    If they don’t, TNSR does which also uses DPDK.

  • MaouniqueMaounique Host Rep, Veteran
    edited February 2022

    @airmass1 said: For just routing it doesn't really matter if it's UDP or TCP.

    True, but i am talking about pps, UDP usually means way less pps than a syn flood at the same pipe size. Also, the syn flood would have other characteristics which would tax the CPU way more than UDP.

  • jmginerjmginer Member, Patron Provider
    edited February 2022

    @Maounique said:

    @airmass1 said: For just routing it doesn't really matter if it's UDP or TCP.

    True, but i am talking about pps, UDP usually means way less pps than a syn flood at the same pipe size. Also, the syn flood would have other characteristics which would tax the CPU way more than UDP.

    I don't have any tool to do that kind of test. I'm sorry.
    When we receive an attack, we divert it to Voxility.
    Our goal is not to mitigate attacks with 6wind.

  • @Maounique said:

    @airmass1 said: For just routing it doesn't really matter if it's UDP or TCP.

    True, but i am talking about pps, UDP usually means way less pps than a syn flood at the same pipe size. Also, the syn flood would have other characteristics which would tax the CPU way more than UDP.

    Minimum UDP packet size is smaller than TCP though, so UDP would give you more pps if you fill up the pipe. Or do you mean things like DNS/NTP/etc. amplification attacks?

    I guess the real question is what the UDP packet size was - was it 0 byte packets or something approximating IMIX?
    I have seen plenty of DPDK showcases that easily do more than 10 Mpps in a single core on quite low-end hardware, so I easily can see it do quite a bit more than dual 10G line-rate without really breaking a sweat.

  • @stratagem said:

    @doughnet said:
    They have any free versions for small home networks?

    If they don’t, TNSR does which also uses DPDK.

    Thanks. Didn't TNSR have huge limitations on homelab usage? I know that Sophos did some lame things like that.

  • stratagemstratagem Member, Host Rep

    @doughnet said:

    @stratagem said:
    If they don’t, TNSR does which also uses DPDK.

    Thanks. Didn't TNSR have huge limitations on homelab usage? I know that Sophos did some lame things like that.

    IIRC you couldn't in place update you had to blow it away and reinstall to get newer versions but that was about it.

  • MaouniqueMaounique Host Rep, Veteran

    @airmass1 said: Or do you mean things like DNS/NTP/etc. amplification attacks?

    I think UDP attacks are exclusively amplification ones, has anyone seen another kind last years? I haven't.

  • jmginerjmginer Member, Patron Provider
    edited November 2022

    Hi, I received a private message from a user who found out that we are finally using Juniper MX204 and have stopped using 6wind. what is the reason?

    The main problem we had with 6wind was that IPv6 routing was not within the FastPath.

    We deliver /48 addressing to our customers, this implies that there are customers with hundreds of thousands of IPs running on their servers... it was causing us network stability problems.

    Incomprehensibly, while IPv6 uses NDP, the overloads caused by IPv6 also affected the IPv4 ARP and the router would crash, CPU was running at 100% and we had to flush the ARP table or even reboot the router.

    We contacted 6wind support, although they were aware of the problem, they could not give us an immediate resolution, so it was unfeasible to continue using it and then we migrated to Juniper MX204.

    Last month we integrated a second MX204 as a backup and we are delighted with the performance. Together with the Arista switches, they provide us with an outstanding network.

Sign In or Register to comment.