How do you handle SYN attack? Does your router/firewall becomes the bottleneck?

lowprofile

Hi,

Testet PFsense, Mikrotik CCR and Fortigate 1000C
Was not able to get a satisfied result.

Isseu: A "small" SYN attack on 10-20Mbit is enough to max out the CPU on router (instead of just routing the "attack" to end-device) - This is devastating due to everyone sitting behind the router gets "offline"

I can't be the only one who is challenged? Please share your thoughts! I would really appreciate it.

jackb

That's what, 60kpps?

Something is seriously wrong with your configuration. Even a regular 1gbit server without any significant tuning can cope with forwarding approx 100kpps, a router should be far higher.

HostSlick

@jackb said:
That's what, 60kpps?

Something is seriously wrong with your configuration. Even a regular 1gbit server without any significant tuning can cope with forwarding approx 100kpps, a router should be far higher.

Mikrotiks seem not really.... Good.

A client of ours used mikrotik ccr1036-8g-2sEM with 10Gbit Uplink and bgp session.
The said router itself was advertised by mikrotik to handle 28Gbit / 41mpps

BUT

Mikrotik CPU always spiking at 80/90% once traffic reached 1,3/1,4Gbit and customer getting packet loss, sometimes outages for like 5-10min

Config already optimized to save resources. No luck.
Also I know someone else who had a MK and experienced something like this too.

lowprofile

@HostSlick said:

@jackb said:
That's what, 60kpps?

Something is seriously wrong with your configuration. Even a regular 1gbit server without any significant tuning can cope with forwarding approx 100kpps, a router should be far higher.

Mikrotiks seem not really.... Good.

A client of ours used mikrotik ccr1036-8g-2sEM with 10Gbit Uplink and bgp session.
The said router itself was advertised by mikrotik to handle 28Gbit / 41mpps

BUT

Mikrotik CPU always spiking at 80/90% once traffic reached 1,3/1,4Gbit and customer getting packet loss, sometimes outages for like 5-10min

Config already optimized to save resources. No luck.
Also I know someone else who had a MK and experienced something like this too.

What router did you end up with?

Father_Michael

If you see high cpu usage by just routing you don't use fastpath on your ccr. Get someone involved that has knowledge and you won't suffer pl and cpu spikes. Before replacing it with Juniper MX gear I ran ccr > 4-5GBit/s without any spikes at all.

HostSlick

@lowprofile said:

@HostSlick said:

@jackb said:
That's what, 60kpps?

Something is seriously wrong with your configuration. Even a regular 1gbit server without any significant tuning can cope with forwarding approx 100kpps, a router should be far higher.

Mikrotiks seem not really.... Good.

A client of ours used mikrotik ccr1036-8g-2sEM with 10Gbit Uplink and bgp session.
The said router itself was advertised by mikrotik to handle 28Gbit / 41mpps

BUT

Mikrotik CPU always spiking at 80/90% once traffic reached 1,3/1,4Gbit and customer getting packet loss, sometimes outages for like 5-10min

Config already optimized to save resources. No luck.
Also I know someone else who had a MK and experienced something like this too.

What router did you end up with?

None.
We do bgp for him on our Cisco now

HostSlick

@Father_Michael said:
If you see high cpu usage by just routing you don't use fastpath on your ccr. Get someone involved that has knowledge and you won't suffer pl and cpu spikes. Before replacing it with Juniper MX gear I ran ccr > 4-5GBit/s without any spikes at all.

Which ccr you had?
fastpath was enabled in customers case.

jh_aurologic

Personally, I wouldnt use Mikrotik for any professional usage, whenever you have to expect traffic spikes or ddos attacks. Either get a Layer 3 switch or a real hardware router for routing, performance will be much better as no software routing is involved.

Father_Michael

@HostSlick said:

@Father_Michael said:
If you see high cpu usage by just routing you don't use fastpath on your ccr. Get someone involved that has knowledge and you won't suffer pl and cpu spikes. Before replacing it with Juniper MX gear I ran ccr > 4-5GBit/s without any spikes at all.

Which ccr you had?
fastpath was enabled in customers case.

CCR1036-8G-2S+EM and CCR1072-1G-8S+.
Maybe enabled but not used. Maybe you had firewalling enabled or similar. The cpu peaks only occur when the packets are not passed through fastpath.

Still I agree on "don't use it for production" though

@combahton_it said:
Personally, I wouldnt use Mikrotik for any professional usage, whenever you have to expect traffic spikes or ddos attacks. Either get a Layer 3 switch or a real hardware router for routing, performance will be much better as no software routing is involved.

100% agreed on the real hardware router part.

SplitIce

None of those are appliances targeted towards mitigation. Although I would expect more forwarding capacity than 60Kpps.

Given that SYN floods regularly hit some pretty high PPS mitigating them on consumer hardware is likely not very feasible.

jsg

@combahton_it said:
Personally, I wouldnt use Mikrotik for any professional usage, whenever you have to expect traffic spikes or ddos attacks. Either get a Layer 3 switch or a real hardware router for routing, performance will be much better as no software routing is involved.

That statement doesn*t make sense because Mikrotik use quite a few different processors over their product range - and often quite powerfull ones (for a given product). Similarly one can't say "Oh, that's just a (e.g.) dual core Mips" because the processors used by Mikrotik often have built-in data plane support.

But their router OS can be a problem unless one really groks it.

Clouvider

The solution here will be to mitigate before it touches this soho router; take a remote tunel with @SplitIce or have your datacenter help you. Clouvider for example offers free protection to all Customers.

lowprofile

@Clouvider said:
The solution here will be to mitigate before it touches this soho router; take a remote tunel with @SplitIce or have your datacenter help you. Clouvider for example offers free protection to all Customers.

Thats not the correct approach for a small traffic flood - You wont kick in a BGP re-route because of 10-20mbit of UDP flood/SYN - that should just be forwarded to end device and not causing everything behind to go offline

I am testing thinks out now, and hopefully will share some experience

Clouvider

well, I disagree; if you can't handle it - push it elsewhere. Or invest in the capability to handle it in-house.

@lowprofile said: You wont kick in a BGP re-route because of 10-20mbit of UDP flood/SYN

Even if you were to set a threshold to, well, I don't know, 1000 pps ?

jsg

@lowprofile said:
Thats not the correct approach for a small traffic flood ...

Nope, that's exactly the right approach. I understand your personal policy of just eating it up at some, presumably more powerful, internal system, but @Clouvider is right anyway for diverse reasons incl. the "golden rule" to have trouble taken care of upstream, especially when you are on customer premises. Another reason being that routers and firewalls tend to not be among the most performant systems at most customers while upstream they usually are much beefier. Yet another reason is that most customers look for e.g. crypto performance ("Can this box do AES near line rate?") but not for firewall performance.

inetzero

@lowprofile said: 20Mbit

Details here are pretty scarce, but I would ask myself the following:

• are the SYN attacks targeted at your router or something behind it. From the description it seems like the attack is destined to one of your router's interfaces. If the destination ports match well know services, either add a firewall (permit only specific, save destinations), or close those service all-together (Mikrotik has default ports open for telnet, http and https, if memory serves me well).
• even with default ports open and no firewall, 10-20Mbps should not be an issue. Mikrotik also provides fast path that "allows to forward packets without additional processing in the Linux kernel. It improves forwarding speeds significantly". More details here.

Now, there's also the other side of the spectrum, where you have a couple of hundred of megs traffic. It this saturates your upstream bandwidth, there's really not much you can do, other than to rely on your provider offering some sort of scrubbing solution(Arbor Networks is an interesting example here), or have some Scrubbing as a Service provider (Akamai, Imperva, you name it).

lowprofile

@jsg said:

@lowprofile said:
Thats not the correct approach for a small traffic flood ...

Nope, that's exactly the right approach. I understand your personal policy of just eating it up at some, presumably more powerful, internal system, but @Clouvider is right anyway for diverse reasons incl. the "golden rule" to have trouble taken care of upstream, especially when you are on customer premises. Another reason being that routers and firewalls tend to not be among the most performant systems at most customers while upstream they usually are much beefier. Yet another reason is that most customers look for e.g. crypto performance ("Can this box do AES near line rate?") but not for firewall performance.

I don't agree with you. We are talking about 20mbit SYN flood. The pipe itself is minimum 1G. One need a proper router which can do some line rate forwarding.

@inetzero said:

@lowprofile said: 20Mbit

Details here are pretty scarce, but I would ask myself the following:

• are the SYN attacks targeted at your router or something behind it. From the description it seems like the attack is destined to one of your router's interfaces. If the destination ports match well know services, either add a firewall (permit only specific, save destinations), or close those service all-together (Mikrotik has default ports open for telnet, http and https, if memory serves me well).
• even with default ports open and no firewall, 10-20Mbps should not be an issue. Mikrotik also provides fast path that "allows to forward packets without additional processing in the Linux kernel. It improves forwarding speeds significantly". More details here.

Now, there's also the other side of the spectrum, where you have a couple of hundred of megs traffic. It this saturates your upstream bandwidth, there's really not much you can do, other than to rely on your provider offering some sort of scrubbing solution(Arbor Networks is an interesting example here), or have some Scrubbing as a Service provider (Akamai, Imperva, you name it).

It was targeted a server behind firewall. I have given up Mikrotik and trying some other vendors and "real" routers which can perform at line rate.

This issue has nothing to do with saturation. We are talking about an advanced syn attack - not a volume attack.