All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Hardware Routers with Vyatta/VyOS or Similar?
randvegeta
Member, Host Rep
When it comes to configuring routers, for me, nothing is easier than the CLI of VyOS. Coming from a Juniper background, it was easy to pick up, and really it is my preferred router for smaller to medium sized networks.
Now I'm looking at building a multi 10G network and need decent routing capacity.
On my multi-1G setups, the hardware I've used has been more than sufficient to handle the traffic (using pretty decent Intel network cards and a high clock speed 4-core CPU) and that can handle full line rate with tiny packets without much issue.
Somehow I think a 10G setup may be more demanding.
I was looking at the Ubiquiti EdgeRouter Infinity , but I somehow doubt it's claimed specs. I have a number of the EdgeRouter Pros in a test lab and they really struggle handling lots of small packets (just normal traffic!). The performance I get from my Xeon E3 based routers tend to better than the EdgeRouter and so I saw no benefit of buying and using them.
A 10G Juniper router would do fine I suppose, but I'm trying to keep my budget to within US$600-$700 for the router, and I will have 2-3 routers (for redundancy). I don't think I can get a decent 10G router from Juniper in that price range. Even the shitty J series cost more, and they are terrible!
Anyone have any experience running a 10G VyOS DIY router? Or know of any decent routers that have Vyatta, or Vyatta like CLI that are available for within my target price range.
I'm open to 2nd hand equipment if that makes it easier to find.
Comments
Won't work linerate for small PPS, might just do while you're not attacked. Won't give you the same reliability as Juni, not even close, but I guess you have to compromise for the budget.
Line rate 10G with small packets is tough on software routers. Probably not going to find anything under $1K that will be able to do much.
In regards to the Ubiquity, the last model our Customer had hands on, and we tried to support, was loosing BGP, increasing latency or heaving heavy packet loss when a full table was sent and the Customer logged in to the web interface (just logged in).
That's despite it was supposedly rated for 10G, 5m PPS IIRC.
I wouldn't go Ubiquity based on past experience.
Drop a Linux box with Intel network Cards instead, but really if you need 10s Gbps you should figure out a way to increase this budget to be able to put up a proper router instead.
I think one of the biggest problem with software routers is that virtually all of their routing is handled by a single core no matter how tricked out the device is- and there's little information as to when this might change.
I've had great experience with Ubiquity for indoor/outdoor wireless, and not much else. I second the idea of getting a proper router- because you're going to be caring and feeding for this longer than it really deserves. Pay up front, or pay later.
I figured as much.. shame..
Anyone know of any Vyatta based or like routers that can properly handle 10G?
Funny because the whole reason I got interested in Vyatta was because we had a catastrophic failure of our Juniper router. Luckily I had an old Cisco laying around that could handle a few 100Mbit for a short time. The fastest way to get back online was to use a software router, and Vyatta seemed to make the most sense. I've since not had a single DIY vyatta router fail on me, but even if I did, I was able to setup multiple routers for redundancy.
Granted it was a 'J' series Juniper Router (which is like the budget end) but it still cost a few grand, and the DIY routers both outperformed and cost less.
I have since not really put much stock in so called 'Enterprise' grade routers. At least not the ones that are less than $5k....
I mean... You're comparing Apples with Oranges here. You're comparing a computer with two or more network cards to a purpose build equipment.
Huh, where do I start...
Why you should use Juniper MX over Software routers:
(...) I can go on for hours.
And yes, even the most redundant routing devices fail. Precisely why you pay for support, precisely why you deploy them in the most resilient way possible, i.e. with dual routing engines, fpcs, pics, redundant power suppliers, etc. to minimise chance of a 'catastrophic failure'. The on And yet you're still advised to deploy them in pairs.
I simply don't see a PC handling linerate any time soon and that means that you're unable to handle a DoS attack made out of $7 Kimsufi, and that in turn means that you're not capable of hosting anything serious - no offence, but FNM won't detect an attack spread evenly over a /22 that is very much capable of taking you down but still well under any threshold you can configure in production.
In regards to even using a J series in a DC - it's not built for DCs - it's a branch router. On top of that EOLed 4 years ago that had close to none HA functionality, so I don't understand how you can even take on a job comparing Juniper routers in 2017 with Software routers with your experience ending on J series, especially in a DC...
If there was Juniper wouldn't be charging £10k+ for MX104.
Erm, you can spread interrupts to a number of cores, depending on how many queues you have, with Intel network cards, it still doesn't help the case much. If you get hit with a couple M pps of small packets to a various IPs, especially IPv6, there's just no chance of the router surviving it. I've seen PC routers used by some even with 2x 10 core CPUs and Intel network cards kernel panic with a single attacker from OVH -> Customer flood over IPv6 to a couple of addresses only just after severe packet loss followed by a couple of BGP drop outs and re-connects. It just ain't designed for DC routing, simple as that.
That should have told you something about x86/64 processors. 4 cores at high clock speed for a couple of 1 Gb pipes? There are architectures doing that at 1 GHz or even lower (which translates to system reliability and lower power usage). And yes, there is a reason for the big network corporations to have their own ASICs (which make sense even with n x 10 Gb).
As I don't know much about vyatta I can't offer much in terms of concrete products but some guiding lines (and factors):
Which architectures does vyatta incl. all sub components run on? (And no, if it's linux based that doesn't translate to "runs on anything linux runs on")
x86/64 is a "business" CISC architecture. For routing and similar other architectures, particularly RISC ones are preferable
What you want is a processor with many (and modern) SerDes lines and, if any possible, some hw engines e.g. for 64 bit (or. min 32 bit) wide matching, and a reasonable L2 cache and fast memory interface. (Do not go for "has built in dual 10 Gb interface" but for raw serdes and plenty of them)
Be sure the architecture/manufacturer provide good/full and free linux BSP (assuming vyatta is linux based).
Being at that, forget anything not linux based. Support for non linux is rare and 2nd tier and much less tested. (No, I'm not a linux fan but that is the reality).
Think again and very hard about your price range. Below 1 k$ you won't get much but stone-age 2nd hand or utterly unprofessional hobbyists crap in terms of n x 10 Gb (n reasonably being in the range 4..8)
If you managed to get something professional and reliable with, up to 8 x 10Gb with 2 x 10Gb already installed or on the board below 2.5 k$ you'd be a lucky man.
From what I see, it's not worth the effort unless you are very knowledgeable and experienced. You'll hardly save a lot (compared to, say, a 2nd hand brand name router). And that is even more true if you need only one or a couple of those. If you needed 50 or even hundreds the story would change (but so would it with brand name stuff).
And don't forget that with brand name products you get a (more or less) nice and professional interface and you'll find people knowing it much easier.
Finally, a "trick" approach that might be more easily and cheaper feasible: Don't ask for a router but for a n x 10 Gb switch with open source firmware, drivers, software - and turn them into a router and/or firewall.
That's still quite demand in terms of know-how and experience for most people but price-wise it's a different ball-park...
>
Yes, that's what I meant with brand-name routers.
"Somehow routes n * 10Gb" does not equate to "can be reasonably used in a DC".
x86/64 architecture was always "business optimized". There are many many details that intel (even today) doesn't do so well with applications in switching, routing, industrial control, etc. It simply wasn't intels target. At some point they introduced quick assist and those chips do better but only for the control plane; their real value is to offer a better and faster interface to the processors doing the actual work and data plane.
I guess those thingies are OK for a couple of 1 Gb pipes in an SME router box but I wouldn't even think about using them for anything more on a professional level.
Looking around in the professional field one will hardly find an intel processor other than for a nice user interface (for which intel is a good solution due to an extremely rich resource pool in terms of dev., libraries, etc).
Without concrete practical use today but maybe interesting anyway: I expect Risc-V to change quite a bit in the professional network field. That's the kind of processor we want there (or powerpc, sparc, cavium, even mips) - not x86/64.
Aye, but even if you do implement this kind of CPU into a server it won't be a close match to a recent router. While you have data plane and control plane merged you cannot do redundant routing engines, and you're bound to have issues like small pps causes my BGP to drop. This by definition prevents using that kind of construction in a datacentre network.
The MX series, for example, uses x64 or ppc for the control plane, but each fabric or fpc or pic has it's (ASIC) packet processor for dealing with routing, forwarding, filtering, etc. It's more like you'd take a number of small computers merged together into one machine responsible for efficiently routing your packets. The moment you start replicating this into a PC, no matter what CPU it runs, you start re-developing the wheel and will soon realise that the R&D cost of developing the solution, justifies prices charged by likes of Juniper and Cisco.
You can get 10Gbps comfortably (and stable) on x86_64 with tuning. I'm writing up a blog post with details and will link it in here when it's posted.
Yes and no. One can support redundancy and resilience - but doing that one quickly approaches the prices of brand name routers.
So, all in all, 99.9% of all providers will certainly be better of with simply buying a ready made brand name product.
How about 4 x 10Gb or 1 x 40Gb and 4 x 10 Gb?
I have had both Cisco and Juniper (both new) fail on me. Admitadly they were at the lower end of the market, and costing around the HK$50k or below price range (which around US$6.5k or under.
I am completely aware that most of the low end Cisco / Juniper stuff actually are actually software, and not really ASIC routers, but that does not make them any more reliable just because they are 'brand-name'. You cannot make a statement about how Juniper is more reliable and then later qualify that actually you are only referring to their $100K+ routers.
We know the lower prices stuff isn't going to be as high performance as the top of the line models, but the question then becomes why buy the lower end stuff at all when you can get better performance for a lower price when building your own software router? Familiar command line?
I was using those J series routers years ago, and required less than 1G total routing capacity. It was not a recent thing.
I also did not initially choose that range of Juniper routers. It was recommended by my upstream, and actually I purchased the router through them, new, and with 'support'. When the router failed, their 'support' meant "we can replace it on Wednesday", when it had in fact failed on a Sunday... The support issues were not strictly a Juniper issue but it left a sour taste in my mouth none the less. Using and configuring Junos is nice and easy but I have no confidence in the reliability of their products, and thus making deploying Juniper routers twice as expensive in order to ensure redundancy... which was pointless at the time given our network was single homed back in 2008 (9 ish years ago).
A quick look online and the J6350 seems pretty cheap now.... Which is sort of depressing.
I don't see any issues when dealing with a multiple gigabit uplinks. In HK, 10G is generally prohibitively expensive, with the exception of Telstra, Cogent and HE.net. But I'm not willing to invest $10k+ in a router only for HE.net. Maybe you consider 10G small, but among HK providers, I think that is pretty significant. And for those that I am aware of running 10G, it's mostly to HKIX where they probably push less than 500Mbit at any given time, while using DIY RouterOS or Mikrtotik. Not all, but most of the providers I personally know, yes. But given that's pretty much only for HKIX, I don't really see that as much of an issue either.
To be honest, if I find anything 'professional' for anything under $5k, I'd consider that a win! I'm open to a proper 'brand-name', 'professional' router, but I don't see that happening for under $10k and that is not what I'm looking to start with.
Some decent Brocade or Intel 10G NICs and a solid E3 or E5 CPU comes in much cheaper. Multiply that by 3 and it's still about 1/5 the price from a single 'professional' router. But I don't know what kind of performance I can expect out of it. Maybe some one here has experience with this?
If it can handle an attack of 2mpps or 3-4G, I think that would be acceptable. Our current policy is either to re-route via DDG or null-route the attacked IP any way, so being able to handle a full 10G or more I think is overkill. To begin with it would just be a single 10G upstream with everyone else being 1G any way. The 1G ports would be flooded long before the CPU falls over (in my experience).
Generally switches can't do BGP, and the ones that can, don't handle full routing tables.
Although I suppose one could have use a smaller table or static route everything. Can switches normaly route at line rate?
@randvegeta
First there is something that bewilders me. On the one hand you look for n x 10Gb routing, on the other you explain how 10Gb pipes are extremely expensive and not attractive where you are.
From what I see you are in a somewhat unlucky spot. Not anymore well served by n x 1Gb but not yet (anytime soon) needing n x 10Gb (with n >= 4).
While @Clouvider is generally right in what he says, well OK, not everyone can afford 20+ k$ for what is a lower end professional brand name box; I understand that.
But even understanding that I will not provide advice for non-redundant boxen because this comes down to lottery and to f*cking ones customers. What I am ready to advise on, however, is "light redundancy" which pretty much comes down to a human switching cables and boxes.
Basically my understanding there is that not everyone needs - and is ready to pay for - quasi-instantaneous failover and that, say, 10 min failover time is quite acceptable in many scenarios.
Also, for the sake of fairness one must see the quite considerable differences between markets. What's considered as inacceptable in LON or AMS might well be more than acceptable in other parts of the world.
So, here goes:
If it's about pure routing, some open switching platform might be your preferred choice. But be careful to avoid switch-chips only solutions; be sure to get something with a control plane and UI processor, too. While I don't have any concrete recommendations, I know that there are some 8 or 16 x 10 Gb open switches out there, some of which should meet what you need. From what I remember there are even ones with redundant power supplies which is highly desirable as power supplies are classical elements of failure. Some of those boxes even come as L3 "switches" which are basically simple(!) routers, too.
Getting 2 of those under 5 k$ (total) should be feasible.
If you need more than plain and simple routing, in particular, firewalling or even Anti-[D]DOS things get more complicated and more expensive; not reaching the 10 or even 20 K$ barrier but still, more expensive. Expect to pay between 5 k$ if you are very lucky and 10 k$ - plus loads of work to be done by yourself. What you'll get for that kind of money will typically be "barebones" (if you are lucky incl. case w/redundant PSs) and a CD-ROM with board support packages, typically for linux.
If you have the knowledge and the time you can build a kind-of-redundant 2 boxen system with up to 16 x 10Gb or 1 x 40 Gb plus 12 x 10 Gb that is capable to carry the full load, even with small packets and (depending on the kind of hw) more or less (quantitative and qualitative) firewalling and anti-[D]DOS.
What you can achieve largely depends on your knowledge and time invested; using a cheap "arbitrator" (an arm board would do) you could even achieve nice levels of rather fast failing over (y 5 sec) redundancy.
You should expect, however, to edit/adapt/write C code yourself.
If you have the knowledge, interest, and time you could then make this into a niche business, such recovering your investment.
Kind of. Basically switches are massive data pumps with some decision logic and a more or less crude ui. As for bgp: depends on the control plane and ui processor and memory. Principally it can be done; how well and how big tables pretty much comes down to the hw.
But again: Careful! I made that suggestion expressly for simple routing. Something like "I have an AS, a couple of public IP ranges (typ. /20 - /25) and some non routing internal IP range".
As you mentioned, bgp can (but must not necessarily) be a problem, depending on the hw chosen.
Theoretically, you could even add (frankly, very limited) firewalling but I strongly advise against that as the processors in those systems are good enough for massive pumping/switching and some ui but that's pretty much about it.
However: This could still be part of a quite good solution in that it could allow for a much simpler - and cheaper - core router with only very few interfaces. What you can play with is that many (most?) small and midsize providers have everything done in their core router. If you seperate some job (like firewalling and/or anti-[D]DOS) out and put them on other boxen you can save money without (if it's well done) significant delays.
The fact that you purchased that level of support doesn't mean Juniper is bad. Also, they could replace it on Wednesday. They could probably fix your JunOS if you got hit by a bug sooner. I'd take it with both hands instead of hoping someone can help me out on the Quagga mailing list on Sunday. I might never get a response. Don't tell me they couldn't - we had a bug with graceful switchover that was fixed in hours by cooking us a service release. Something that wouldn't be guaranteed with any Linux based solution.
You could take on a hybrid as well, like QFX5100 (https://www.juniper.net/uk/en/products-services/switching/qfx-series/qfx5100/specs/).
If you throw in a license for BGP on them they can cope very well with a partial table, they are simply 'mini routers', can be used as MPLS nodes, etc. We use a pair of QFX5100-48S in VC as a distribution switch between ToR's and geographically dispersed MX480s.
That's naturally far from what MX series will offer you, but can run dual routing engines (if you have two QFX5100) and have non-stop routing HA feature in case one of them is faulty.
I honestly don't understand the point in spending $10k for nothing, and later spending another $15k for a proper router down the line. Doesn't it make more sense to just bite the bullet and spend 15k in one go, maybe in a few months if you have no budget now ? Or maybe even take a lease?
Router is the core of your business. If network is causing you grief, you can have the most amazing support, and prices, and kit, and all, but this will impact the Customer's satisfaction severely.
Bonus point with a proper router and your unique location in HK is that you could in-fact have VRFs and use multiple L3 tables to create more revenue and differentiate by delivering various bandwidth options, some including direct China, some not, etc, giving you a potential to outrun the competitors.
The desire to get 10G routers is not out of any need. At least not yet.
In HK, even the cheapest of upstreams would cost around US$5 - $10k /month for a 10G uplink (with the exception of HKIX). Only a tiny amount of our traffic goes out through HE.net with the rest going out through much more costly routes.
HGC, PCCW, Pacnet, all charge considerably more. I'm not entirely sure how much a 10G would cost for these providers but I would imagine upward of US$60k /month (and that would be optimistic). If it were CN2 bandwidth I would expect $500k or more!
So there has been no need for full 10G routing gear in HK for most of the smaller - mid-size providers and why most will use either DIY or 'budget' routers like Mikrotik. Even if the routers are technically affordable, it's overkill to buy an 8 port 10G router when your aggregate capacity (excluding HKIX) does not even hit 10G.
As mentioned above, I am not currently in need of even a single 10G uplink, let alone a router than can handle multiple 10Gs. The vast majority of my traffic goes through my upstreams of which I only have 1G uplinks and our existing routers can more than handle the workload. We have redundancy in place that pretty much guarantees router or uplink failure will result in only a few seconds of downtime (pretty much 1-2 timeouts on ping.
First I want to get to play around with some 10G routers and see what kindof througput could be achieved in a LAB and then later (thinking maybe end of the year or early 2018) upgrade our HE.net and HKIX to 10G. But again, this is not so much out of necessity as it is just preparation.
Yes I know, and already mentioned it was not actually a Juniper problem. But the HARDWARE failure part was. Budget or not, I would expect more from Juniper. I dont remember exact figures, but the router that failed I believe cost around US$6K or so at the time. Even 10 years ago, a DIY router could compete with the J series on performance, and hammer it on price. So I don't understand the price tag. My guess is.. you're paying for brand name, and not quality.
I disagree entirely. A single unit failure is not enough sample to judge quality.
I also challenge you to replicate the same feature set on the software router. It just ain't going to happen. With exception if you go with vMX but still less performance.
What you pay for is naturally mainly R&D they put into developing this.
In regards to your support, if you bought X day SLA then they delivered X day SLA. Nothing more, nothing less. I wouldn't expect them to do anything better. If you go cheap on the support contract you should keep spares. Or suffer downtime. Be it Juniper or Cisco or Dell for servers. Have reasonable expectation. It's not their fault you didn't prepare appropriately for a potential outage be it in contracts, spares or plans.
I don't think there's a point in giving you any more evidence or discussing it any further, your mind is set in stone on this, no matter what me or anyone else proves here.
There I'm completely with you. If only all providers had that engraved in their mind ...
>
Which could also be understood as "get the cheapest 10 Gb pipe available, pump your traffic to the (time wise) next hop with decent and affordable international connectivity (taiwan? singapur? malaysia?) and route from there. 7.5 k$ (average taken) in HK plus, say, 3 k$ is still much much less than 50 k$ and offers about 90% - 95% of the expensive route quality. Just as a sidenote.
Returning to the router problem, also keep in mind that a 10 Gb port is a sound investment for a future where larger pipes become cheaper (or pressure on your clients becomes large enough to pay insane prices) while the shiny new 4 x 1 Gb box you buy today might be next to useless and quite limiting "tomorrow morning" if/when some backend provider lowers prices.
As for the juniper question I'm largely with @Cloudvider. You simply get much much more than hardware. Also keep in mind that while hw know-how is desirable, in the end the hw is but a tool. With Juniper you pretty much buy and get and keep to get "plug it in and go!"
Let me tell you a story: I can design and build an n x 10 Gb router and deep inspection multi-layer firewall. I have actually done it (albeit in another context which was about security). I know where I could could get the parts and even relatively cheap, I have the experience to muck with an OS, to adapt drivers, etc. I could, in fact, build you the box you are looking for and for less than 5 K$.
But - and here comes the part you won't like: Support beyond 2 weeks until you have it set up and running? Nope. If I have some time left over and if I like you a lot and if my wife doesn't beg me for sex (OK, OK, this one is just my phantasy g) I might help you out occasionally and probably only after some days delay.
With Juniper there are hundreds of support people, day and night, from human phone droid up to engineers who designed the box and software.
Spare parts? Forget it. Neither am I willing or capable to have them laying around nor do I have the infrastructure needed to get that part you so urgently need shipped out at 2 am in the night.
And that list goes on and on. It's those many "invisible" things that I can't possibly offer and that are just plain everyday standard for Juniper that you get for the tens of k$ you pay for their boxes *plus all the other factors and items.
That's why I insinuated again and again that you must have lots of knowledge yourself with the solution paths I elaborated on. Put a price tag on that and translate that to a university degree and many years of experience ... and you'll see that Juniper boxes are actually quite cheap for what you get and compared to the real alternative routes...
True I have only had 1 Junpier fail on me. But it is not the only 'branded' router to fail.
I am sure Juniper's Enterprise stuff is more reliable, but I'm not looking to spend $100k right now.
At the time I needed only basic BGP functionality. It is not all that relevant if the router could do more than another router if I do not need those extra features. But the point was that for the features I needed, the Juniper could not come close to the advertised specs (at least not with BGP). Supposedly the J6350 could handle 2mpps but when tested with a small attack, it would fall over at just 200-300kpps. Which was interesting.
I got the router and support contract from HGC. The support contract was actually kind of expensive (IMO) for about US$130 /month. The SLA offered by HGC was 4 hours for a repair/replacement. So when the failure occurred on a Sunday morning, and they told me they can come on a Wednesday, I was pretty livid.
Now of course HGC is not Juniper and I do not blame Juniper for HGC's failing. But when you spend $6k on a router, you do expect it to last more than 1 year before it konks out.
I do! And I was backup within 20 mins when I deployed an old Cisco I had laying around. The cisco could handle the routing no problem but it couldn't handle 2 full BGP tables at the time so I was basically doing a static route via my 2nd uplink. The ease and speed at which I could deploy a VyOS router made sense to setup a number of them all over the place to maximize availability.
Unrelated to server reliability, the person who HGC sent (who worked for yet another company contracted out by HGC) was completely incompetent and managed to shut down my network by pulling on power cables connected to my core switches. That was pretty maddening. Any way.. 3rd party support, even from seemingly large companies who are supposed to know what they are doing, can completely fuck up and screw you over. Needless to say, the support contract was terminated after that.
You seem to be thinking that I suffered a major downtime and that it was my fault for being cheap on the hardware, support and not having any backup.
I grant that I haven't got in place the cutting edge of networking gear, but we are talking about something that happened almost 10 years ago when we were first moving out of a DC (who took care of the all the major networking stuff) and into our own facility where we were specifically recommended a router by our upstream. At the time, our total traffic was about 100Mbit for which we paid about $6k for a brand-name router, and a further $130 /month for support.
My expectation at the time was that a brand-name router that were recommended by the upstream would be sufficient and be reliable, and that any issues would be resolved under the terms of the SLA. Arguably the performance was sufficient as it could handle 100Mbit just fine but the reliability and support was sub-par. Perhaps I'm off-base here but I don't consider $6k + $130 /month to be so cheap considering the DIY routers I setup could run circles around the Juiper as well cost just a fraction. Am I missing something here?
Small businesses have to start somewhere. You can expect a company with just a few dozen servers to invest in a $100k router that can handle 100x more capacity than needed.
I'm saying I'm not going to use Juniper. I know they make good kit, but I'm not looking to spend over $10k right now. As I have explained above, I do not even have the NEED for it atm but I am still interested in deploying something that would be capable of handling 10G. If it cannot be done I will postpone this until it can be.
Also I don't know what 'proof' you are referring to anyway.... What is it you are trying to say? That Juniper and Cisco are more reliable? That it is my fault they failed? That no amount of failure proves that they are unreliable? I'm genuinely lost here. What do you recommend a small company do when they are looking to grow beyond colocating at a DC that controls all things network? I'm not being rhetorical here, this is a genuine question.
Not sure I understand what you mean by this.
I appreciate everything you're saying and I do not disagree in principal. Perhaps my mistake was going through a 3rd party, and getting support service from this 3rd party. Though I think getting service support from a large telco who's base of operations are actually in Hong Kong actually sounds like a good idea.... but apparently I was just unlucky, or naive for thinking a multi-billion dollar company would actually provide a service they had sold (silly me!).
The whole experience has left me questioning the value of said support services. Now maybe in USA and the EU things are different and the support services can all be relied upon, but the only company I have seen in HK that actually provides a 'decent' support with regards to replacing failed hardware is Dell. Now I'm sure HP and other big hardware vendors all do good stuff in HK and fulfil their contractual obligations, but even they need hours in the best of scenarios. I generally need things fixed in minutes IF something goes wrong, and I need things to be as familiar as possible if trying something new.
I am actually very interested in hearing real world experience with the edgerouter infinity. It seems like a huge step above the edgerouter pro's. Dual hot swap PSU and 9 times more PPS compared to the er pro, on paper.
Also to keep the discussion going - what about lower end juniper mx-series. Mx5 for example - these seem to be in the 5k range second hand. What are the limitations there? (tagging @Clouvider and @bsdguy or anyone else who has experience with enterprise networking).
Look into 6WIND, its a software router using DPDK so you can achieve higher PPS than with something like VyOS.
A E3-1230v6 can handle about 30 million PPS.
Real world experience with that thingy I don't have to offer - and most probably won't ever.
But I know similar boxes (x tilera cores plus some switching chips). To make it short: I do not like it, not at all.
I'll give you 2 reasons:
16 cores at 1.8 GHz? Ridiculous. That shows that that thingy is either a marketing driven solution or they are clueless, or both. 16 GB memory, OK, but what would one use 16 high clock cores for in a router and a relatively small one at that?
From what I see it's a classical case of "take a 10 Gb 12 port switching chip, put 10 ports on fiber plugs and 1 on copper and use the last one to connect to the processor". Cheap but mindless and severely limited by design.
Second point: 80 Gb with (rather poor) 18 mio pps? Ridiculous. Do the math and see yourself; those 18 mio pps are 5.5 KB packets! In other words, that's a theoretical mix of switched(!) jumbo and 1KB packets. Plus: It's a clear indicator of [pls. give me another, a nicer more polite word for "nicely packaged scam"].
And finally, just guess what the 1 switch port to processor link can carry ... I think I'm generous in assuming that it's 2 or max. 4 serdes lines of 5 Gb each.
A glorified pimped up switch. That's what this toy is. The only positive point is dual PS, which they probably included to make the toy look more professional and to draw attention away from the real meat.
My personal view: Stay away!
P.S. In case someone needs an all in one core device for a small scenario or some small pop or the like, you can consider that box as a dual 4 Gb backend pipes router plus 8 not fully used switch ports to collect/distribute internal traffic (8 Gb in total of which can be pumped to/from the backend pipe). As such it might be an acceptable solution.
New MX5 will cost you way more than MX104 which is much more capable, I'd perhaps recommend checking with your local reseller and try to negotiate down a little ;-). You can really get a new one cheaper than a used one on eBay quite often.
How about mikrotik cloudcore routers with 10Gbit? Im currently running one with 1x10Gbps and i have No packet loss (Yeah 0.3% at most) dropped
Isn't the MX104 about $50k?
All those cores and still only uses 1 for BGP. Slow in that respect.