Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


BuyVM LU Outage
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

BuyVM LU Outage

This email is to document and explain the issues we've been having with our network in Luxembourg over 3 - 4 weeks.

As you're all aware, we've had a rough month in Luxembourg to put it lightly. The first issue was due to a supposed optic failing in the aggregation switch connecting the two suites the facility operates in LUXCONNECT. This first outage lasted over an hour.

The next one about a week later was supposedly a large DDOS attack hitting one of their cores can causing it to max out its management blade CPU. We feel that if this really did happening, the issue was them failing to ACL/protect their management blade from being attacked directly (gateway IP's, uplink IP's, etc). We still believe there is a likely chance it was the port channel/aggregation switch failing and them making up excuses.

The latest issues have gone on all week where there has been these 30 second ~ 1 minute network drops. These drops were supposedly the port channel failing again forcing the facility to replace all optics on both sides of the port channel. Unfortunetly this didn't fix anything and the switch started to crash every couple hours, automatically rebooting. This morning they replaced this unit...and proceeded to break the network once more just for good measure.

The very latest outage lasting around 10 minutes was them moving us from a daisy chain of switches (uplink switch->aggregation switch->(port channel)->cores) by removing the aggregation switch/port channel from the mix. Unfortuntely they didn't bother informing us that this was being done, allow us no time to alert anyone.

On our end, we're currently awaiting the arrival of a new brocade router to replace the current setup. This new router will allow us to pull a 10gbit port right off their core, skipping the uplink switch as well, removing another point-of-failure. On top of also redundant uplinks with them (which aren't available in their current configuration), we're pending an additional network connection from Cogent as a fallback ISP in the off chance something breaks again.

We're sorry that this all sounds like a big list of excuses for why it isn't our fault. We know many feel we're making it all up and we don't blame you. RootLU has been extremely slim on any sort of details and Francisco has near threatened to overthrow their government to get any sort of feedback on what's happening.


Aldryic C'boäs,
Frantech Staff

Comments

  • Hope this gets better soon

  • Does root.lu have a monopoly in Luxemburg? Otherwise I'd consider moving from this incompetent datacenter.

    Thanked by 1linuxthefish
  • Luxembourg is a very very small country, smaller than a big city. I doubt there are that many datacenters there.

  • rds100 said: Luxembourg is a very very small country, smaller than a big city. I doubt there are that many datacenters there.

    But it's huge compared to Lichtenstein, and even that 160km² country got multiple datacenters.

  • LeeLee Veteran

    With an associate a couple of years ago we bought over an online firm that had it's equipment with them and to be honest they were great overall. Of course that may have changed.

    When Fran said they were placing their EU set up there I thought it was a good choice.

  • FranciscoFrancisco Top Host, Host Rep, Veteran

    @tr1cky said:
    Does root.lu have a monopoly in Luxemburg? Otherwise I'd consider moving from this incompetent datacenter.

    I've yet to really find anyone else in LUXCONNECT. It seems to mostly be local ISP's and or corporations that aren't doing customer colocation.

    They have a nice setup, my biggest issue with them is they're very light on details when things break. They refuse to write emails anywhere or post it anywhere on their site, meaning I sit there sounding like i'm making excuses.

    Prior to this last few things they've been mostly fine. There was an issue with the interlink between suites in the past but it was usually fixed quick enough and their worker Ben explained things the next day.

    Them feeding our uplink off their core now is a step in the right direction, the next will be right from the cores and then the cogent line. Once they get us feeding right from the core it'll likely be solid, but adding a Cogent drop isn't overly expensive.

    @Frecyboy said:
    But it's huge compared to Lichtenstein, and even that 160km² country got multiple datacenters.

    I think it's more that RootLU is just that big of a group. Supposedly They have mid X,XXX in dedicated customers so it'd probably be pretty hard for someone to move in on that and stay afloat.

    We'll see. Been a long week with that + finding some buggy equipment at Voxility.

    Francisco

  • Francisco said: They refuse to write emails anywhere or post it anywhere on their site, meaning I sit there sounding like i'm making excuses.

    Feel free to move over to one of the nice datacenters in Zurich.

  • @Francisco said:
    They refuse to write emails anywhere or post it anywhere on their site, meaning I sit there sounding like i'm making excuses

    Yes, indeed. Honestly said, it sounds like that.

  • FranciscoFrancisco Top Host, Host Rep, Veteran

    @trvz said:

    I'm not up for a Europe trip to go shuffle things :P I've done that too many times already. I'll just have to nail it into their heads that being informative makes everyone happier when things break.

    @Amitz said:
    Yes, indeed. Honestly said, it sounds like that.

    Yes, quite :P

    Francisco

  • Frecyboy said: But it's huge compared to Lichtenstein, and even that 160km² country got multiple datacenters.

    There is Kyberna (Mauren), Telecom Liechtenstein (Vaduz) and Newsnet (Mauren i think), all others colo in one of these (at a hefty price tag, a full rack with 10Mbit BW and 10A power costs around 5000CHF/4600EUR/5200$). The "university" runs a small server room but the Gov mainly uses Telecom Liechtenstein (which is obviously state owned). Topnet has a small room somewhere in Vaduz for their carrier equipment.
    Kyberna is 25%+ state owned as well (and has gov and royal family on board/chair), Newsnet is privately owned. LI also has a large sector (25%+ of GDP) that REQUIRES local hosting (all bank data and consumer data has to be stored inside Liechtenstein in a certified DC) while LU banks/etc can use any EU (and also EFTA) country for it.

    In LU you have a few DCs (Root in Steinsel, Carrier neutral Luxconnect, datacenter.lu as example) but not much more than in LI either.

    Thanked by 3Frecyboy mpo vimalware
  • mpompo Member

    They could do better with pre sale communication too. But If they are that big as @Francisco said, then I can understand that they don't really care if they loose a few customers. (But economically speaking even if they'd have to get one more staffer that would probably be worth it to communicate more!)

  • @mpo said:
    They could do better with pre sale communication too. But If they are that big as Francisco said, then I can understand that they don't really care if they loose a few customers. (But economically speaking even if they'd have to get one more staffer that would probably be worth it to communicate more!)

    I dont want to be that guy ... but... *lose.

  • Sounds like a typical day at BuyVM :) Too many problems no one else has.

  • @4n0nx said:
    Sounds like a typical day at BuyVM :) Too many problems no one else has.

    Them fighting words son.

  • ATHK said: Them fighting words son.

    Pls don't discredit my experience

Sign In or Register to comment.