Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


OVH Strasbourg Power Failure - Page 10
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

OVH Strasbourg Power Failure

1567810

Comments

  • @jetchirag said:

    @citrix said:
    my VPS is live since the outage in SBG

    My VPS was also up!

    Edit: Just that it wasn't hosted at OVH

    typo :P.... now live.

  • southysouthy Member
    edited November 2017

    @Clouvider said:

    BasementCluster said: The outage was caused by a fault in power failover. The generators did work, they didn't start. This is a problem which affects all providers because it's fundamentally difficult to test for.

    Yeah, and clearly keeping only 8 minutes of juice didn't allow anyone to react quick enough to prevent it. This particular outage was simple and extremely easy to prevent.

    Have you ever been present during a site full failover test? You'd notice that it takes a good couple moments for the generators to start and normalise in normal conditions when everything goes well. I don't work with any datacentre that has such a short margin on batteries as it was in this case.

    I have to agree with you on this one, even if it hurts: it is really strange:
    There is an outage and UPS takes over. So far so good. So the first thing I would do in such a case is clearly to open my ears to check if I can hear the gensets coming up.
    And if not, I would run for the emergency power switchboard. And yes, all those clever mechanisms that Ole mentioned do or should have manual overrides. You can’t tell me that they bought a gen that doesn’t have a local control panel for manual control.

    And yes, even if that automatic announcement that he mentioned, failed (never heard such a humiliatingly clear case of „overengineered“), everyone on site will normally Note when power goes away. Light is different, maybe even sound (some unimportant AC)...
    So what the heck did all those guys there do in those 8 minutes?
    Only explanation would be if the site was totally remote controlled without personnel on site.
    Or those on site were not trained enough.

    But that aside I stick to my point: a team that experienced such a desaster for me is quite likely to not make such mistakes again. Which is why I would assume, at any given price point, it’s not the worst choice to go to where people have already had problems.

    Thanked by 2Clouvider WebDude
  • ClouviderClouvider Member, Patron Provider

    The problem here @southy, is that 8 minutes is an awfully short time between the buzzer going off on the monitoring screen vs the time needed to run wherever you need to run, in their case of this container DC probably outside of the 'building', potentially get a key, perform an operation that one might be either untrained for, or maybe not up to speed with (since you don't do it every day), get the generators running, obtain normalised power and switch. Manual switchover in <8 minutes is very much unrealistic in real life scenario even if everyone on shift is trained for the operation (doubtful). Double that and you have a good chance.

    Thanked by 2Maounique WebDude
  • NeoonNeoon Community Contributor, Veteran
    edited November 2017

    @Clouvider said:
    Relevant to support my sentence.
    Future investment is irrelevant to what I was saying. Furthermore, you might have noticed that they learned quite a bit through the years and are increasing the prices, and my hunch is they will continue to steadily do so now.

    I call fake news, I see also some prices dropping.

    The majority may go up, but no everything.

    OVH had the balls to admit the failure, and I respect them for that.

    I understand you are in love with them, but this love makes you blind when even they admitted it.

    Well, we raged enough about OVH, with the SBG thing, no worries.

    I may like some of my providers yes, only because it works, I do not love them.

  • @Setsura said:

    @szarka said:

    @Setsura said:
    I know that feel, especially as an early BHS user, the power issues, and then the "fiber cuts". Good times at BHS.

    Triggered.

    I forgot which one, but during one of the "fiber cuts" they had all of BHS routing through like a single 1G backup line or something, so everyone got to collectively stare at their terminals, unable to actually do anything just because of how slow it was. BHS was fun.

    This is the one I remember being pretty painful... http://status.ovh.com/?do=details&id=11304

  • MaouniqueMaounique Host Rep, Veteran

    @Clouvider said:
    The problem here @southy, is that 8 minutes is an awfully short time between the buzzer going off on the monitoring screen vs the time needed to run wherever you need to run, in their case of this container DC probably outside of the 'building', potentially get a key, perform an operation that one might be either untrained for, or maybe not up to speed with (since you don't do it every day), get the generators running, obtain normalised power and switch. Manual switchover in <8 minutes is very much unrealistic in real life scenario even if everyone on shift is trained for the operation (doubtful). Double that and you have a good chance.

    This is where he crosses the Ts and puts the points on Is.
    Yes, 8 min of ups power is not enough. I believe this is because they expanded but not the UPSes too. Or they were too old, or some of them were too old which generated cascade failure, I dont have their electrical scheme to see what was it, but 8 min is too little, period.
    I also agree, the manual override is probably under lock and key, only 1-2 people if that have the key to the key drawer, take the required key out, go unlock the panel, start generators, wait 1-2 min at least all this after waiting up to a min to see if they start by themselves, then panic, run, maybe even drop the key, i presume 3-4 min passed since the power failure until the guy was at the panel with the key, then unlock it, hands shaking due to adrenaline, and potentially hearing people at the UPS monitoring panel calling there are only 2 min left, also maybe hearing the blaring alarms, not enough time even for the best case scenario.
    1 hour is probably enough, 2 to make sure, also you can juggle it if the power failure is only partial, i.e. only one feed failed, you need to plan for this too, wire the floors accordingly while making sure the SPoFs are as few as possible, because non-existant I don't think can be.

  • CConnerCConner Member, Host Rep

    I was considering moving my infrastructure in the UK to Clouvider, but I am having seconds thoughts now :(

  • RhysRhys Member, Host Rep

    @CConner said:
    I was considering moving my infrastructure in the UK to Clouvider, but I am having seconds thoughts now :(

    Because he's being honest about OVH? lol.

    Thanked by 1Clouvider
  • CConnerCConner Member, Host Rep
    edited November 2017

    Rhys said: @CConner said: I was considering moving my infrastructure in the UK to Clouvider, but I am having seconds thoughts now :(

    Because he's being honest about OVH? lol.

    "Honest"? Rather "shit talking". Making assumptions left right and center where there isn't even any information that you base it upon.

    You can keep bitching and moaning, but I don't see a better alternative. So far, I am satisfied with what OVH has provided me with. This is the only time any of my servers went down because of them.

    There are so many more things you get when you go with a "premium" provider, however, it's not worth paying 200-300% more. If I need to pay that for that one occasion in many years to get my servers up a little bit faster then no thank you.

    Thanked by 2rm_ Profforg
  • RhysRhys Member, Host Rep
    edited November 2017

    CConner said: You can keep bitching and moaning

    Nobody is bitching nor moaning calm down.

    CConner said: 200-300% more

    source this baseless stat.

    CConner said: making assumptions left right and center where there isn't even any information that you base it upon

    Based on what has been said by Oles on both Twitter and on the Status page.

    I'm sure Dom won't miss your money.

  • CConnerCConner Member, Host Rep
    edited November 2017

    @Rhys

    I am sure he won't miss my money.

    The plans you pointed out are all 16 GB and the OVH plan is 32 GB, adding another 10 GBP on top, which makes them about equal. Check your facts if you so desperately want to prove me wrong.

    If you can not relate to the bitching and moaning in this thread, I suggest you read it again. "Dom" / Clouvider has been called out for his childish snipes at OVH multiple times. The guy with the most posts on this thread is someone who is competing with OVH in the UK and Europe.

  • RhysRhys Member, Host Rep

    @CConner said:
    @Rhys

    I am sure he won't miss my money.

    The plans you pointed out are all 16 GB and the OVH plan is 32 GB, adding another 10 GBP on top, which makes them about it equal. Check your facts if you so desperately want to prove me wrong.

    If you can not relate to the bitching and moaning in this thread, I suggest you read it again. "Dom" / Clouvider has been called out for his childish snipes at OVH multiple times. The guy with the most posts on this thread is someone who is competing with OVH in the UK and Europe.

    So when did equal become 200-300% more? I feel like I'm teaching 1st-year maths.

  • CConnerCConner Member, Host Rep

    I am referring to premium providers. Defintely not Clouvider.

    Premium providers often have stricter SLA's, and often have a more direct line of communication with the customer.

    You sure are a fast reader. Are you sure you reread the whole thread?

  • @Neoon said: Every where OVH is, you keep bashing it.

    Its not just OVH. He bashes just everyone in the vicinity. OVH, Zare, Dada, Iomart to name a few. You can find critical comments from him for almost every other provider/DC in UK or near by locations. Then there is cozy group of Clouvider's re-sellers on LET where A recommends B, B recommends C, C recommends A, A B C recommend Clouvider and Clouvider recommends A B C.

    I am not commenting on the services provided. They might very well be economical & stellar but I so wish that he kept his marketing strategies limited to sending chocolates, post cards and gifts instead of trying to belittle his competition (most of them do not have a presence on LET or do not have as much time as Clouvider has to counter his arguments).

  • CConnerCConner Member, Host Rep
    edited November 2017

    -

  • CConnerCConner Member, Host Rep
    edited November 2017

    dynamo said: @Neoon said: Every where OVH any other provider that competes with him in Europe and the UK, you keep bashing them.

  • So this has become a @Clouvider bashing thread now that OVH is back?

    Fuck you, Dom. I'm too damn cheap and not geocentric to your services!

    Thanked by 3Clouvider szarka Rhys
  • MaouniqueMaounique Host Rep, Veteran

    WSS said: So this has become a @Clouvider bashing thread now that OVH is back?

    OVH has quite a following, stirring the wasp nest is never a good idea :P

    Thanked by 2Clouvider Lee
  • @Maounique said:

    WSS said: So this has become a @Clouvider bashing thread now that OVH is back?

    OVH has quite a following, stirring the wasp nest is never a good idea :P

    I'd make a German joke here, but they were founded in France, so it's close enough.

  • MikePTMikePT Moderator, Patron Provider, Veteran
    edited November 2017

    I still remember the taste of the chocolates that Dominik sent us. Awesome chocolates.
    Btw our server has been running flawlessly with Clouvider. We would only reconsider to move out if 2x1TB SSDs are too expensive should we need to expand soon. No complaints at all.

    Thanked by 1Clouvider
  • @MikePT said:
    I still remember the taste of the chocolates that Dominik sent us. [..] No complaints at all.

    Thanked by 1MikePT
  • MikePTMikePT Moderator, Patron Provider, Veteran

    @WSS said:

    @MikePT said:
    I still remember the taste of the chocolates that Dominik sent us. [..] No complaints at all.

    I love you babe.

  • southysouthy Member
    edited November 2017

    @WSS said:
    So this has become a @Clouvider bashing thread now that OVH is back?

    Come on @WSS, give them some space. The browser wars are over, the editor war has fallen asleep, the german provider war (1&1 vs Strato) has ended with world domination, and now even OVH cleaning lady has managed to put the plug back into the socket... let the guys have one little enemy to fight. Think of it as some sort of psycho-hygiene.
    Btw: have you already posted your „WSS‘ date of the day pic?“ Is it those two up there? Naaa... no good. To high phone bill. Your date yesterday was better.

  • Emails regarding SLA were just sent in. Got 12% compensation for OVH RBX and 0% compensation for SYS RBX. Anybody else in the same boat?

  • Vova1234Vova1234 Member, Patron Provider
    edited November 2017

    SLA RBX - 12%

    37 - id incident.

    https://eu.api.soyoustart.com/console/#/me/sla/{id}/apply#POST
    https://eu.api.kimsufi.com/console/#/me/sla/{id}/apply#POST

    The trick is that there is no this in the panel, but there is an API. But the newsletter has arrived!

    Thanked by 2sunnyg Falzo
  • LeeLee Veteran

    Maounique said: OVH has quite a following, stirring the wasp nest is never a good idea :P

    Bit like VolumeDrive and GVH back in the day, no matter what they do you will never break the commitment of their followers :)

    Thanked by 1Clouvider
  • For me they offered 100% for RBX servers, so I assume they're going to offer at least 100% for SBG servers too. Interesting to see. Many providers state in their SLA contract that the maximum is one months fees, so that might be it. Haven't checked OVH's SLA contract in detail.

  • rm_rm_ IPv6 Advocate, Veteran

    SBG monitoring in the panel is broken since the downtime. I wonder if they even know.

  • teamaccteamacc Member
    edited November 2017

    Welp, while they boast a 99.9% network sla on their website:

    I am not entitled to any SLA compensation for my RBX kimsufi servers that were down for more than 0.1% of the month...

    Thanked by 1Clouvider
  • exception0x876exception0x876 Member, Host Rep, LIR
    edited November 2017

    @sunnyg said:
    Emails regarding SLA were just sent in. Got 12% compensation for OVH RBX and 0% compensation for SYS RBX. Anybody else in the same boat?

    Got 12% for SYS and Kimsufi. RBX only.

Sign In or Register to comment.