Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


OVH Strasbourg Power Failure - Page 8
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

OVH Strasbourg Power Failure

15681011

Comments

  • MaouniqueMaounique Host Rep, Veteran

    CConner said: I fail to see how it is related?

    Cheaper and better managed. A hint about where to go? There are also other connections with the thread at hand.

  • southysouthy Member
    edited November 2017

    @Vova1234 said:

    But I'm interested here. Who after the incident will relocate?

    Nope. First off, nothing important was damaged.

    But more importantly:
    We all know how complex these systems are today and how much can go wrong and that no one is really safe from problems.
    So I do very much apprechiate it if there's people/companies that did actually have the chance to learn from a desaster such as this.
    Because I am rather sure that there are going to be a lot of meetings at OVH over the next weeks and that they will analyse what happened and in due time will improve their systems and processes.

    Of course I know that probably with a lot more ressources one could have installed double as good technology, but that's a budget question then.

    So I guess what I mean is:
    Within a given price-range (= product quality level) I do expect that providers that already suffered disasters are the better choice than the ones that have never had any problem.
    Because those guys probably have just been lucky.

    Incidents will happen. To the very best of us. But how an incident is handled and what consequences are drawn, that's the point to see if a provider is good or bad.
    An incident well handled for me is actually something positive, not negative. I am not sure on how to rate OVHs incident response yet because I didn't have the time to follow todays events properly, but given that there were two seperate problems at the same time and still their stuff came back rather quickly, I tend to give a "thumbs up".

    In times of trouble it is when you learn to seperate wheat from the scuff. Or from the smart-asses.
    So today I learned about one provider that seems (preliminary assessment) to have handled their problems decently. That's OVH.
    And it also occured to me that another one is rather a smart-ass. @clouvider, that would be you.

  • I'd prefer a host with a quick wit than a slow one, all the same, @southy.

  • terrahostterrahost Member, Patron Provider
    edited November 2017

    @southy said:

    @Vova1234 said:

    But I'm interested here. Who after the incident will relocate?

    Nope. First off, nothing important was damaged.

    But more importantly:
    We all know how complex these systems are today and how much can go wrong and that no one is really safe from problems.
    So I do very much apprechiate it if there's people/companies that did actually have the chance to learn from a desaster such as this.
    Because I am rather sure that there are going to be a lot of meetings at OVH over the next weeks and that they will analyse what happened and in due time will improve their systems and processes.

    Of course I know that probably with a lot more ressources one could have installed double as good technology, but that's a budget question then.

    So I guess what I mean is:
    Within a given price-range (= product quality level) I do expect that providers that already suffered disasters are the better choice than the ones that have never had any problem.
    Because those guys probably have just been lucky.

    Incidents will happen. To the very best of us. But how an incident is handled and what consequences are drawn, that's the point to see if a provider is good or bad.
    An incident well handled it for me actually something positive, not negative. I am not sure on how to rate OVHs incident response yet because I didn't have the time to follow todays events properly, but given that there were two seperate problems at the same time and still their stuff came back rather quickly, I tend to giva a "thumbs up".

    In times of trouble it is when you learn to seperate wheat from the scuff. Or from the smart-asses.
    So today I learned about one provider that seems (preliminary assessment) have to handled their problems decently. That's OVH.
    And it also occured to me that another one is rather a smart-ass. @clouvider, that would be you.

  • I think LET ate your response @terrahost

    @AnthonySmith said:

    Vova1234 said: But I'm interested here. Who after the incident will relocate?

    Naa, let's be honest, I don't use OVH because of quality, I use them because they are cheap and in doing so I accept the pitfalls, for me, it obviously had no impact as I have a failover system in place.

    I actually use them because they've achieved a fair level of quality well above what you can reasonably expect from their price ranges at any other host. I'm sure the usual mantra of "you get what you pay for" will be echoed by people for years to come about OVH, but the simple fact is that they really can't be beaten for the price, or often even 2x the price at many other competitors. Their response to this incident was definitely lacking in the communication department, but they managed to fix what seems to be a relatively complex issue at RBX in 2 hours, so I can't really chastise them too harshly there. All in all, I still like OVH, they probably just need better disaster recovery communications.

    Anyone that did not have a failover/redundancy is either not doing anything important or their business does not justify it, in which case, a day or so of downtime a year is probably not a big deal anyway given the cost.

    Or was lazy, ayy.

  • HarambeHarambe Member, Host Rep

    For once I'm just happy it wasn't a BHS outage

  • @Harambe said:
    For once I'm just happy it wasn't a BHS outage

    I know that feel, especially as an early BHS user, the power issues, and then the "fiber cuts". Good times at BHS.

    Thanked by 1Harambe
  • OBHostOBHost Member, Host Rep

    We have Main Server in SBG, Backup server on RBX and Alas both down at the same time.
    And ruin our totally VoIP Services.
    We need to arrange urgently third backup server on Germany, Frankfurt and Shift our traffic on it.

    A day goes bad today...!!!

  • @OBHost said:
    We have Main Server in SBG, Backup server on RBX and Alas both down at the same time.
    And ruin our totally VoIP Services.
    We need to arrange urgently third backup server on Germany, Frankfurt and Shift our traffic on it.

    A day goes bad today...!!!

    ... you put all VOIP services with the same provider?

  • OBHostOBHost Member, Host Rep

    @doghouch said:

    @OBHost said:
    We have Main Server in SBG, Backup server on RBX and Alas both down at the same time.
    And ruin our totally VoIP Services.
    We need to arrange urgently third backup server on Germany, Frankfurt and Shift our traffic on it.

    A day goes bad today...!!!

    ... you put all VOIP services with the same provider?

    We had Germany before but because of some Distortion issue we need to move on OVH services.
    But now we have 1 Main server with 2 backups..
    And thinking to add USA Server as well because uptime is main thing rather it from USA or France or Germany.

  • AnthonySmithAnthonySmith Member, Patron Provider
    edited November 2017

    Setsura said: I actually use them because they've achieved a fair level of quality well above what you can reasonably expect from their price ranges

    Same, I use them myself for a couple of things, but I have had poor support and responses from them when things have gone wrong, I expect it though, that is why they are cheap, they rely on automation heavily and sometimes that square peg will not fit the round hole.

    So I don't rely on them in isolation nor would I suggest anyone uses a budget hosting solution in isolation for anything critical, I put most of my own products in that boat just in the interest of clarity.

    Realistic expectations are key, just because something has hung on by a thread in the background for 600 days does not make it stable.

  • Finally all systems up'n'running. No data loss / corruption as far as I know. Phew! It could have been so much worse...

  • Vova1234Vova1234 Member, Patron Provider

    @WebDude said:
    Finally all systems up'n'running. No data loss / corruption as far as I know. Phew! It could have been so much worse...

    At me too have risen in SBG. True, with the OS that it does not want to start and the servers in rescue-pro, it will be necessary to correct. The whole day probably will have to spend.

  • WebDudeWebDude Member
    edited November 2017

    I don't know why many servers started, but without networking. Only after reboot the networking was restored. So there was something clearly wrong when the servers launched. Also on some servers some services were crashing, before reboot. I guess the storage system was also overloaded when the services started and there could have been potentially some timeout issues etc.

    But 10+ servers required "just" manual reboot, then networking and services started as usual. - It's a good question why that was necessary.

    I'm waiting for juicy postmortem / root cause analysis report.

  • This is some kind of early postmortem about what happened:

  • @WebDude said:
    This is some kind of early postmortem about what happened:

    The final conclusion is gold though

    The quick answer: SBG's power grid inherited design flaws related to the initially low ambition for the site.

  • @teamacc said:

    @WebDude said:
    This is some kind of early postmortem about what happened:

    The final conclusion is gold though

    The quick answer: SBG's power grid inherited design flaws related to the initially low ambition for the site.

    On the bright side - it's nice to see they admit they made a mistake regarding the design of their power grid in SBG.

  • CConnerCConner Member, Host Rep
    edited November 2017

    @Maounique said:

    CConner said: I fail to see how it is related?

    Cheaper and better managed. A hint about where to go? There are also other connections with the thread at hand.

    ....?

    Stay out of discussion if you haven't bothered reading what it is about.

    Any way, glad to see that the problem was fixed somewhat fast. I was wondering whether I had forgotten paying for stuff again.

  • As a OVH customer i like to know if there are more low ambition datacentres?

  • MikeAMikeA Member, Patron Provider
    edited November 2017

    @CConner said:

    Any way, glad to see that the problem was fixed somewhat fast. I was wondering whether I had forgotten paying for stuff again.

    There are many areas of SBG down even after 24 hours actually, just nothing being said by their team or on the status pages now. I don't have any servers there that I use, but I manage some for people and they were up earlier but went down about 2 hours ago again.

  • CConnerCConner Member, Host Rep
    edited November 2017

    @rick2610 said:
    As a OVH customer i like to know if there are more low ambition datacentres?

    I believe OVH built SBG to deal with thr demand quickly, that is why their datacenter is made up of shipping containers stacked ontop of each other with servers inside of them.

  • rm_rm_ IPv6 Advocate, Veteran

    WebDude said: This is some kind of early postmortem about what happened:

    Reading this, it reminds me of that old video: https://www.lowendtalk.com/discussion/79775/when-something-goes-wrong-and-your-server-is-down-due-to-an-electrical-issue/p1

  • Hello
    My OVH server is down more than 30 hours, I understand their current turbulent state, But they should understand their customers too. 30 hours down time is unbelievable for Datacenters like OVH, They don't respond to tickets too

  • AnthonySmithAnthonySmith Member, Patron Provider

    Bahram0110 said: Hello My OVH server is down more than 30 hours, I understand their current turbulent state, But they should understand their customers too. 30 hours down time is unbelievable for Datacenters like OVH, They don't respond to tickets too

    Hello, you should try rebooting your server from the customer panel, it has worked for many people.

  • For me accessing via console and rebooting fixed the issues. But some of the servers allowed console access only today. I did check yesterday the situation and those were unavailable 'in service mode' back then. But today logging on console and reboot, fixed the issue(s).

  • @rm_ said:

    WebDude said: This is some kind of early postmortem about what happened:

    Reading this, it reminds me of that old video: https://www.lowendtalk.com/discussion/79775/when-something-goes-wrong-and-your-server-is-down-due-to-an-electrical-issue/p1

    Great one! just great one.
    I love it how he throws snow into the cabinet at the end! :-)
    If I didn't know for certain that there is currently no snow in france, that would have made me thinking...

  • NeoonNeoon Community Contributor, Veteran

  • @Setsura said:
    I know that feel, especially as an early BHS user, the power issues, and then the "fiber cuts". Good times at BHS.

    Triggered.

  • CConnerCConner Member, Host Rep
    edited November 2017

    @Bahram0110 said:
    Hello
    My OVH server is down more than 30 hours, I understand their current turbulent state, But they should understand their customers too. 30 hours down time is unbelievable for Datacenters like OVH, They don't respond to tickets too

    Look at it from the brightside: If they will honor their SLA agreement, you will get your server(s) for free next month!

    < /sarcasm>

    Thanked by 1southy
  • @CConner said:

    @Bahram0110 said:
    Hello
    My OVH server is down more than 30 hours, I understand their current turbulent state, But they should understand their customers too. 30 hours down time is unbelievable for Datacenters like OVH, They don't respond to tickets too

    Look at it from the brightside: If they will honor their SLA agreement, you will get your server(s) for free next month!

    < /sarcasm>

    Another reason for LET users to be OVH customers - free hardware.

Sign In or Register to comment.