Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Down - OVH - SBG - Lots and lots of tears. - Page 15
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Down - OVH - SBG - Lots and lots of tears.

11315171819

Comments

  • @SirFoxy said:
    3

    Its likely depending on the situation as @JabJab stated. 6 if you were in SBG2. 3 is for other affected DCs.

  • @Makenai said:

    @SirFoxy said:
    3

    Its likely depending on the situation as @JabJab stated. 6 if you were in SBG2. 3 is for other affected DCs.

    three

  • deankdeank Member, Troll

    Threesome.

  • @jsg said:
    Meanwhile ...

    @cociu who seems to have started, let's call it "technically very modest", and ran basically just like OVH ("sell! sell! sell! ... out of a crap box, who cares, just sell!") but of course on a much smaller scale and without the fat make up layer of OVH marketing, ...

    ... has a quite decent DC in a real industrial building, has very significantly enhanced his support, and now has very decent Telia connectivity (I'm currently running benchmarks).

    ... while OVH is forced to do a desperate move (3 - 6 months free) to stop bleeding customers in droves (To quote one competitor:"We've sold our monthly server stock within 2 days" and they even didn't advertise). ... just pre-IPO.

    So my $7 is on cociu, not on OVH.

    The 2 mentioned providers aren’t even closely comparable. Not even close.

    Thanked by 2pbx darkimmortal
  • jsgjsg Member, Resident Benchmarker

    @corbpie said:
    The 2 mentioned providers aren’t even closely comparable. Not even close.

    A couple of days ago I'd have agreed.

    Now however I see the main difference in one growing only quantitatively while the other one, HS, also very considerably grew in terms of quality.

  • @corbpie said: The 2 mentioned providers aren’t even closely comparable. Not even close.

    Cociu is possibly one of the closest match you could find here on LET. I wonder who else here started from scratch, laying fiber and everything apart from him, I bet not too much.

  • yoursunnyyoursunny Member, IPv6 Advocate

    @SCAM_DONT_BUY said:
    Cociu is possibly one of the closest match you could find here on LET. I wonder who else here started from scratch, laying fiber and everything apart from him, I bet not too much.

    I can dig a tunnel to lay an IDE cable 20km long, put a hard drive over there, and connect the cable to the RAID controller. Clients don't need backups. If my data center burns down, I'll send the other hard drive to the client via certified mail.

    Thanked by 10xbkt
  • @JabJab said: Free/Paid Backup pCC in SBG1 was hosted in an separated room of SBG1. Both rooms are destroyed.

    They... stored the backups in the same building? 🤔

    @miroc said:
    I thought all datacenter rooms must have fire protection system.

    I also thought data centres weren't built out of shipping containers, but here we are. https://www.datacenterknowledge.com/archives/2012/06/13/ovh-deploys-container-cube-data-center

    Thanked by 1pbx
  • ShazanShazan Member, Host Rep

    @DianTama said:
    with this incident, I begin to worry and wonder how their Singapore DC looks like...

    Standards in Singapore are usually higher than Europe. You can not find a facility with such a low level of security in Singapore.

  • bobbrbobbr Member

    Being unfamiliar with datacenter construction, let me ask this.

    What mostly fueled that huge blaze, an inferno the size of Building 2? It wasn't the containers, which are steel. It wasn't the racks, which are steel. It probably wasn't the flooring, which probably wasn't wood or something very flammable.

    What part of a server is flammable? Plastic parts, I suppose. Not the circuit boards. I've seen charred parts of PCBs, but fire doesn't spread, the material isn't exothermic. Copper wire and glass fibers aren't flammable, but their plastic insulation might be. Electrolytic capacitors? All of that sounds more or less like small change.

    So, what would amount to tons of fuel for those huge flames leaping high above the buildings? Diesel storage tanks for the backup generators, maybe?

  • deankdeank Member, Troll

    @bobbr said:
    Being unfamiliar with datacenter construction, let me ask this.

    What mostly fueled that huge blaze, an inferno the size of Building 2? It wasn't the containers, which are steel. It wasn't the racks, which are steel. It probably wasn't the flooring, which probably wasn't wood or something very flammable.

    .... The floors were wood.

    Aye, they were wood.

  • MechanicWebMechanicWeb Member, Patron Provider

    @Daniel15 said: I also thought data centres weren't built out of shipping containers, but here we are. https://www.datacenterknowledge.com/archives/2012/06/13/ovh-deploys-container-cube-data-center

    It seems the hole is just getting deeper.

    At this point, I think all their data centers should be thoroughly investigated to find out what other bad practices they are employing to cut corners.

    'Air cooled' servers without chassis?

  • Daniel15Daniel15 Veteran
    edited March 2021

    @miroc said:
    I thought all datacenter rooms must have fire protection system.

    @deank said:

    @bobbr said:
    Being unfamiliar with datacenter construction, let me ask this.

    What mostly fueled that huge blaze, an inferno the size of Building 2? It wasn't the containers, which are steel. It wasn't the racks, which are steel. It probably wasn't the flooring, which probably wasn't wood or something very flammable.

    .... The floors were wood.

    Aye, they were wood.

    I thought you were joking... Then I saw this:

    what.

    Source: Twitter:

  • MechanicWebMechanicWeb Member, Patron Provider

    @MeAtExampleDotCom said: Who/what is/n't to blame for the fire does not change that a proper backup strategy would mean no data was lost (or at least would make it far more unlikely data is lost, cascade failures are a thing). If you have no good backups for data at any DC you take this risk, so the advice on backups is relevant to help prevent repeat losses.

    If you don't want to listen to us "wise guys", feel free to not listen. But be aware that we'll enjoy saying "we told you so" if you are affected by a problem like this in future.

    You are missing one key element in your argument. Those who had servers in SBG and didn't have backups already know that they should have had backups. You do not need to tell them again.

    Not to mention that saying the obvious on their face at this point is cruel.

    But talking more about the irresponsibility's of OVH might force them to think twice before deploying their next DC.

    In your arguments, try to talk more about who is responsible for burning those servers, that's where the blame should be put at.

  • deankdeank Member, Troll

    I don't actually blame OVH for the fire. I mean, they had to cut corners. They had to.

    Don't want to know what caused the fire, either, because I am pretty sure it was electrical fire. That's the only one that'd ignite the wooden floors vigorously so fast to a point that, when the fire trucks showed up, the whole building was up in smoke.

    I just like the drama and pointless cries of demanding data back from molten servers from SBG2.

    No one was hurt. I bet no one was there to begin with. That's all that matters.

    I am a troll after all.

  • MechanicWebMechanicWeb Member, Patron Provider

    @deank said: I don't actually blame OVH for the fire. I mean, they had to cut corners. They had to.

    You could blame the fire ;)

  • tree fiddy final offer

  • deankdeank Member, Troll

    Nah, fire arrive only when invited.

    They were given permission to have a party and they certainly did.

    Thanked by 1MechanicWeb
  • Wooden floors cannot burn down. Only small flames, no more than a birthday cake. 3.6 roentgen nothing bad will happen.

    Thanked by 1bulbasaur
  • yoursunnyyoursunny Member, IPv6 Advocate

    @SirFoxy said:
    tree fiddy final offer

    Premium tree fiddy offer available here:
    https://treefiddy.yoursunny.cn

  • @MeAtExampleDotCom said:

    @TimboJones said:
    What backwards country doesn't have fire suppression requirements by law?

    The law will demand that the workers nearby others are sufficiently protected, and they were: no one was hurt and nothing beyond SBG* was on fire. Fire suppression is a detail that the legislation won't necessarily demand if other sufficient preparations are in place.

    It is different for residential dwellings and public places, though even then things are a lot more iffy than you might like (flammable cladding on the outside of blocks of flats being a-OK until recently in the UK for instance).

    So, 1970's thinking.

  • bobbrbobbr Member

    OVH's restoration plan looks like it involves repairing and reconnecting an incoming 20KV electrical line.

    Let's see... 30,000 servers, was it? At say120V and around 1A each? So that might be 120V at 30,000 Amps as a ballpark figure. That's 3.6 Megawatts, which would be 20KV at 180 Amps.

    Let's say that incoming power was routed through the UPSes somehow. (I really have no idea how it was wired up.) If the UPSes developed some issue and were unable for whatever reason to keep handling it properly or to gracefully shut down, then they could have become overheated, as noticed on the FLIR infrared camera as the hottest spots in the container.

    Let's say you're a piece of nicely kiln-dried and aged pine wood (or pick your favorite kind of cheap wood) and you have had a long career as part of this third-hand shipping container. What do you do when the UPS unit sitting directly on you, or standing above you a few inches away, is radiating an extreme amount of heat because of the megawatts pouring into its damaged circuits? Yes, you start to smolder a little at first. Then as your nice dry fibers reach, say, just a tad over 450 degrees Fahrenheit... well, what would you do if you were this fine upstanding piece of wood?

    Of course this scenario could be completely wrong, and all the wood might be completely sound and untouched. In that case, could anyone suggest what stuff went up in all those flames instead?

  • HostStageHostStage Member, Host Rep
    edited March 2021

    @deank said:
    I don't actually blame OVH for the fire. I mean, they had to cut corners. They had to.

    Don't want to know what caused the fire, either, because I am pretty sure it was electrical fire. That's the only one that'd ignite the wooden floors vigorously so fast to a point that, when the fire trucks showed up, the whole building was up in smoke.

    There was an intervention on an UPS the very afternoon the fire started.
    The firemen through a infrared device were able to notice that it came from an / those UPS.
    Now you know

    Thanked by 1that_guy
  • ClouviderClouvider Member, Patron Provider
    edited March 2021

    @bobbr said:
    OVH's restoration plan looks like it involves repairing and reconnecting an incoming 20KV electrical line.

    Let's see... 30,000 servers, was it? At say120V and around 1A each? So that might be 120V at 30,000 Amps as a ballpark figure. That's 3.6 Megawatts, which would be 20KV at 180 Amps.

    In Europe we do ~230V AC in standard installation.

    Thanked by 1yoursunny
  • SaahibSaahib Host Rep, Veteran

    What started this fire.. bad UPS ?

  • I feel specially sad for those that had full backups - in the same server farm.

    Thanked by 3yoursunny pbx lentro
  • LeviLevi Member

    Does anyone know how many servers was in that DC?

  • FalzoFalzo Member

    @LTniger said:
    Does anyone know how many servers was in that DC?

    supposed to be 30k. at least that's the suggested capacity ... probably a mix of what has been rented out directly and what was part of their cloud/areas.

    UPS failure -> gas (hydrogen), sparks, ignitions, explosions, fire.

    probably does not work so well if wooden constructions are close by...

    Thanked by 3vedran lentro that_guy
  • Will be interesting to see what CPUs will be used to replace them (when the time is right)

  • LeviLevi Member

    @corbpie said:
    Will be interesting to see what CPUs will be used to replace them (when the time is right)

    They will harvest those in sbg2, clean them and you are good to go :D

Sign In or Register to comment.