Down - OVH - SBG - Lots and lots of tears.

xaoc

This: http://travaux.ovh.net/?do=details&id=49471&;

hellb0y

The end is neigh :~

xaoc

@hellb0y said:
The end is neigh :~

It is.

http://travaux.ovh.net/?do=details&id=49484&;

WSCallum

Ouch, major fire at SBG2 - Not good.

yoursunny

@WSCallum said:
Ouch, major fire at SBG2 - Not good.

Blame the FIRE THREAD on LET.

K4Y5

We are currently facing a major incident in our Strasbourg datacentre, with a fire declared in the SBG2 building.
Firefighters intervened immediately on the spot but were unable to control the SBG2 fire.
As a precautionary measure, the electricity was cut off on the whole site, which impacts all our services at SBG1, SBG2, SBG3 and SBG4.
If your production is in Strasbourg, we recommend that you activate your Business Recovery Plan.
All our teams are mobilized alongside the firefighters.
We will keep you posted on this as soon as we have more information.

Oh well..
/me looks at the LE entrepreneurs losing billions due to lack of disaster recovery plans in place.

TL;DR Version:
Always, always and ALWAYS have back-ups in place. Depending on criticality, you may want to setup hourly, daily or weekly back-ups and store the files off-site. Preferably, at a different location / DC and provider.

sin

Ouch.

Moofie

Maybe this will finally get me to setup backups on my servers lol

JabJab

Planned! #insurance scam

I guess I will need to reinstall my VPS soon. I really hope that all of this is just a 'procedure' and not whole building is on fire...

No juicy photos/drone videos from site? Can't be that everyone is sleeping there, get on some French news page!

valk

Yikes! Literally

Jasonhyperhost

Octave OVH CEO has posted on his twitter the below statement /cry

https://twitter.com/olesovhcom/status/1369478732247932929

coolice

Plan is activated but IP movement takes forever -

You cannot move this IP, as it is already in the process of being transferred. -

Does someone manage to move IPs out ?

WSCallum

@JabJab said:

Planned! #insurance scam

I guess I will need to reinstall my VPS soon. I really hope that all of this is just a 'procedure' and not whole building is on fire...

No juicy photos/drone videos from site? Can't be that everyone is sleeping there, get on some French news page!

It’s SBG2 and seems to have affected whole of SBG2 according to octave on twitter, 1,3,4 i assume will be back up when its safe to restore power

lentro

holy shit. that's a huge nightmare situation.

JabJab

@WSCallum said: , 1,3,4 i assume will be back up when its safe to restore power

Problem is that those all buildings are connected. Bah, you can even say it's one building.

https://www.google.com/maps/place/9+Rue+du+Bassin+de+l'Industrie,+67000+Strasbourg,+Francja/@48.5858015,7.7975347,193m/data=!3m1!1e3!4m5!3m4!1s0x4796c8daab82f9af:0x538b85a8eb23fadb!8m2!3d48.5858131!4d7.7973832

JabJab

https://twitter.com/olesovhcom/status/1369504527544705025

Update 5:20pm. Everybody is safe.
Fire destroyed SBG2. A part of SBG1 is destroyed. Firefighters are protecting SBG3. no impact SBG4.

WSCallum

@JabJab said:

@WSCallum said: , 1,3,4 i assume will be back up when its safe to restore power

Problem is that those all buildings are connected. Bah, you can even say it's one building.

https://www.google.com/maps/place/9+Rue+du+Bassin+de+l'Industrie,+67000+Strasbourg,+Francja/@48.5858015,7.7975347,193m/data=!3m1!1e3!4m5!3m4!1s0x4796c8daab82f9af:0x538b85a8eb23fadb!8m2!3d48.5858131!4d7.7973832

Interesting, never looked at the plans so didn't know they were almost wall to wall connected, assumed they'd be separate "buildings" in a small area as such.

That'll make it more interesting then...

serv_ee

@WSCallum said:

@JabJab said:

@WSCallum said: , 1,3,4 i assume will be back up when its safe to restore power

Problem is that those all buildings are connected. Bah, you can even say it's one building.

https://www.google.com/maps/place/9+Rue+du+Bassin+de+l'Industrie,+67000+Strasbourg,+Francja/@48.5858015,7.7975347,193m/data=!3m1!1e3!4m5!3m4!1s0x4796c8daab82f9af:0x538b85a8eb23fadb!8m2!3d48.5858131!4d7.7973832

Interesting, never looked at the plans so didn't know they were almost wall to wall connected, assumed they'd be separate "buildings" in a small area as such.

That'll make it more interesting then...

Dont think "interesting" is the right word right now..

lentro

@WSCallum said: almost wall to wall connected

P sure they are basically a huge building.

https://cdn.baxtel.com/data-center/ovh-strasbourg-campus/photos/OVH-Strasbourg-Campus-OVH-Strasbourg-Campus-showing-all-3-Buildings-2019.png

Vova1234

This is sudden. In SBG, mainly game servers dropped out: i7-4790k, Ryzen 5/7, i7-6700k, i7-7700k.

There was already a global incident in 2017. But for everything to burn out in 2021.

JabJab

http://status.ovh.com/vms/index_sbg2.html

Nice monitoring you got there /s

servarica_hani

if you are affected and urgently need VPS for few days you can order from us and cancel in 7 days and you will get full refund
If you need more time pm me will figure something out

if you are looking for storage vps also pm will provide it although at this moment the performance is not great as we are doing major moves to update our storage servers

WSCallum

Anyone who has servers in SBG2 it's pretty much guaranteed toast, SBG1 partial destroyed basically according to Oles on Twitter:

Update 5:20pm. Everybody is safe.
Fire destroyed SBG2. A part of SBG1 is destroyed. Firefighters are protecting SBG3. no impact SBG4.

valk

Hope it doesnt escalate to SBG3, or I piss my pants!!!!

xaoc

@WSCallum said:
Anyone who has servers in SBG2 it's pretty much guaranteed toast, SBG1 partial destroyed basically according to Oles on Twitter:

Update 5:20pm. Everybody is safe.
Fire destroyed SBG2. A part of SBG1 is destroyed. Firefighters are protecting SBG3. no impact SBG4.

5:20pm, eh?

mike1s

If anyone needs assistance, please reach out to me. I know we're very far from this OVH datacenter location, but if there's anything that can be done, I'm glad to help.

serv_ee

Guess the French gov. needs to open up their wallet again after this.

nurex

The word "destroyed" really shocked me. Hope everyone is OK.
I recall there was a mega power outage several years ago, and today another major incident hits. Sad enough.

NoComment

ovh needed better firewalls

serv_ee

Reading the Twitter comments just...smfh. People really are idiots.

xaoc

@serv_ee said:
Reading the Twitter comments just...smfh. People really are idiots.