1GServers outage?

concept

@OhJohn said:
I was wondering about GTHost as well as they moved DCs in Phoenix last month....

Had they been in PhoenixNAP before?

From what I know, they are connected to PhoenixNAP but no servers there. They are online still.

OhJohn

yes, GThost in PHX is online and was online all time today.

corpdecker

My server just came back online, uptime of 11 minutes, so it was def power cycled.

Maximus

@corpdecker said:
My server just came back online, uptime of 11 minutes, so it was def power cycled.

Mine still down

parapara

4 of em for me still down. 1 up.

aru

1 up 26 minutes ago, but
Package id 0: +99.0°C (high = +93.0°C, crit = +103.0°C)

shutdown now

a2razor

Machine came back online here:

Chassis: 47°C

CPU 1 / CPU 2: 62°C / 55°C

Motherboard NIC sensors: 67°C / 67°C

Peripheral area: 51°C

DIMMs: up to 56°C

SSDs: 46°C primary, 43°C secondary -- these aren't M.2's, they got up to 70c earlier according to BMC logs but seem back down now.

Fans: 10,700 – 11,400 RPM

Temps way down from earlier.

a2razor

@corpdecker said:
My server just came back online, uptime of 11 minutes, so it was def power cycled.

Yep, abrupt shutoff observed here too, not a clean shutdown according to logs.

paradoxo

Damn, my server is at 62°C right now, and the network is still unstable. I hope they fix it as soon as possible 😥

thane

According to LiquidWeb we may be looking at 10-16 hours of downtime here: https://status.liquidweb.com/

Mainfrezzer

Holy moly. Fun

hekris

@paradoxo said:
Damn, my server is at 62°C right now, and the network is still unstable. I hope they fix it as soon as possible 😥

How to check the temperature when the server is powered off and cannot be accessed?

Maximus

still down. 3+ hours down and zero email or call answering. Would you trust this company in the future if anything goes wrong? I wouldn't.

freelanceonline

@OhJohn said:
ok, at least i seem to have the servers still in the old colo location and not the new DC.

And here is the status page by phoenixnap: https://status.phoenixnap.com/

Incident: https://status.phoenixnap.com/incidents/dnjp5pfv2mmh

It looks like it has a DDoS attack flow.

concept

@freelanceonline said:

@OhJohn said:
ok, at least i seem to have the servers still in the old colo location and not the new DC.

And here is the status page by phoenixnap: https://status.phoenixnap.com/

Incident: https://status.phoenixnap.com/incidents/dnjp5pfv2mmh

It looks like a DDoS attack.

Site works for me.

iohn

what happened? all of sudden my server rebooted and i can't ssh into it anymore

thane

They restored more cooling capability, seeing some servers come online again and temps dropping. Another major cooler is expected to be installed in about 2.5 hours, hopefully servers dont take that long to be safe to boot back up...

1743528077

Since the reboot, my server’s outbound TCP traffic has been limited to about 56 KB/s with high packet loss, while inbound speed and the local NIC remain normal.

1743528077

@1743528077 said:
Since the reboot, my server’s outbound TCP traffic has been limited to about 56 KB/s with high packet loss, while inbound speed and the local NIC remain normal.

I don't know if I'm the only one like this; is there anyone else?

szarka

@1743528077 said:

@1743528077 said:
Since the reboot, my server’s outbound TCP traffic has been limited to about 56 KB/s with high packet loss, while inbound speed and the local NIC remain normal.

I don't know if I'm the only one like this; is there anyone else?

I don't even have signs of life.

1743528077

@szarka said:

@1743528077 said:

@1743528077 said:
Since the reboot, my server’s outbound TCP traffic has been limited to about 56 KB/s with high packet loss, while inbound speed and the local NIC remain normal.

I don't know if I'm the only one like this; is there anyone else?

I don't even have signs of life.

Don't worry, at least it didn't catch fire.

1743528077

Their official website has yet to come back online—it is already well past 10 a.m. there, which is truly baffling.

emgh

insane to me how companies like namecheap and namesilo seemingly have absolutely 0 redundancy and also 0 disaster recovery measures when a singular DC is down

OhJohn

Again, this is the current update stream:

@emgh said: insane to me how companies like namecheap and namesilo seemingly have absolutely 0 redundancy and also 0 disaster recovery measures when a singular DC is down

yepp, that is embarrassing, esp. if you think about their size. (But CF centralization in Portland is even more embarrassing if we recall their outage a few years ago)

zGato

@emgh said:
insane to me how companies like namecheap and namesilo seemingly have absolutely 0 redundancy and also 0 disaster recovery measures when a singular DC is down

Them actually calling it an "emergency maintenance" is hilarious. Like how hard is it to call it for what it is? An outage?

Mainfrezzer

@zGato said:

@emgh said:
insane to me how companies like namecheap and namesilo seemingly have absolutely 0 redundancy and also 0 disaster recovery measures when a singular DC is down

Them actually calling it an "emergency maintenance" is hilarious. Like how hard is it to call it for what it is? An outage?

I'm gonna shit bricks if it's because of some weird terms that outages are reimbursed differently

concept

LOL

Just to think if the dc goes up in flames like OVH Strasbourg, there will be nothing left. They are hosting everything out of PhoenixNAP.

OhJohn

@concept said: Just to think if the dc goes up in flames like OVH Strasbourg, there will be nothing left. They are hosting everything out of PhoenixNAP.

With no backups - YOLO

OhJohn

Let's just hope that an unpingable mail server IP does not mean the mail server is also down... (this in regards to 1gservers, not namecheap).

concept

@OhJohn said:

@concept said: Just to think if the dc goes up in flames like OVH Strasbourg, there will be nothing left. They are hosting everything out of PhoenixNAP.

With no backups - YOLO

Surely a company with a 1.5 billion dollars valuation and making nearly $400 million in revenue would have their own DC by now or some kind of backup strategy.
https://hostdean.com/news/cvc-capital-partners-majority-stake-namecheap/