Anyone else having issues with SolidVPS? / Solidvps.com down?

Mainfrezzer

@forest said:

@rpqu said:

@SolidVPS said:
Our entire infrastructure, including all communication systems, is located in the same datacenter that was affected. Our focus over the past 24 hours was not only to restore service as quickly as possible, but to ensure the issue is permanently resolved.

Zero backup == Zero worth

Restoring a huge infrastructure from backup (plus verifying and testing things) can easily take 20 hours. If there was no backup, I'd expect it to be a lot worse than under a day to recover.

Having a backup SMTP or status page is now absolutely not 20 hours of work. We don't wanna pretend that's much work.

Let alone that external communication system exist. Hello, welcome to lowendtalk. But I reckon there was surely something important going on for all non-technical personal. Classic d-class.

rpqu

@Mainfrezzer said:

@forest said:

@rpqu said:

@SolidVPS said:
Our entire infrastructure, including all communication systems, is located in the same datacenter that was affected. Our focus over the past 24 hours was not only to restore service as quickly as possible, but to ensure the issue is permanently resolved.

Zero backup == Zero worth

Restoring a huge infrastructure from backup (plus verifying and testing things) can easily take 20 hours. If there was no backup, I'd expect it to be a lot worse than under a day to recover.

Having a backup SMTP or status page is now absolutely not 20 hours of work. We don't wanna pretend that's much work.

Let alone that external communication system exist. Hello, welcome to lowendtalk. But I reckon there was surely something important going on for all non-technical personal. Classic d-class.

Yes. The point was about h2h communications blocked as it's trapped within failing system. Early alert + apology is practical.
Now, on the m2m. What @forest said is right, restoring from backup, verifying the integrity, testing the fabric took time. But this exceed the double 9 (month) and triple 9 ( year) SLA. That means they should evaluate their recovery plan + conduct rehearsals

jackboy

This is n00b hour stuff, they lost my business over this stunt. We are just going to cleanly gloss over them gaslighting the first people and blaming the customer? They have no way to communicate to anybody and live under rocks? No staff to do damage control until after the fact? Pathetic. Communication is key. No matter how good your deals are, going down for ~24 hours in radio silence is priceless.

KeyGenMe

Appreciate the follow-up, @SolidVPS!

Having your website, billing, and communication channels hosted in the exact same facility that went dark is definitely the classic sysadmin trap — «Hard to tweet about the fire when the fire hose is locked inside the burning building“.

Definitely a good reminder to throw a lightweight status page or mirror somewhere off-site for next time, just so people don't start drafting eulogies in the forums.

Jokes aside, respect for showing up, giving a detailed post-mortem, and fixing the actual architecture instead of applying duct tape.
Everything is back up, credits are sorted, and the context is clear. I Hope the incident is fully resolved!)

@angstrom could you please close this thread? Time to let the uptime roll!