New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
He just copied from his own email to clients....
Is that an issue?
Those things demand time, I understand this but it's really annoying to read all those days how they suggest clean container reinstall to speed up proces instead restore from backups and yet they ignore those clean vps reinstall requests but constantly bomb us with more or less meaningless RFOs how many VPSs they restored from backup already.
@CVPS_Chris why don't your guys get up after 5 days of downtime atleast those who requested clean VPS reinstall?
This page always gets unformatted to me. Seems the others are OK...
Now, how can you know that no VPS data has been compromised if the database containing the passes has been leaked ?
I think you cant.
@MrObvious why does it hurt you?
Your love for cvps is obvious and it may hurt you, but nobody but you likes Chris.
What a shitshow of a recovery process. It's been 5 days of straight downtime, multiple support tickets unanswered for days, and I still have NO idea if my data has been lost or is recoverable.
While I do appreciate all the effort they have put into getting their services back online, I cannot feel like they have a skeleton crew of people working for them given the amount of time this has taken.
All I really want to know is if my data has been completely lost or if there is a chance of recovery. I have asked multiple times for a definitive answer to this, but have got nothing but the runaround and vague responses.
I've already switched providers due to this ongoing downtime as I had no choice but to get my clientele back online and couldn't afford to wait around for CVPS' false promises and timelines to elapse.
@ouija not to mention that after 5 days of downtime they aren't capable to do even fresh VPS reinstall to speed up recovery process.
Well this is fantastic. Finally received a response and have been informed that it's going to be at least another 3-4 days (or let's say a week given CVPS' concept of time) before my data is at least attempted to be restored (if it can even be restored in the first place).
Guess I'll just bite the bullet and begin rebuilding my databases from scratch
You have no idea the amount of work that went into getting everything back to where it is today. Any other provider damaged like this would take much longer, I can assure you that.
Please keep your panties on. We are doing our best.
Where were you told this? It should not take that long
All tickets are receiving fast personalized responses and that a tremendous effort is being dedicated to getting everyone back online and restored on request. Thanks everyone for your patience.
My major issue is that I was in the process of a website redesign for one of my clients for a few weeks leading up to this fiasco, and had literally been working day and night on this right up until SolusVM was hacked, and unfortunately my most recent backup of this database is outdated by this week in which the majority of the work had been completed.
I just want a definitive answer if my server data can be restored to a point where I can access this MySQL database and get the data I need. Otherwise, I will have to face the reality of needing to rebuild from scratch.. If it's going to take another 3-4 days to even find out if this is possible, then I might as well cut my losses here and now and write off your service entirely.
Update: Just received another reply that restoration might be within 6 hours. That is quite an improvement over 3-4 days. Now let's just see if that holds up.
@CVPS_Chris I understand that things demand time but why lies? Your support guy keep repeating: "At this time virtually all customers are back online; some with all original files in tact and others with new containers.
To clients who are offline 5 days already this answer don't really help to calm down, don't you think?
Also you keep repeating over last few days how many clients data are restored and yet your team isn't capable to do explicitely requested fresh VPS reinstalls to speed up recovery process. Why is that? After 5 days of downtime? Because you work too hard to restore data and those who requested fresh VPS reinstall to start to work with VPS sooner can wait longer? I fail to see logic in that.
I keep thinking of Virpus and when they had 19 nodes wiped in 2011. They needed about a week to create new containers to their customers and all customers were given new containers. A restore always takes longer time then to create an empty new container. So I feel for all of you during this process.
We are actively restoring backups for customers who have requested it.......
further updates to follow.
From FB:
So as I was on an ATL node, I can assume that everything I had there was more or less lost? I want to ask if there is any compensation planned? I didn't really have anything important, but I sure don't feel like setting it all up again if there isn't any compensation planned for those who's data was destroyed.
I appreciate your efforts, and I understand this was in many ways not exactly your fault.
What a load of crap. I've been with ChicagoVPS for a long time. They lost my data in November and probably now too. I don't have access to my VPS unless you count a web control panel that says offline and has a "Boot" button that doesn't actually boot my vps. I have submitted several tickets in the past few days and today and never received a personal response. I'm sure ChicagoVPS makes a lot of money by not hiring a support staff. Great a marketing, just put out a bunch of BS saying how everyone is up and getting great support. Lies, lies, lies.
Erik,
That is very untruthful as we have a 5 man support team, ALL American based. As for the control panel, most providers have it shut off because of the on going issues. I dont think anyone wants any chance of being vulnerable to a compromise.
Both hacks both fall back and relate to the same vulnerability, if you dont believe me, you can search this forum and see how I was called a liar for the past 8 months.
I ask that you have a little more understanding as I think myself and the support team got everything back up and running in a reasonable time considering the amount of work we completed.
mine have received copy and paste replies
I opened a ticket to requets it a couple of days ago (which is more than 6-12 hours). Instead of a restoration of a backup I got a new install with the wrong OS distro.
But everything isn't normal as my VPS is still down. I've been down nearly a week and have received nothing but copy/paste canned responses to my support tickets so you ARE understaffed otherwise my vps would be up by now. Too many customers, not enough staff. Obvious to me. I guess this is acceptable to you since nobody really expects the 99.9% uptime that you guarantee for a low end vps.
I'm aware of the issues with SolusVM and also I'm aware that many other providers haven't been down all week. Rumor has it the first time was caused by a weak brute forced password, and that this time you failed to delete a vulnerable file in time. I'm aware you deny these things, but you've lost credibility with me.
Rumours are just rumours.
One thing is certain, nobody in this market can have enough staff to make up for a general catastrophe like this.
5-6 people should be enough to manage day to day operations even with so many customers.
Nobody can have "reserve" staff for events like this, but I do agree that, even so, it took way too long so far.
100 servers to restore with 5 people and 3-4 hours on a server (that much it took for ours when raid failed) a similar 32 gb ovz e3 would mean a couple of days, 3 at max, considering not all were wiped, considering those people need to eat (sleep might be skipped or done 1-2 hours a day for a couple of days), need to shower, etc, I would say 3 days is a very generous time in which it should have been expected to have all back up.
After the second server you make a script, or know the drill so well that you can work on more at once.
As for the staff being American based, what is that supposed to mean ? That Europe has bad technicians ? Can't have good support staff ? Doesnt know enough English ? Does this include South America, Carribean and central america ?
If they are all American and this is such a great thing, then they dont need 3 days, will do the job in just two.
It's sad that the remaining downtime before my service is restored has to be measured in multiple DAYS.
Most companies in America have/will advertise as "American" support -- telcos, utilities, cablecos, etc. because it plays into the "buy American" thing that used to be all the rage and because it's a way of saying that "our support isn't outsourced" -- I assume that's what he means here.
I used to with them when they posted a $5 offer for 2048MB OpenVZ possibly last year. At first, the virtual machine and the network were really good in LA, then when more people take the deal, the oversale was a crap and there was an all day downtime later, due to "broken parts taken down". But that's only $5, so you should expect some downtime or even some data loss.
But isn't their support "outsourced" since they had ColoCrossing help them out?
I thought ColoCrossing said Chris was a good client of theirs? Sorry if I'm late to the party.
No need to start with that again. ChicagoVPS and their clients have this moment other worries. Lets keep this thread clean for downtime and data restore updates.
I got some personal replies at the very beginning. Now its all copy and paste crap about they are working hard to get to my request, blah blah blah. All I want to know is if my data was or was not lost. I do have a VPS now, with the wrong OS and cannot get it fixed. They restored it with the same root password that was leaked... SMART!
Can I get CentOS 6 fresh on my VPS? Do you have my data or not? I don't feel like wasting anymore time. Since you have answered more customers on the public boards than in your own support system maybe I'll finally get some answers.
Every time I request a restore from Central Backup I get a canned response to use my own backups or wait 3-4 days. Then the ticket gets flagged as "answered" and nothing happens.
Your post on facebook says this process should take 6-12 hours if used Central Backup. I did use the Central Backup facility for my VPS and want it restored. I have been asking this for days.
Chris,
THANK YOU FOR REDEEMING YOURSELF IN MY EYES.
My VPS has finally been restored, within the timeline that I was actually given to boot!
Some of my data has been lost, however, but that was to be expected. But the data I was most concerned about seems to be unaffected, and I am relieved that I've (or should I say CVPS) managed to recover it!
Thank you and your team for all your hard efforts over the past week. I know that a lot of your userbase (present company included) has been giving you a lot of grief that just adds to an already stressful and busy situation, and I want you to know that your efforts are TRULY appreciated and haven't gone unnoticed!
Please, for the love of god, NEVER go back to using SolusVM (which seems to be the direction you are heading) to avoid a terrible scenario like this in the future, and hopefully all of your efforts won't go unrewarded! Really really appreciate this, and words cannot express.
At lot of my frustrations were just with the mixed timelines and unknowing uncertainty if my data was compromised. I apologize for being impatient and adding to your levels of stress, and wish I could have waited longer for your support but was becoming frustrated with the ever increasing turn around time. I hope you understand, and wish you all the best in your remaining recovery process.
Thanks again.
to have a guarantee VPS the best offer is 4USD/month