Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


ChicagoVPS hacked - Page 8
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

ChicagoVPS hacked

168101112

Comments

  • Those who are yelling opensource/hypervm blah blah.

    http://www.itwire.com/business-it-news/security/25559-hypervm-boss-hangs-himself-after-exploit-damages-100000-websites

    Everyone and everything has issue. What makes you better is how you handle it. Look at whmcs, they had some intense hack in the past, wiping out providers such as K-disk. But they are still going strong.

  • @mpkossen what I find most interesting is that it does not make use of PDO. If the code has that many spots without proper sql procedures then we are looking forward to a long battle.

  • LeeLee Veteran

    @rds100 said:
    Until you remember that OVH's control panel got hacked just a few months ago and Hetzher's robot got hacked just a few weeks ago :)

    Indeed, however I have yet to hear of anyone loosing any data as a result of either incident. Unlike here where it is the complete opposite.

    And to be fair people should be quite thankful in some ways it happened to ramnode, not that I would wish it on anyone however if it is going to happen to someone I use then I want it to be someone that can deal with it and recover.

    Could you imagine the result of this for some providers on here, it would have if nothing else revealed just how amateurish they actually are.

  • @zulualpha said:
    Not really the biggest issue in all this, but looking at the data leak, there are some REALLY bad passwords.

    how are you finding the passwords from the hashes? And what sort of hashes are they?

  • AnthonySmithAnthonySmith Member, Patron Provider

    @mpkossen said:
    I've seen some code allegedly from SolusVM and if they can say it's secure this fast, I doubt they did a thorough search. The code I saw, well, was quite horrible and had a lot of potential security issues. Besides, mysqli_real_escape_string was a concept unknown to the person that wrote that code. That's PHP/MySQL 101 and even in the PHP docs. Also, they escaped every variable manually rather than using a centralized function, making it even harder to test/check.

    Well it is not like it has changed much and the style of code has never changed, and potential security issues do not necessarily mean actually exploitable, i.e. simply having the exec function does not mean someone can exploit it.

    If there are further exploits out there then they have not been made public so what are we supposed to do, just leave the solusvm masters down for good or switch.... I am still saying something is fishy about this, solusvm have not confirmed they are assisting cvps, and cvps have pulled this blame solusvm in the past.

    The flip side of the coin being.... I do not want to be first to bring them back online, that said a ton of hosts have remained online and so far unaffected.

    so @CVPS_Chris why are you saying it was a solusvm exploit... just because you dont know what else to blame or do you know this for a fact, if I had to guess.. you got hacked and backdoored before you patched the centralbackup.php

  • @upsetcvps said:
    how are you finding the passwords from the hashes? And what sort of hashes are they?

    i think he is talking about the root passwords for the vps's

  • MaouniqueMaounique Host Rep, Veteran

    They are SHA1.
    I dont need to decrypt them, while investigating hack/scan reports i get to find files with paswords discovered by bots running at random, things like changeme and 123456.
    So many braindead people these days.

  • zulualphazulualpha Member
    edited June 2013

    @PcJamesy said:
    i think he is talking about the root passwords for the vps's

    Yep, some are hashed & others are not. I think if you changed your password from the original one that was sent when you signed up, it's probably visible. To be fair, some of these passwords are also really impressive 50 character long monstrosities....which I guess isn't much use when they're leaked as plain text.

  • Best root password I found "root"

  • DomainBopDomainBop Member
    edited June 2013

    @Jack said "who's lost data?"

    So far all 3 of the known hosts who were hit by this exploit have lost at least some of their customers data:


    1. Host1Free- 100% data loss of all 17000+ customer accounts and no backups

    2. RamNode- at least 1 of their nodes was wiped and people have posted here they lost data. The node ! had a VPS on was wiped (a backup of my data has now been restored...on HostVirtual)

    3. CVPS- unknown yet, but it would be a miracle if there's no data loss since some nodes (including at least 1 I had a VPS on and possibly 2) were apparently wiped.

  • SpiritSpirit Member
    edited June 2013

    @AnthonySmith said:
    Just had a confirmation from SolusLabs that no further vulnerable code is known about and that the only known exploitable code was the centralbackup.php file.

    What that means is that either CVPS is not being truthful or SolusLabs are not being truthful.

    Not necessary if hack happened prior centralbackup.php removal. Many host here patched centralbackup.php vulnerability very fast. But not at same moment it was posted. Not in the same moment people saw it. There could be plenty hosts who reacted and removed centralbackup.php fast hacked already and just because nodes aren't deleted, databases aren't publicly posted it doesn't mean that they aren't "owned" one or another way already with this same vulnerability without knowing it.

    tl,dr If you removed centralbackup.php fast and your database will leak in some week or.. month logical conclusion will be that there's new vulnerability, correct? But what if hack happened already and bad guys have your data already? Prior centralbackup.php removal? Just hypothetical situation of course.

  • @akz said:
    sigh, is chicagovps at least looking into replacing solus with a different panel?

    I love @serverian's frontend to proxmox. There are a few good security practises here too, like not sending the root password by mail, not allowing to change the root password on the panel. If you lose access to the vm, so does anyone else gaining access to the panel. They'd have to reinstall the vps to use it. No access to the data.

  • @Holoshed said:
    mpkossen what I find most interesting is that it does not make use of PDO. If the code has that many spots without proper sql procedures then we are looking forward to a long battle.

    Hehe, I know, right? I heard they'll finally introduce it in the next version.

  • AnthonySmithAnthonySmith Member, Patron Provider

    @Spirit Chris has confirmed that the entry point was not centralbackup.php while I understand why you can claim that may not be the case the fact is both solusvm and cvps are staying very quiet and frankly it is pissing me off.

    Even a sorry guys we don't know how to find out what the entry point was and cannot confirm solusvm was the entry point or otherwise would be fine, this blame and run game is just annoying.

  • dnwkdnwk Member

    Is it possible to run a reverse proxy in front of Solus and do some escape for them?

  • SpiritSpirit Member

    @AnthonySmith what do you expect from them? First party try to save what's possible to save and other party try to find out what happened. Crappy situation. Imho. there's simply not much to say yet.

  • fapvpsfapvps Member

    Very depressing past few days with with all these security breaches... I think SolusVM should be open-sourced and the company should charge extra for some of the higher functionality and support.

  • @fapvps said:
    I think SolusVM should be open-sourced and the company should charge extra for some of the higher functionality and support.

    Not going to happen. However I'd divine that by next year, we'll probably see some good open source panels coming up.

  • Is there a list anywhere that shows the nodes online/offline for chicagovps? Can anyone that had an outage earlier confirm that their servers have come back online? I'm interested to see what / if any progress is being made.

  • He probably you know, working to get his customers back up and running.

  • fapvpsfapvps Member

    Good open source panel: Cloudmin GPL and the paid solution Cloudmin Pro!

  • One of my VPS's is up while the other is still down sure hope it is available soon with or without my data so at least I can get started with reconstruction.

  • Nick_ANick_A Member, Top Host, Host Rep

    To confirm, SolusVM is aware that many hosts are sitting around waiting right?

  • solus :"We have been working hard to audit all of the SolusVM code to find any further potential security issues that may pose a threat.At this moment we have been unable to locate any problems "

    Based on the poor coding in the code snippets I've seen posted the past couple of days I'd say two things: #1 the entire code base is a security disaster waiting to happen again and again. #2 the Solus developers wouldn't recognize a security problem if it bit them in the ass.

    One of my LA VPS's has been responding to pings since the attack occurred but when I was logged into it shortly after the attack every command was returning a seg fault (and some commands like reboot were returning file not found) so I'm assuming that many of the VPS's on the node were trashed/wiped even though it's still responding to pings.

  • lennierb5lennierb5 Member
    edited June 2013

    According to this, none of the servers that were down more than an hour have been brought back online yet. Unless I am missing something?

    EDIT: I take that back, I see a few that were offline for a few hours and are now showing online.

  • InfinityInfinity Member, Host Rep
    edited June 2013

    @Nick_A said:
    To confirm, SolusVM is aware that many hosts are sitting around waiting right?

    Yeap, they're well aware.. not that it will make them speed up at all. And no I don't mind if they take their time and actually get it right and correct everything thoroughly once and for all.

  • Just got an email about this:

    Around 3am Eastern Standat Time (EST) today, there was a security breach, due to a vulnerability in SolusVM that allowed a command line to be run to dump the ChicagoVPS SolusVM client database and attempt to delete all data from our nodes. Our staff is working tirelessly to get everything back online, along working with SolusVM to address the root issue and no furthur impact is expected.

    Now what does this mean for the customer? All passwords should be changed, this includes passwords for SolusVM control panel and your VPS. This data leak does not include billing information or credit card information. Thus far we are having great success in getting nodes back online with no data loss, however, there are a few that were not recoverable and will be restored using our offsite backups.

    Once the situation is 100% complete and back to normal we will send another email out. We understand the sevarity and importance to get everything back online quickly. With that in mind, please try to refrain from opening a ticket or replying to an old one as it only slows us down even more. We are doing our best, and hope to have this fully resolved within 24 hours.

    Thank you for your patience and understanding.

    Regards

    Your ChicagoVPS Team

  • Looks like everyone will be switching over to HyperVM and VePortal.

Sign In or Register to comment.