Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


DelimiterVPS' mystery
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

DelimiterVPS' mystery

I recently got a machine with DelimiterVPS, which I set up with a very bare configuration (just SSH and Apache). So far I barely used it but nonetheless had a very interesting, or maybe rather strange :), experience in the past few days.

A few days ago suddenly all HTTP monitoring requests failed, even though the machine was still online. Trying to access it via SSH also failed and this was when I realised that it switched from its custom SSH port back to the default port.

As there was no way to access the machine I eventually had to reprovision it to get it back working.

Contacting support was unfortunately rather fruitless as they only speculated that the machine was most likely compromised (without evidence though) and couldnt provide any further assistance either.

Now I am definitely not completely ruling out the option that it was compromised but considering the overall setup it would imply the attacker (in chronologically order) successfully

  • discovered and determined the port in question as SSH port
  • managed to guess the only (uncommon) username allowed to login
  • managed to bruteforce this user to get a successful login
  • managed to bruteforce from this valid login the root password

and subsequently, with the newly obtained root privileges, stopped Apache, changed the SSH port back to its default and restarted the SSH daemon. All of which without abuse reports so far however.

As I already said, I am not saying this scenario is impossible but I wonder how likely it is, respectively if a configuration and/or storage failure could be the reason behind it (I wouldnt have evidence for this either though).

And this is where my actual question comes in. Did anyone else ever had a similar case, be it with or without DelimiterVPS?

Thanks

«1

Comments

  • AnthonySmithAnthonySmith Member, Patron Provider
    edited June 2014

    It is not surprising they could only speculate and provide no evidence, if you cant get in they would not be able to either :) you could maybe ask them about traffic during the period that may be a give away.

    It sounds more like an automated reinstall was done on the server to be honest which would fit all symptoms.

    The other possibility is that what ever you use to store such info was compromised instead the old putting a home PC in DMZ with no AV running and a WAMP install still running from 3 years ago and a txt document in my documents called logininfo.txt or mypasswords.txt

    Sounds stupid but it happens :)

  • nerouxneroux Member
    edited June 2014

    AnthonySmith said: It sounds more like an automated reinstall was done on the server to be honest which would fit all symptoms.

    That is exactly what worries me.

    AnthonySmith said: The other possibility is that what ever you use to store such info was compromised

    I can assure you, there is no such thing ;)

    AnthonySmith said: Sounds stupid but it happens :)

    I see your point and yes, mistakes happen. But nonetheless, if it was a reinstall, then this should really not "simply" happen.

  • AnthonySmithAnthonySmith Member, Patron Provider

    Yep, my money would be on human error automated re install, and I hate to see it but cover-up as no evidence will exist.

  • akzakz Member
    edited June 2014

    I have had the same thing happen to my idle CVPS when I first initially got it and was setting up the machine. Albeit I was more of a noob back then but I followed the guides to a T and the passwords were definitely not insecure. I had the same issue where my custom ssh port was changed back to default despite changing it and the machine was no longer accessible, monitoring showed box offline cvps panel showed box online, but nothing worked (this was a vps). CVPS claimed it was compromised, however I have not run into the same issue on any of my other boxes nor have I run into the issue again.

  • earlearl Member

    Not sure if they use SolusVM.. but if the VM was rebuilt should it not show up in the log?

  • You couldn't even get in from the control panel console?

  • M66BM66B Veteran

    Maybe the control panel login was compromised instead of the VPS itself.

  • AnthonySmithAnthonySmith Member, Patron Provider

    Oh wait this is a VPS? not sure why I thought it was a dedi.

  • AnthonySmithAnthonySmith Member, Patron Provider

    If it is a VPS what was stopping you from doing a root password reset in solusvm and using the serial console to investigate?

  • nerouxneroux Member

    AnthonySmith said: Oh wait this is a VPS? not sure why I thought it was a dedi.

    You are correct, it is a dedicated machine.

  • AnthonySmithAnthonySmith Member, Patron Provider

    It was the word machine that made me assume that.

    I assume no IPMI or KVMoIP is/ was available?

  • nerouxneroux Member
    edited June 2014

    AnthonySmith said: It was the word machine that made me assume that.

    And it was right on the spot :)

    AnthonySmith said: I assume no IPMI or KVMoIP is/ was available?

    Not to my knowledge unfortunately.

  • Guessing it's one of the Atoms with the network based storage. As the HP blades all have iLO.

  • AnthonySmithAnthonySmith Member, Patron Provider

    Shame you could have used that to boot in to single user mode.

    In future you could always ask them to boot it in to a rescue environment, mount your file system and investigate.

    The other more scary possibility is that they have a compromise somewhere on their system.

  • nerouxneroux Member

    DigitalDuke said: Guessing it's one of the Atoms with the network based storage.

    Precisely.

    AnthonySmith said: In future you could always ask them to boot it in to a rescue environment, mount your file system and investigate.

    In this particular case support was not too helpful unfortunately.

    AnthonySmith said: The other more scary possibility is that they have a compromise somewhere on their system.

    Now, that would be bad.

  • You can always boot your system into the rescue system using the provisioning system, then you can SSH into the ramdisk based system and mount the disk to examine the log files.

  • nerouxneroux Member

    @MarkTurner said:
    You can always boot your system into the rescue system using the provisioning system, then you can SSH into the ramdisk based system and mount the disk to examine the log files.

    Thanks Mark. Could you please elaborate on this? Unfortunately I cannot find any such option and the support did not mention it either I am afraid.

  • @Neroux - Ask Delimiter support about this, we make a rescue system available to them so they should make it available to you.

  • nerouxneroux Member

    MarkTurner said: @Neroux - Ask Delimiter support about this

    I will if the need arises again. Even though I naturally hope this particular incident does not repeat itself (or for that matter any which requires a rescue system ;) )

  • nerouxneroux Member

    said: And this is where my actual question comes in. Did anyone else ever had a similar case, be it with or without DelimiterVPS?

    So I guess nobody else ever had a similar issue.

  • NeoonNeoon Community Contributor, Veteran

    an instance for galileo mystery.

  • nerouxneroux Member

    @Infinity580 said:
    an instance for galileo mystery.

    I dont know what you mean but its most probably not related to this thread. Anything useful?

  • netomxnetomx Moderator, Veteran

    I have 2 with them without problema

  • The interesting thing with the atoms is that even if the control panel login was compromised you can't actually reinstall the OS unless you reboot the machine via SSH.

  • nerouxneroux Member

    @elwebmaster said:
    The interesting thing with the atoms is that even if the control panel login was compromised you can't actually reinstall the OS unless you reboot the machine via SSH.

    How come? I couldnt imagine how the operating system could continue working if you pulled the rug from under it storage-wise.

    Also, you can always power down the machine via the control panel.

  • rds100rds100 Member

    @neroux i believe their atoms don't have remote power control. you either have to reboot it, or someone must go and powercycle it manually. And reinstalling is done by network booting via PXE, loading the installer, etc. No pulling the rug of the storage.

  • nerouxneroux Member

    rds100 said: i believe their atoms don't have remote power control. you either have to reboot it, or someone must go and powercycle it manually. And reinstalling is done by network booting via PXE, loading the installer, etc. No pulling the rug of the storage.

    They do have remote power control. Reinstalling is done via the control panel as well, as the storage in question is a SAN.

  • rds100rds100 Member

    @neroux have you tried the remote power control? Try powering off your server and see if it really powers off.

  • nerouxneroux Member

    rds100 said: have you tried the remote power control? Try powering off your server and see if it really powers off.

    Are you saying it is not functional?

Sign In or Register to comment.