All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
DelimiterVPS' mystery
I recently got a machine with DelimiterVPS, which I set up with a very bare configuration (just SSH and Apache). So far I barely used it but nonetheless had a very interesting, or maybe rather strange , experience in the past few days.
A few days ago suddenly all HTTP monitoring requests failed, even though the machine was still online. Trying to access it via SSH also failed and this was when I realised that it switched from its custom SSH port back to the default port.
As there was no way to access the machine I eventually had to reprovision it to get it back working.
Contacting support was unfortunately rather fruitless as they only speculated that the machine was most likely compromised (without evidence though) and couldnt provide any further assistance either.
Now I am definitely not completely ruling out the option that it was compromised but considering the overall setup it would imply the attacker (in chronologically order) successfully
- discovered and determined the port in question as SSH port
- managed to guess the only (uncommon) username allowed to login
- managed to bruteforce this user to get a successful login
- managed to bruteforce from this valid login the root password
and subsequently, with the newly obtained root privileges, stopped Apache, changed the SSH port back to its default and restarted the SSH daemon. All of which without abuse reports so far however.
As I already said, I am not saying this scenario is impossible but I wonder how likely it is, respectively if a configuration and/or storage failure could be the reason behind it (I wouldnt have evidence for this either though).
And this is where my actual question comes in. Did anyone else ever had a similar case, be it with or without DelimiterVPS?
Thanks
Comments
It is not surprising they could only speculate and provide no evidence, if you cant get in they would not be able to either you could maybe ask them about traffic during the period that may be a give away.
It sounds more like an automated reinstall was done on the server to be honest which would fit all symptoms.
The other possibility is that what ever you use to store such info was compromised instead the old putting a home PC in DMZ with no AV running and a WAMP install still running from 3 years ago and a txt document in my documents called logininfo.txt or mypasswords.txt
Sounds stupid but it happens
That is exactly what worries me.
I can assure you, there is no such thing
I see your point and yes, mistakes happen. But nonetheless, if it was a reinstall, then this should really not "simply" happen.
Yep, my money would be on human error automated re install, and I hate to see it but cover-up as no evidence will exist.
I have had the same thing happen to my idle CVPS when I first initially got it and was setting up the machine. Albeit I was more of a noob back then but I followed the guides to a T and the passwords were definitely not insecure. I had the same issue where my custom ssh port was changed back to default despite changing it and the machine was no longer accessible, monitoring showed box offline cvps panel showed box online, but nothing worked (this was a vps). CVPS claimed it was compromised, however I have not run into the same issue on any of my other boxes nor have I run into the issue again.
Not sure if they use SolusVM.. but if the VM was rebuilt should it not show up in the log?
You couldn't even get in from the control panel console?
Maybe the control panel login was compromised instead of the VPS itself.
Oh wait this is a VPS? not sure why I thought it was a dedi.
If it is a VPS what was stopping you from doing a root password reset in solusvm and using the serial console to investigate?
You are correct, it is a dedicated machine.
It was the word machine that made me assume that.
I assume no IPMI or KVMoIP is/ was available?
And it was right on the spot
Not to my knowledge unfortunately.
Guessing it's one of the Atoms with the network based storage. As the HP blades all have iLO.
Shame you could have used that to boot in to single user mode.
In future you could always ask them to boot it in to a rescue environment, mount your file system and investigate.
The other more scary possibility is that they have a compromise somewhere on their system.
Precisely.
In this particular case support was not too helpful unfortunately.
Now, that would be bad.
You can always boot your system into the rescue system using the provisioning system, then you can SSH into the ramdisk based system and mount the disk to examine the log files.
Thanks Mark. Could you please elaborate on this? Unfortunately I cannot find any such option and the support did not mention it either I am afraid.
@Neroux - Ask Delimiter support about this, we make a rescue system available to them so they should make it available to you.
I will if the need arises again. Even though I naturally hope this particular incident does not repeat itself (or for that matter any which requires a rescue system )
So I guess nobody else ever had a similar issue.
an instance for galileo mystery.
I dont know what you mean but its most probably not related to this thread. Anything useful?
I have 2 with them without problema
The interesting thing with the atoms is that even if the control panel login was compromised you can't actually reinstall the OS unless you reboot the machine via SSH.
How come? I couldnt imagine how the operating system could continue working if you pulled the rug from under it storage-wise.
Also, you can always power down the machine via the control panel.
@neroux i believe their atoms don't have remote power control. you either have to reboot it, or someone must go and powercycle it manually. And reinstalling is done by network booting via PXE, loading the installer, etc. No pulling the rug of the storage.
They do have remote power control. Reinstalling is done via the control panel as well, as the storage in question is a SAN.
@neroux have you tried the remote power control? Try powering off your server and see if it really powers off.
Are you saying it is not functional?