Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


DelimiterVPS Admin, can you send an email notice before suspend server?
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

DelimiterVPS Admin, can you send an email notice before suspend server?

IrisIris Member

Recently, some of websites on the delimitervps server were compromised by php code injection.
I've been working on them. Two days ago, when I reviewing a website by browser, find it was also get code injection, then I connect to the server to clean codes up. But several hours later, I find my sever down. After check client area, the server was suspended with no notice.
The compromised codes are already clean. Server is down for more than 24 hours. The support ticket get no reply about when this can be fixed.
Why can't they send an notice email about that before just shutdown the server? And all this downtime can be avoid.

Comments

  • Delimiter never suspends a server without good cause. Please provide the ticket number.

  • IrisIris Member

    Ticket #308308

    I already switched all domain name to other server at present. Thus, there will be no more harm to your server.

  • MarkTurnerMarkTurner Member
    edited May 2015

    Not quite the whole story there:

    01/13/2015 - Outgoing brute force attacks. Single IP Nulled, assured issue resolved
    05/07/2015 - Outgoing brute force attacks. Single IP Nulled, assured issue resolved
    05/08/2015 - Outgoing brute force attacks. Single IP Nulled, assured issue resolved
    05/12/2015 - Outgoing brute force attacks. Server suspended, ILO access

    Seems like you make promises to them to get the server reactivated, then attack some more people and plunge the reputation of our IP space further.

    Clearly whatever you have done was insufficient. The server should have been suspended on the 8th after your assurances were proven wrong.

    You have been provided in the ticket with a handful of the hundreds of complaints. This server has been bruteforcing hundreds of Wordpress servers. Lets be clear this was not one little 1 minute instance, this was going on for days and thousands of connections a minute.

    There is a responsibility when you run a server which is to keep it secure not only for your sake but to stop the idiots using it as a launchpad to attack other peoples servers.

    Moving your domain names to another server won't really help the situation, the server is being used to attack the other servers. You need to just reinstall the server and this time set it up properly.

    Anyway you will need to follow it up with Delimiter during business hours. I expect you'll be told to download your data and reinstall the system before they release the IPs. I wouldn't like to see this server back online in its current state. You don't seem to possess the administration skills to secure it and clearly somehow or other the server has been compromised.

    Thanked by 3deadbeef ATHK comXyz
  • IrisIris Member
    edited May 2015

    As per your details. But don't mix things up.

    01/13/2015 - Outgoing brute force attacks. Single IP Nulled, assured issue resolved

    This is resolved. It's another old story. And the attacks issue happens after about 4 month. Which means I do handle that issue well and secure the server. The hacker control the attack by visiting backdoor of php file which he compromised by code injection. That what I find, and solve the issue.

    And 4 months later, there was another guy compromised the server. It happens all around the network as long as there are hackers. I can only fix it after it happen, and then upgrade the codes.
    For the 05/07/2015 and 05/08/2015, that's 4 months later, don't mix things up. Only this time I miss several php pages, which lead to the attacking thing happen again.

    Here's the real story:
    On 05/11/2015, I find I misse one compromised php file. Then go cleaned up. And get server secure before you shut down server at 05/12/2015, as your admin only work at business hours to get those feedbacks of attacks.
    And can you let me know who notice you with those attack information? And how the plunging the reputation of IP happens?

    @MarkTurner said:
    Not quite the whole story there:

    01/13/2015 - Outgoing brute force attacks. Single IP Nulled, assured issue resolved
    05/07/2015 - Outgoing brute force attacks. Single IP Nulled, assured issue resolved
    05/08/2015 - Outgoing brute force attacks. Single IP Nulled, assured issue resolved
    05/12/2015 - Outgoing brute force attacks. Server suspended, ILO access

    Seems like you make promises to them to get the server reactivated, then attack some more people and plunge the reputation of our IP space further.

    Clearly whatever you have done was insufficient. The server should have been suspended on the 8th after your assurances were proven wrong.

    You have been provided in the ticket with a handful of the hundreds of complaints. This server has been bruteforcing hundreds of Wordpress servers. Lets be clear this was not one little 1 minute instance, this was going on for days and thousands of connections a minute.

    There is a responsibility when you run a server which is to keep it secure not only for your sake but to stop the idiots using it as a launchpad to attack other peoples servers.

    Moving your domain names to another server won't really help the situation, the server is being used to attack the other servers. You need to just reinstall the server and this time set it up properly.

    Anyway you will need to follow it up with Delimiter during business hours. I expect you'll be told to download your data and reinstall the system before they release the IPs. I wouldn't like to see this server back online in its current state. You don't seem to possess the administration skills to secure it and clearly somehow or other the server has been compromised.

  • @Iris said:
    And can you let me know who notice you with those attack information? And how the plunging the reputation of IP happens?

    People who get attacked contact the hosting company and complain (as they should). Other entities keep track records of bad servers and compile reputation lists.

    Thanked by 1comXyz
  • Iris said: And 4 months later, there was another guy compromised the server. It happens all around the network as long as there are hackers. I can only fix it after it happen, and then upgrade the codes. For the 05/07/2015 and 05/08/2015, that's 4 months later, don't mix things up. Only this time I miss several php pages, which lead to the attacking thing happen again.

    Having you taken basic precautions like securing your webserver, limiting PHP's access to certain functions, jailing PHP and so on.

    More than likely this is just a stock Apache install with stock PHP and no hardening.

    As I said, you need to discuss this matter with Delimiter. They've been more than accommodating, lenient and helpful getting your server reactivated in the past. But that many 'compromises' in less than a week just looks like you didn't do anything to combat the problem.

    @deadbeef in this instance most of the complaints were from other hosting companies as well as one blacklist which the whole /24 is alleged to now be listed upon.

    This type of thing is just disruptive to customers that take the time to secure their systems.

    Thanked by 1deadbeef
  • IrisIris Member
    edited May 2015

    @MarkTurner said:
    Having you taken basic precautions like securing your webserver, limiting PHP's access to certain functions, jailing PHP and so on.

    More than likely this is just a stock Apache install with stock PHP and no hardening.

    Debian 7 is installed on this server. Whole debian system was upgraded using apt-get update and apt-get upgrade include php and apche to newest after 05/07/2015 IP nulled .

    The issue is happened because I missed to clean up several php files which get code injection as mentioned before. I should check more carefully before request activation on 05/07/2015.
    And now all are cleaned set proper php file permission.
    I try to discuss this issue with delimiter. It's 24hrs after I submit the ticket, which should cover their business hours for at least 8 hours, but delimiter admin neither bring up my server, nor a reply about it.

  • Iris said: Debian 7 is installed on this server. Whole debian system was upgraded using apt-get update and apt-get upgrade include php and apche to newest after 05/07/2015 IP nulled .

    And what did you do to harden Apache/PHP? Upgrading to the latest versions of the software is great, but you also need to lock down the config. Work out which functions are unnecessary to your needs, do you have something like Mod_security installed? Do you have something to harden PHP eg Mod_ruid2 or SUPHP? You need to take the time to actually secure the software, otherwise give it a few days and you'll get hit again.

  • IrisIris Member
    edited May 2015

    @MarkTurner said:
    And what did you do to harden Apache/PHP? Upgrading to the latest versions of the software is great, but you also need to lock down the config. Work out which functions are unnecessary to your needs, do you have something like Mod_security installed? Do you have something to harden PHP eg Mod_ruid2 or SUPHP? You need to take the time to actually secure the software, otherwise give it a few days and you'll get hit again.

    Actually, the server is using nignx with apache backend to handle php.
    Mod_security and SuPHP will be install before put any live websites on after server online.

    It's more than 24hrs after I submit the ticket, which should cover their business hours for at least 8 hours, but delimiter admin neither bring up my server, nor a reply about it.

  • BruceBruce Member

    @Iris I'm interested to understand what happened here. did your server get compromised by someone, or was this customer doing bad things? I presume it wasn't you doing the attacks

    if this was from an external attack, what php were you using? was it your own code, or something known to have security issues (like out of date wordpress)

    I'm just interested in what happened, rather than any criticism. I take security seriously, but can always learn more, especially from others experiences.

  • NomadNomad Member

    @Iris, you are avoiding @MarkTurner's questions.

    It's clear that you did not do anything to harden the security of php and apache.

    You claim you WILL install Mod_security and SuPHP. But if you have no live websites, how come you get compromised?

  • jvnadrjvnadr Member
    edited May 2015

    So, did you have in your compromised servers live sites, or not?

    Iris said: Recently, some of websites on the delimitervps server were compromised by php code injection. I've been working on them. Two days ago, when I reviewing a website by browser, find it was also get code injection, then I connect to the server to clean codes up.

    According with you, you had several sites installed in your server, and a lot of them were compromized. Also, according to you, you didn't stop the server and the web sites accessing the internet and you tried to "clean" them while all was online...

    Iris said: The issue is happened because I missed to clean up several php files

    Hardening a server is not enabling mod security and do an update. Hardening a server is not cleaning infected php. Before you requested to become online again, you should make sure that you actually hardeden the node, doing all of that:

    • Reinstall the server from scratch
    • Disable password ssh access
    • Using keys to access the server
    • Change the port to something other than 22
    • Install clamav
    • Install a malware detector like Linux Malware Detect (LMD)
    • Disable root access
    • install CSF/LFD and disable access to all not used ports
    • Install fail2ban
    • Harden network with sysctl
    • Check all php files for suspected code. ALL of the php files!

    If you do all that, then, you plug the server to the net again and monitor it for several days continously, to see you first if there is any weird activity running there.

    Thanked by 2Aene Iris
  • IrisIris Member

    @Nomad said:
    Iris, you are avoiding MarkTurner's questions.

    It's clear that you did not do anything to harden the security of php and apache.

    You claim you WILL install Mod_security and SuPHP. But if you have no live websites, how come you get compromised?

    Did you read all my posts? It's happened after compromised. The server is offline now. And then, domain names are switch to somewhere else.

  • AeneAene Member

    Good list @jvnadr although I would add:

    • keep up to date on security patches

    As far as I can tell Iris only upgraded after the hack.

  • IrisIris Member
    edited May 2015

    @Bruce said:
    Iris I'm interested to understand what happened here. did your server get compromised by someone, or was this customer doing bad things? I presume it wasn't you doing the attacks

    if this was from an external attack, what php were you using? was it your own code, or something known to have security issues (like out of date wordpress)

    I'm just interested in what happened, rather than any criticism. I take security seriously, but can always learn more, especially from others experiences.

    It's an plugin no one updating source codes now. So I am amending the codes myself now.

    @Bruce said:
    Iris I'm interested to understand what happened here. did your server get compromised by someone, or was this customer doing bad things? I presume it wasn't you doing the attacks

    I paid the server yearly, why would I do that? Or will any one buy a whole year dedicated server only to attack other dudes' wordpress blog? Will you do that?

  • jvnadrjvnadr Member

    Aene said: keep up to date on security patches

    This is first! I didnt add it because OP said that he did an apt-get upgrade to his server, so, he did patched any out dated OS element.
    Any one should setup automatic security updates to his OS. And, of course, maintain his server frequently.

  • tehdantehdan Member

    @iris it must be obvious by now that you're fighting a losing battle. Delimiter are earning points for treating you fairly while protecting their more competent customers and you're advertising yourself as a customer who can't keep his servers secure.

    Perhaps you should let it go, and concentrate on learning a bit more about protecting your servers at your new provider before you get hacked + booted out of there?

    Thanked by 1pechspilz
Sign In or Register to comment.