Weird stuff going on at online.net

oijpghjighoji

Hello fellow sysadmin,

Weird stuff happening at online.net.

One of my servers got locked because of "flood".

-Server is ubuntu server
-Only has 1 non-standard port open for openssh. Everything else is closed.
-ssh is protected with 20 characters password
-server gets patched and rebooted daily

Only some genius hacker with a remote openssh exploit could hack this server and cause a DDoS.

So I really doubt it got hacked.

Anyway I don't have time to waste so I just wiped the server and cancelled it.

In 7 years I never got 1 server hacked. Maybe some genius just popped my cherry or Online.net is just being shady.

Aidan

oijpghjighoji said:

One of my servers got locked because of "flood".

They have a lot of false-positive detections.

joepie91

Your actual problem aside...

oijpghjighoji said: -ssh is protected with 20 characters password

Don't do that. Use keypair authentication and disable password authentication. There's zero reason to keep using passwords with SSH today.

oijpghjighoji

I know keypair is more secure but 20 chars password should be secure enough.....Maybe. Bruteforcing 20 chars password over internet would probably take a long time.

lion

@oijpghjighoji said:
I know keypair is more secure but 20 chars password should be secure enough.....Maybe. Bruteforcing 20 chars password over internet would probably take a long time.

You deserve to get owned

oijpghjighoji

You deserve to get owned

Thank you for your constructive comment.

needavps

@lion said:

@oijpghjighoji said:
I know keypair is more secure but 20 chars password should be secure enough.....Maybe. Bruteforcing 20 chars password over internet would probably take a long time.

You deserve to get owned

not true. I still have a centos 5 with ssh in non standard port like over 10000 with only a 10 alphanumeric password and still going strong for over 5 years.

oijpghjighoji

Same. I have plenty of servers with 20 chars random generated passwords. And never got a problem. I would be amazed by anybody that could bruteforce this.

Lee

oijpghjighoji said: should be secure enough

Usually is good enough though. Oh wait, you might have been just been compromised by only settling for good enough.

oijpghjighoji said: Online.net is just being shady.

Yeah., always look for someone else to blame.

Just because you have been "fine" for x years using passwords does not confirm it is effective security and nothing could ever happen, i.e brute force.

Not trying to be harsh here but if your position is "I have never been hacked" and "Online.net is being shady" when you have no idea what the issue is and your resolution is to wipe the server and cancel before even trying to establish why it happened then I really don't know what you expect anyone here to do or say.

You appear to be posting this just to get back at online.net but without any reason.

And all that aside, there is just no reason not to be using keys these days to eliminate any potential issues you would have with passwords.

MikeA

Any chance you're running a type of virtualization on it?

mfs

My impression is that you now regret a little bit the fact you didn't investigate this incident, and now you have some doubts about what actually happened.

needavps said: centos 5

pls upgrade

joepie91

@oijpghjighoji said:
I know keypair is more secure but 20 chars password should be secure enough.....Maybe. Bruteforcing 20 chars password over internet would probably take a long time.

Again: there is no reason to use passwords today. The amount of characters doesn't really matter here; it's that using password authentication is inherently less secure because it uses a different authentication model (namely a shared key between the client and the server). Even if you used as many characters for your password as the length of a typical key, it would still be less secure.

There's no reason to settle for what you believe is 'good enough', especially if security is not your specialization. Use the best option that you can afford (which, in this case, is keypair authentication) and then you don't run the risk of having overestimated your security.

hawkjohn7

your web script maybe injected by hacker.

WSS

apachenode.knockd will happen some time within millennials.

pike

Is really simple to setup key authentication. I have done it myself this weekend and it took me 5 minutes. +1 for key auth

jar

This seems to make the assumption that the only way to make a server perform an action is to gain root access. Frankly, I wouldn't even guess that to be in the top 5 list of ways to leverage someone else's server to perform outgoing actions. For that reason I'd suggest, based purely on probability, that you consider other options first.

Vulnerable php script used to upload another script (let's be honest it's this, easy money if you bet on it), reflection attack from poorly configured service or badly coded script, mail relay (more specific play on my second idea), just to name a few of the possibilities.

tomle

Had a similar issue with flood, turned out to be the IPv6 configuration. Can't remember exactly what is was but I put a limit with iptables. Think their support pointed me towards the issue by providing the log of the flood.

Shot2

https://www.lowendtalk.com/discussion/40695/ipv6-dhclient-auto-configuration-script-online-net

Keywords: dhcp, flood

vovler

@joepie91 said:
Your actual problem aside...

oijpghjighoji said: -ssh is protected with 20 characters password

Don't do that. Use keypair authentication and disable password authentication. There's zero reason to keep using passwords with SSH today.

lAzINnEsS

sureiam

@oijpghjighoji said:
I know keypair is more secure but 20 chars password should be secure enough.....Maybe. Bruteforcing 20 chars password over internet would probably take a long time.

Did you setup fail2ban?

oijpghjighoji

@MikeA said:
Any chance you're running a type of virtualization on it?

No virtualization. It was just a server used for backups via ssh.

oijpghjighoji

@hawkjohn7 said:
your web script maybe injected by hacker.

There was no webscript or any webserver installed on the server.

oijpghjighoji

@jarland said:
This seems to make the assumption that the only way to make a server perform an action is to gain root access. Frankly, I wouldn't even guess that to be in the top 5 list of ways to leverage someone else's server to perform outgoing actions. For that reason I'd suggest, based purely on probability, that you consider other options first.

Vulnerable php script used to upload another script (let's be honest it's this, easy money if you bet on it), reflection attack from poorly configured service or badly coded script, mail relay (more specific play on my second idea), just to name a few of the possibilities.

PHP not even installed on the server
No web server.

WSS

dns amplification?

Xei

@WSS said:
dns amplification?

Is that possible out of the box? I don't think I have SNS server running on any nix instance so must be manually installed?

pinc

@WSS said:
dns amplification?

Ubuntu doesn't relay by default and OP says that all ports were closed except for non-standard SSH. However I think something is missing in this story.

xyz

joepie91 said: There's zero reason to keep using passwords with SSH today.

One reason: it's a pain to put the private key onto every system you wish to log in from, not to mention that having the key on disk can be a weakness itself. Okay, you can password protect the key, but then that's even more effort.

Unless you get a hard over how theoretically secure your system is, using a strong password is perfectly fine. I don't need military grade protection which can protect me against a state-level adversary, I just need enough protection against my adversaries. If someone really did want to get in, I'm sure they could just break into my home and steal all my stuff, or coerce me to give it up via other means, anyway - it'd likely be much simpler than trying to do some complicated SSH attack to scrape the password.

angstrom

@xyz said:

joepie91 said: There's zero reason to keep using passwords with SSH today.

One reason: it's a pain to put the private key onto every system you wish to log in from, not to mention that having the key on disk can be a weakness itself. Okay, you can password protect the key, but then that's even more effort.

Unless you get a hard over how theoretically secure your system is, using a strong password is perfectly fine. I don't need military grade protection which can protect me against a state-level adversary, I just need enough protection against my adversaries. If someone really did want to get in, I'm sure they could just break into my home and steal all my stuff, or coerce me to give it up via other means, anyway - it'd likely be much simpler than trying to do some complicated SSH attack to scrape the password.

Not to mention that based on the info that we have about the OP's server, it's far-far from clear that his password was cracked. Without more info, it's not at all clear what happened exactly.

Shot2

@angstrom said:

@xyz said:

joepie91 said: There's zero reason to keep using passwords with SSH today.

One reason: it's a pain to put the private key onto every system you wish to log in from, not to mention that having the key on disk can be a weakness itself. Okay, you can password protect the key, but then that's even more effort.

Unless you get a hard over how theoretically secure your system is, using a strong password is perfectly fine. I don't need military grade protection which can protect me against a state-level adversary, I just need enough protection against my adversaries. If someone really did want to get in, I'm sure they could just break into my home and steal all my stuff, or coerce me to give it up via other means, anyway - it'd likely be much simpler than trying to do some complicated SSH attack to scrape the password.

Not to mention that based on the info that we have about the OP's server, it's far-far from clear that his password was cracked. Without more info, it's not at all clear what happened exactly.

This.

Since "malicious PHP script" and "cracked 20-char openssh password" have been suggested already, my two-cent speculation: misconfigured dhcp client and/or misconfigured firewall resulting in OP's server going atrociously talkative.

angstrom

@Shot2 said:

@angstrom said:

@xyz said:

joepie91 said: There's zero reason to keep using passwords with SSH today.

One reason: it's a pain to put the private key onto every system you wish to log in from, not to mention that having the key on disk can be a weakness itself. Okay, you can password protect the key, but then that's even more effort.

Unless you get a hard over how theoretically secure your system is, using a strong password is perfectly fine. I don't need military grade protection which can protect me against a state-level adversary, I just need enough protection against my adversaries. If someone really did want to get in, I'm sure they could just break into my home and steal all my stuff, or coerce me to give it up via other means, anyway - it'd likely be much simpler than trying to do some complicated SSH attack to scrape the password.

Not to mention that based on the info that we have about the OP's server, it's far-far from clear that his password was cracked. Without more info, it's not at all clear what happened exactly.

This.

Since "malicious PHP script" and "cracked 20-char openssh password" have been suggested already, my two-cent speculation: misconfigured dhcp client and/or misconfigured firewall resulting in OP's server going atrociously talkative.

Perhaps. But it's hard to tell anything for sure based on the very limited info that the OP has presented.

pinc

@Shot2 said:
misconfigured firewall resulting in OP's server going atrociously talkative.

+1. Would be interesting to know how exactly did OP 'close all ports'. So this is hardly about breaking 20 chars SSH pass.