Hello people.

My system runs a lot of perl scripts for an application I depend on and it hogs a lot of RAM. Now sshd (protocol 2 only) is running on this machine. Some kiddies are trying lots of username/password combos to ssh in. Well, I get a lot of authentication errors in my /var/log/messages and secure files. That's still tolerable, because after failed attempts, the same IPs usually never try to log in again.

Yesterday after about 149 failed ssh logins from the same IP, the machine just froze.

sshd died, and then this is what /var/log/messages showed

Mar 21 16:29:49 otto sshd(pam_unix)[13849]: authentication failure; logname= uid=0 euid=0 tty=NODEVssh ruser= rhost=XXX.XXX.XXX.XXX user=root
Mar 21 16:29:55 otto sshd(pam_unix)[13852]: authentication failure; logname= uid=0 euid=0 tty=NODEVssh ruser= rhost=XXX.XXX.XXX.XXX user=root
Mar 21 16:30:02 otto kernel: swap_free: Bad swap file entry 00000010
Mar 21 16:30:02 otto kernel: swap_dup: Bad swap file entry 00000010
Mar 21 16:30:02 otto kernel: VM: killing process sshd
Mar 21 16:30:02 otto kernel: swap_free: Bad swap file entry 00000010
Mar 21 16:30:06 otto kernel: swap_free: Bad swap file entry 00000020
Mar 21 16:29:49 otto sshd(pam_unix)[14099]: authentication failure; logname= uid=0 euid=0
Mar 21 16:29:49 otto sshd(pam_unix)[14099]: authentication failure; logname= uid=0 euid=0

There was a similar VM:killing my_needed_app.pl

and then, here's what took the cake:

Mar 21 17:41:27 eto kernel: memory.c:304: bad pmd 146cd087.

and the machine just froze, though it did answer ping requests. I had to reboot it then.

How can i stop this from recurring? portsentry/psad only works for ports I'm not offering services on right? Is there a script which can log these multiple authentication failures and then then put them into a DROP ruleset in iptables??

Thanks

While it may not work for your particular situation, I recently changed my sshd server to listen on a different port than the default, port # 22, and I haven't had one attempted login attempt since then. I picked a non-standard port and my /var/log/auth.log has been clean since.

$ man sshd_config

Should provide you with the necessary changes you need to make to your sshd configuration.

Now when I ssh in I use the command:

$ ssh -p <port #> <hostname>

Furthermore I changed my ssh login to only accept public key logins, that way any attempts to try and login with a password is automatically rejected.

$ man sshd_config

has more info on this.

If this is a solution that would work for you I could elaborate on how I made these changes.

Hope this helps, good luck.

Unless you need access to SSH from everywhere and anywhere, then probably the simpliest solution to stop the attacks altogether is limit access to it in your firewall, ie only allow access from a certain IP/s. If you need access from everywhere then you could flip that and write a script to automatically ban IPs as they attempt entry. Beyond that I don't think there is much you can do to actually stop the attempts altogether otherthan switch to a different port, which is basically security through obsecurity and probably soon the bots doing this will figure that out and start scanning all ports to find it. Anyway, this subject has been discussed at length at http://www.linuxquestions.org/questi...hreadid=215431

Yeah, but has this ever lead to a Denial of Service? I realize that disallowing password authentication and using a different port will help. As for a range of IPs - well, you know the problems with multiple ISPs, DHCP, etc.

So what about the DoS - was that due to memory shortages due to repeated authentication failures?

Honestly, I have no idea. Perhaps one of the security gurus in the security forum could better answer that question, as they likely deal with DOS attacks routinely. At any rate, you'd think a hardened piece of software like ssh/sshd would have an internal means of combating attacks where the shear # of attempts from the same host caused problems. But from what I understand that isn't the case here. There are a couple of directives available that you may want to play with, MaxAuthTries & MaxStartups. You can set them in your sshd_config file, see man sshd_config for details. But they are somewhat restrictive by default, so that would seem to suggest the bots doing the attacks are immune to them. The newer versions of the bot seem to be alot peskier than the original one, trying 100s of combos vs just a few. You might see if you can find source for one of them and set up a testbed to play with to see if you can get better control of it and/or recreate the crash. Anyway, good luck and if you discover a good way of dealing with it, be sure and let us know.

Moved: This thread is more suitable in Linux Security and has been moved accordingly to help your thread/question get the exposure it deserves.

Yeah, I don't believe it's an actual intentional DoS attack, but rather some kind of handware error or resource depletion (esp memory) that results fron having a large number of concurrent login sessions. As DaHammer suggested, decreasing the number of simultaneous login sessions allowed by lowering MaxStartups to 1 or 2 should help. You can also try increasing RAM, using iptables to limit the rate of packets, or just move ssh to an alternate port.

If that still doesn't help, try capturing some packets with tcpdump. I'd be hesitant to use tcp wrappers to add iptables rules dynamically like that, as you could lock yourself out rather easily.

How much memory do you have? What are these perl scripts doing that they require so much of it?

Well, I have 512 MB of RAM. Which you't think is a lot. BUT: these perl scripts are part of a software which is used to gather data from all over the world that these anthropologists at our univ are using.

So at some instants, there may be a lot of people submitting data. Each such process takes up some_ram, and if there are n people connected, thats n*some_ram amount of ram consumed. So there you go.

But even if I were to use RSA based public key authentication, would that stop a DoS if there were enough cracker attacks pointed to the system?

can one make password auth only after RSA-key based auth?

also google for pam-tally that makes the #5 wrong attempts do sshd downtime (just an example) or auto add to deny hosts