I see a lot of ssh attempts in my auth.log. Here’s a little snippet from one of my servers:

Nov 18 21:55:49 mcremove sshd[15991]: reverse mapping checking getaddrinfo for [] failed - POSSIBLE BREAK-IN ATTEMPT!
Nov 18 21:55:49 mcremove sshd[15991]: Invalid user business from
Nov 18 21:55:49 mcremove sshd[15991]: input_userauth_request: invalid user business [preauth]
Nov 18 21:55:49 mcremove sshd[15991]: pam_unix(sshd:auth): check pass; user unknown
Nov 18 21:55:49 mcremove sshd[15991]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=
Nov 18 21:55:50 mcremove sshd[15989]: Failed password for invalid user kent from port 46908 ssh2
Nov 18 21:55:50 mcremove sshd[15989]: Received disconnect from 11: Bye Bye [preauth]
Nov 18 21:55:51 mcremove sshd[15991]: Failed password for invalid user business from port 41781 ssh2
Nov 18 21:55:51 mcremove sshd[15991]: Connection closed by [preauth]
Nov 18 21:59:22 mcremove sshd[15994]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=  user=root
Nov 18 21:59:25 mcremove sshd[15994]: Failed password for root from port 40025 ssh2
Nov 18 21:59:25 mcremove sshd[15994]: Received disconnect from 11: Bye Bye [preauth]

I haven’t totaled up how many I average, but I can assure you the attempts are constant.

Anyways, all these bots trying to get in via SSH made me curious. What passwords are they using? A little googling around led me to a pam-cifs module that logs passwords, and more specifically the libpam source code so you can compile it yourself!

This is a PAM module written by Florian Lohoff, and with a little bit of tweaking you can get it to log those password attempts.

First, I downloaded the tar.gz and extracted it. Everything is already in there all nice and ready for you to run make on it. However, you do need the PAM development files on your machine:

sudo apt-get install libpam0g-dev

I found this package from searching for debian packages: https://packages.debian.org/cgi-bin/search_contents.pl?word=pam_appl.h&searchmode=searchfiles&case=insensitive&version=stable&arch=i386

I believe if you’re on RHEL/using yum, it would be: yum install pam-devel


Anyways, this module is great and all but it needs some modifications to work for logging SSH. The exact modifications were posted on adeptus-mechanicus.com in this great post.

Here’s all the required files:

The modified pam_storepw.c

After you run make, just add the new pam_storepw.so as /lib/security/pam_storepw.so

Final step, modify pam to load the new module! In the adeptus-mechanicus post, they modified /etc/pam.d/sshd. On my server though, I have an @include common-auth in my /etc/pam.d/sshd file, so I just modified my /etc/pam.d/common-auth file to look a little like this:

># here are the per-package modules (the "Primary" block)
auth    [success=2 default=ignore]      pam_unix.so nullok_secure
auth    optional        pam_storepw.so

Now all the failed password attempts are logged in /var/log/passwords

IMPORTANT!!! Change success=1 to success=2, let me explain why:

auth    [success=1 default=ignore]      pam_unix.so nullok_secure
auth    requisite                       pam_deny.so
auth    required                        pam_permit.so
auth    optional                        pam_cap.so

Without comments, this is what the common-auth file looks like. It’s important to explain the auth [success=1 default=ignore] pam_unix.so nullok_secure line.
This line is calling the “pam_unix” module, which checks the /etc/passwd and /etc/shadow files. I don’t have deep understanding of this, so seek information from the man pages or elsewhere online, but thats the basics of what it does.

The argument “nullok_secure” means what it says “no password is ok”, but ONLY from ttys that are listed in the /etc/securetty file. I believe that this only applies to root, but I’m not 100% sure on that. PS this doesn’t mean you can just leave the password blank for root and be able to login. From the manpage:

The default action of this module is to not permit the user access
to a service if their official password is blank. The nullok_secure
argument overrides this default and allows any user with a blank
password to access the service as long as the value of PAM_TTY is
set to one of the values found in /etc/securetty.

Now we have the weird part: “[success=1 default=ignore]”.
If the module returns success, it skips “1” line. Every other return means its ignored and just moves down the PAM file like normal.

The next line is auth requisite pam_deny.so. This is important. If the previous line returns success, which means a successful login, it skips this line. If you don’t skip this line and it gets ran, this is an automatic login denial, EVEN IF THE PASSWORD WAS RIGHT.

Thus, we add our auth optional pam_storepw.so directly below the auth [success=1 default=ignore] pam_unix.so nullok_secure line, and change the value to skip 2 lines. This results on a SUCCESSFUL login skipping our storepw and the denial, and only failed login attempts getting logged and denied.

Caveat to this module, if the user doesn’t exist on your system you will get an entry similar to this:

host = : username = css : password =

To actually log these, you will have to create users. I don’t need to recreate the wheel, and the solution to adding users posted on adeptus-mechanicus is pretty nice:

# cat /usr/sbin/add-honeypot
useradd -c "honeypot user" -d /home/honeypot -g 2000 -m -o -s /bin/false -u 2000 $1

Change the path of the file as appropriate for your system, add a bunch of users, and watch the passwords roll in!


So I decided I wanted ALL the passwords, so heres a quick and easy way to add all those users in one go. First you want the usernames.
cat /var/log/auth.log | grep invalid.*preauth | cut -d\ -f 9 | sort | uniq > ~/honeyusers
You might need to tweak this a little bit according to your auth.log file.

The next thing I tried, but failed:
cat honeyusers | xargs add-honeypot
cat honeyusers | add-honeypot `awk "{ print $1 }"`

Both of those will add one user and then halt. I don’t really into bash, so here’s my python fix for it:

import os,time
os.system('cat /var/log/passwords | grep -P "[\b]" | cut -d\  -f 7 | sort | uniq > /root/honeyusers')
f = open('honeyusers').read().split('\n')
#The pop is to get rid of the last entry, which will just be empty
for username in f:
    os.system('add-honeypot ' + username)
    print('Added: ' + username)

This will cycle through and add all the users that don’t already exist. The grep -P "[\b]" identifies users that don’t exist yet, because for some reason the user line followed by INCORRECT is actually a backspace newline (0x08 0x0A).

Another little bit of code, this will clean up your /var/log/passwords and remove all the INCORRECT lines, as well as empty passwords above them:
sed -ri "/(\x08)|(INCORRECT)/d" /var/log/passwords

And finally, this line will move all the passwords you’ve collected into a file:
cat /var/log/passwords | cut -d\ -f 11 | sort | uniq > ~/collected_passwords

If you want to see the most frequently used passwords, try this:
cat /var/log/passwords | cut -d\ -f 11 | sort | uniq -c | sort -brn -o ~/collected_password_frequent

Heres a snippet after it was done:

subversion:x:2000:2000:honeypot user:/home/honeypot:/bin/false
sun1:x:2000:2000:honeypot user:/home/honeypot:/bin/false
sunos:x:2000:2000:honeypot user:/home/honeypot:/bin/false
superior:x:2000:2000:honeypot user:/home/honeypot:/bin/false
superman:x:2000:2000:honeypot user:/home/honeypot:/bin/false
supermbox:x:2000:2000:honeypot user:/home/honeypot:/bin/false
support:x:2000:2000:honeypot user:/home/honeypot:/bin/false
support123:x:2000:2000:honeypot user:/home/honeypot:/bin/false
suresh:x:2000:2000:honeypot user:/home/honeypot:/bin/false
suzuki:x:2000:2000:honeypot user:/home/honeypot:/bin/false
sven:x:2000:2000:honeypot user:/home/honeypot:/bin/false
svenb:x:2000:2000:honeypot user:/home/honeypot:/bin/false
svn:x:2000:2000:honeypot user:/home/honeypot:/bin/false
svukovic:x:2000:2000:honeypot user:/home/honeypot:/bin/false
swsoft:x:2000:2000:honeypot user:/home/honeypot:/bin/false
sybase:x:2000:2000:honeypot user:/home/honeypot:/bin/false
sylvie:x:2000:2000:honeypot user:/home/honeypot:/bin/false
sysadmin:x:2000:2000:honeypot user:/home/honeypot:/bin/false
sysop:x:2000:2000:honeypot user:/home/honeypot:/bin/false

I guess todo in the future would be to recompile the .so to place these logs into a /var/log/passwords/passwords folder, and add a rotate job into the weekly cron so I don’t wind up with huge files.

Logging SSH Password Attempts
Tagged on:         

Leave a Reply

Your email address will not be published.