On Sun, Jan 1, 2012 at 2:55 PM, Eero Volotinen <eero.volotinen at iki.fi>wrote: > 2012/1/2 Bennett Haselton <bennett at peacefire.org>: > > (Sorry, third time -- last one, promise, just giving it a subject line!) > > > > OK, a second machine hosted at the same hosting company has also > apparently > > been hacked. Since 2 of out of 3 machines hosted at that company have > now > > been hacked, but this hasn't happened to any of the other 37 dedicated > > servers that I've got hosted at other hosting companies (also CentOS, > same > > version or almost), this makes me wonder if there's a security breach at > > this company, like if they store customers' passwords in a place that's > > been hacked. (Of course it could also be that whatever attacker found an > > exploit, was just scanning that company's address space for hackable > > machines, and didn't happen to scan the address space of the other > hosting > > companies.) > > > > So, following people's suggestions, the machine is disconnected and > hooked > > up to a KVM so I can still examine the files. I've found this file: > > -rw-r--r-- 1 root root 1358 Oct 21 17:40 /home/file.pl > > which appears to be a copy of this exploit script: > > http://archive.cert.uni-stuttgart.de/bugtraq/2006/11/msg00302.html > > Note the last-mod date of October 21. > > > > No other files on the system were last modified on October 21st. However > > there was a security advisory dated October 20th which affected httpd: > > > http://mailinglist-archive.com/centos-announce/2011-10/00035-CentOSannounce+CESA20111392+Moderate+CentOS+5+i386+httpd+Update > > https://rhn.redhat.com/errata/RHSA-2011-1392.html > > > > and a large number of files on the machine, including lots of files in */ > > usr/lib64/httpd/modules/* and */lib/modules/2.6.18-274.7.1.el5/kernel/* , > > have a last-mod date of October 20th. So I assume that these are files > > which were updated automatically by yum as a result of the patch that > goes > > with this advisory -- does that sound right? > > > > So a couple of questions that I could use some help with: > > > > 1) The last patch affecting httpd was released on October 20th, and the > > earliest evidence I can find of the machine being hacked is a file dated > > October 21st. This could be just a coincidence, but could it also > suggest > > that the patch on October 20th introduced a new exploit, which the > attacker > > then used to get in on October 21st? > > (Another possibility: I think that when yum installs updates, it > > doesn't actually restart httpd. So maybe even after the patch was > > installed, my old httpd instance kept running and was still vulnerable? > As > > for why it got hacked the very next day, maybe the attacker looked at the > > newly released patch and reverse-engineered it to figure out where the > > vulnerabilities were, that the patch fixed?) > > > > 2) Since the */var/log/httpd/* and /var/log/secure* logs only go back 4-5 > > weeks by default, it looks like any log entries related to how the > attacker > > would have gotten in on or before October 21st, are gone. (The secure* > > logs do show multiple successful logins as "root" within the last 4 > weeks, > > mostly from IP addresses in Asia, but that's to be expected once the > > machine was compromised -- it doesn't help track down how they originally > > got in.) Anywhere else that the logs would contain useful data? > > sshd with root login enabled with very bad password? > > Forgot to mention: the root password was: 1WyJstJZnQ!j (I have since changed it). (I have already practically worn out my keyboard explaining the math behind why I think a 12-character alphanumeric password is secure enough :) ) Bennett