[CentOS] an actual hacked machine, in a preserved state

Ross Walker rswwalker at gmail.com
Fri Jan 6 14:46:22 UTC 2012


On Jan 5, 2012, at 6:34 PM, Johnny Hughes <johnny at centos.org> wrote:

> On 01/05/2012 02:51 PM, Bennett Haselton wrote:
>> On 1/5/2012 6:53 AM, Johnny Hughes wrote:
>>> On 01/04/2012 07:47 PM, Bennett Haselton wrote:
>>>> On 1/4/2012 1:59 PM, Lamar Owen wrote:
>>>>> [Distilling to the core matter; everything else is peripheral.]
>>>>> 
>>>>> On Jan 4, 2012, at 2:58 PM, Bennett Haselton wrote:
>>>>>> To be absolutely clear: Do you, personally, believe there is more than a
>>>>>> 1 in a million chance that the attacker who got into my machine, got it
>>>>>> by brute-forcing the password?  As opposed to, say, using an underground
>>>>>> exploit?
>>>>> Here's how I see it breaking down:
>>>>> 
>>>>> 1.) Attacker uses apache remote exploit (or other means) to obtain
>>>>> your /etc/shadow file (not a remote shell, just GET the file without
>>>>> that fact being logged);
>>>>> 2.) Attacker runs cloud-based (and/or CUDA accelerated) brute-forcer
>>>>> on 10,000,000 machines against your /etc/shadow file without your
>>>>> knowledge;
>>>>> 3.) Some time passes;
>>>>> 4.) Attacker obtains your password using distributed brute forcing of
>>>>> the hash in the window of time prior to you resetting it;
>>>>> 5.) Attacker logs in since you allow password login.  You're pwned by
>>>>> a non-login brute-force attack.
>>>>> 
>>>>> In contrast, with ssh keys and no password logins allowed:
>>>>> 
>>>>> 1.) Attacker obtains /etc/shadow and cracks your password after some
>>>>> time;
>>>>> 2.) Attacker additionally obtains /root/.ssh/*
>>>>> 3.) Attacker now has your public key.  Good for them; public keys
>>>>> don't have to be kept secure since it is vastly more difficult to
>>>>> reverse known plaintext, known ciphertext, and the public key into a
>>>>> working private key than it is to brute-force the /etc/shadow hash
>>>>> (part of the difficulty is getting all three required components to
>>>>> successfully reverse your private key; the other part boils down to
>>>>> factoring and hash brute-forcing);
>>>>> 4.) Attacker also has root's public and private keys, if there is a
>>>>> pair in root's ~/.ssh, which may or may not help them.  If there's a
>>>>> passphrase on the private key, it's quite difficult to obtain that
>>>>> from the key;
>>>>> 5.) Attacker can't leverage either your public key or root's key pair
>>>>> (or the machine key; even if they can leverage that to do MitM (which
>>>>> they can and likely will) that doesn't help them obtain your private
>>>>> key for authentication;
>>>>> 6.) Attacker still can't get in because you don't allow password
>>>>> login, even though attacker has root's password.
>>>>> 
>>>>> This only requires an apache httpd exploit that allows reading of any
>>>>> file; no files have to be modified and no shells have to be acquired
>>>>> through any exploits.  Those make it faster, for sure; but even then
>>>>> the attacker is going to acquire your /etc/shadow as one of the first
>>>>> things they do; the next thing they're going to do is install a
>>>>> rootkit with a backdoor password.
>>>>> 
>>>>> Brute-forcing by hash-cracking, not by attempting to login over ssh,
>>>>> is what I'm talking about.
>>>> I acknowledged that the first time I replied to someone's post saying a
>>>> 12-char password wasn't secure enough.  I hypothesized an attacker with
>>>> the fastest GPU-driven password cracker in the world (even allowing for
>>>> 100-factor improvements in coming years) and it would still take
>>>> centuries to break.  I understand about brute-forcing the hash vs.
>>>> brute-forcing the login, but some others had posted about brute-forcing
>>>> the login specifically and I was commenting on how ridiculous that was.
>>>> 
>>>>> This is what I mean when I say 'multilayer metasploit-driven attacks.'
>>>>> 
>>>>> The weakest link is the security of /etc/shadow on the server for
>>>>> password auth (unless you use a different auth method on your server,
>>>>> like LDAP or other, but that just adds a layer, making the attacker
>>>>> work harder to get that all-import password).  Key based auth is
>>>>> superior, since the attacker reading any file on your server cannot
>>>>> compromise the security.
>>>>> 
>>>>> Kerberos is better still.
>>>>> 
>>>>> Now, the weakest link for key auth is the private key itself.  But
>>>>> it's better protected than any password is (if someone can swipe your
>>>>> private key off of your workstation you have bigger problems, and they
>>>>> will have your /etc/shadow for your workstation, and probably a
>>>>> backdoor.....).  The passphrase is also better protected than the
>>>>> typical MD5 hash password, too.
>>>>> 
>>>>> It is the consensus of the security community that key-based
>>>>> authentication with strong private key passphrases is better than any
>>>>> password-only authentication, and that consensus is based on facts
>>>>> derived from evidence of actual break-ins.
>>>> Well yes, on average, password-authentication is going to be worse
>>>> because it includes people in the sample who are using passwords like
>>>> "Patricia".  Did they compare the break-in rate for systems with 12-char
>>>> passwords vs. systems with keys?
>>>> 
>>>> I have nothing in particular against ssh keys - how could anybody be
>>>> "against ssh keys"? :)  My point was that when I asked "How did
>>>> attackers probably get in, given that the password was a random
>>>> 12-character string?" people pounced on the fact that I was using a
>>>> password at all, and kept insisting that that had a non-trivial
>>>> likelihood of being the cause (rather than the
>>>> less-than-one-in-a-billion it actually was), even to the point of making
>>>> ridiculous statements like Mark saying that an attacker trying
>>>> "thousands of times per hour" would get in "sooner or later".  This was
>>>> to the exclusion of what was vastly more likely to be the correct
>>>> answer, which was "Apache, sshd, and CentOS have enough exploits that
>>>> it's far more likely an attacker got in by finding one of those (and
>>>> tools like SELinux help mitigate that)."
>>>> 
>>>> Again, if I hadn't stood behind the math on the issue of long passwords
>>>> vs. keys, I probably never would have gotten the answer that was
>>>> actually useful.
>>>> 
>>>> Do you think it's possible that people focused so much on the use of a
>>>> "password" as a possible cause, vs. the existence of exploits, despite
>>>> the former being literally about 1 billion times less probably than the
>>>> latter, because the former puts more of the blame on the user?  (Not
>>>> that anyone is "to blame" that CentOS and Apache have bugs -- everything
>>>> does -- but that password security would be an issue even with a perfect
>>>> operating system.)
>>>> 
>>>>> While login-based brute-forcing of a password that is long-enough
>>>>> (based upon sshd/login/hashing speed) is impractical for passwords of
>>>>> sufficient strength, login-based brute forcing is not the 'state of
>>>>> the art' in brute-forcing of passwords.  Key-based auth with a
>>>>> passphrase is still not the ultimate, but it is better than only a
>>>>> password, regardless of the strength of that password.
>>>>> 
>>>>> If your password was brute-forced, it really doesn't matter how the
>>>>> attacker did it; you're pwned either way.
>>>>> 
>>>>> It is a safe assumption that there are httpd exploits in the wild,
>>>>> that are not known by the apache project, that specifically attempt to
>>>>> grab /etc/shadow and send to the attacker.  It's also a safe
>>>>> assumption that the attacker will have sufficient horsepower to crack
>>>>> your password from /etc/shadow in a 'reasonable' timeframe for an MD5
>>>>> hash.
>>>> Well I disagree that that's a "safe assumption".  If you think that
>>>> 12-character passwords are within striking distance, try 20-char
>>>> passwords -- 1^36 possible values to search, so with a botnet of over 1
>>>> billion infected computers each checking 10 billion passwords per second
>>>> (both orders of magnitude beyond what's in play today, and
>>>> unrealistically assuming that every resource in the world is focused on
>>>> this one password), it would take on the order of 10 billion years to crack.
>>> Then we get back to rainbow tables and hashes that have been generated
>>> by someone else (with super computer access) and published for "X" sized
>>> passwords (you pick the size: 8,12,20,24,etc).  Then they don't need to
>>> calculate anything, just do a sql lookup against a database with what
>>> they get from the shadow file.  Someone else already cracked all "X"
>>> size logins for all possible iterations.
>> But if the system adds a salt to the password before taking the hash, 
>> then the size of the precomputed rainbow table grows exponentially, 
>> because it has to store the hash of all possible passwords to test, with 
>> all possible salts.  If a 12-char random password (72 bits of 
>> randomness) is salted with a 32-bit salt you now have to precompute 
>> 10^31 values in your rainbow table instead of "only" 10^21.
> 
> OK ... you continue to use passwords on your servers.  I'll use keys and
> require a vpn to access mine.

Keys are definitely more secure then passwords, but a VPN is probably a step back as it provides a pseudo-layer2 gateway to your trusted network.

I might use a key based authenticating reverse-proxy (client cert) that then allows one to ssh in and authenticate using a password. Basically authenticate to a landing page using a client cert, the proxy then adds your IP to a list of authenticated clients which opens the ssh port for that client. As long as your browser has the page open and the proxy re-authenticates every X minutes the ssh port remains open for the client.

-Ross




More information about the CentOS mailing list