I'm running a mix of CentOS 5 and 6 servers reachable by ssh from the Internet. Of course I allow only public key authentication and no root login. In addition I'm running fail2ban to block obnoxious brute force attack sources.
On CentOS 6 this is working pretty well, but on CentOS 5 there's one class of attacks fail2ban fails to ban. (No pun intended.) This isn't fail2ban's fault, but openssh's. When the source IP address of a failed attempt fails the reverse mapping check, CentOS 6 (openssh-server-5.3p1-81.el6_3.x86_64) logs:
Mar 3 04:06:34 posthamster sshd[1718]: reverse mapping checking getaddrinfo for hn.ly.kd.adsl [61.163.113.72] failed - POSSIBLE BREAK-IN ATTEMPT!
from which fail2ban can pick up and block IP address 61.163.113.72 just fine. CentOS 5 (openssh-server-4.3p2-82.el5) OTOH logs:
Mar 3 04:44:48 gimli sshd[12870]: reverse mapping checking getaddrinfo for hn.ly.kd.adsl failed - POSSIBLE BREAK-IN ATTEMPT!
without the IP address. The name is of no use because sshd just confirmed that it doesn't really correspond to the attacker's IP address.
Any ideas how to remedy that situation?
TIA T.
Am 06.03.2013 19:20, schrieb Gordon Messmer:
On 03/06/2013 09:45 AM, Tilman Schmidt wrote:
Any ideas how to remedy that situation?
As long as you get the IP address for failed logins, ignore reverse mapping failures.
Trouble is, I don't:
Feb 8 00:03:09 dns01 sshd[6119]: reverse mapping checking getaddrinfo for mbl-99-61-82.dsl.net.pk failed - POSSIBLE BREAK-IN ATTEMPT! Feb 8 00:03:10 dns01 sshd[6120]: Disconnecting: Too many authentication failures for root Feb 8 00:03:19 dns01 sshd[6121]: reverse mapping checking getaddrinfo for mbl-99-61-82.dsl.net.pk failed - POSSIBLE BREAK-IN ATTEMPT! Feb 8 00:03:20 dns01 sshd[6122]: Disconnecting: Too many authentication failures for root Feb 8 00:03:22 dns01 sshd[6123]: reverse mapping checking getaddrinfo for mbl-99-61-82.dsl.net.pk failed - POSSIBLE BREAK-IN ATTEMPT! Feb 8 00:03:23 dns01 sshd[6124]: Disconnecting: Too many authentication failures for root [...]
And at the end of the day, logwatch tells me:
--------------------- SSHD Begin ------------------------
Disconnecting after too many authentication failures for user: root : 149 Time(s)
Not good.
You could deny all by default and only allow your locations in tcp_wrappers.
Add this to /etc/hosts.deny:
sshd: ALL
And this to /etc/hosts.allow
sshd: 12.34.56.78 your.ip.here 123. 12.34.
I exaggerated the spaces. You'd still get the failures in your logs, but access to the service won't be granted as it wouldn't match the allow.
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Tilman Schmidt Sent: Thursday, March 07, 2013 11:45 AM To: CentOS mailing list Subject: Re: [CentOS] CentOS 5 sshd does not log IP address of reverse mapping failure
Am 06.03.2013 19:20, schrieb Gordon Messmer:
On 03/06/2013 09:45 AM, Tilman Schmidt wrote:
Any ideas how to remedy that situation?
As long as you get the IP address for failed logins, ignore reverse mapping failures.
Trouble is, I don't:
Feb 8 00:03:09 dns01 sshd[6119]: reverse mapping checking getaddrinfo for mbl-99-61-82.dsl.net.pk failed - POSSIBLE BREAK-IN ATTEMPT! Feb 8 00:03:10 dns01 sshd[6120]: Disconnecting: Too many authentication failures for root Feb 8 00:03:19 dns01 sshd[6121]: reverse mapping
checking
getaddrinfo for mbl-99-61-82.dsl.net.pk failed - POSSIBLE BREAK-IN ATTEMPT! Feb 8 00:03:20 dns01 sshd[6122]: Disconnecting: Too many authentication failures for root Feb 8 00:03:22 dns01 sshd[6123]: reverse mapping
checking
getaddrinfo for mbl-99-61-82.dsl.net.pk failed - POSSIBLE BREAK-IN ATTEMPT! Feb 8 00:03:23 dns01 sshd[6124]: Disconnecting: Too many authentication failures for root [...]
And at the end of the day, logwatch tells me:
--------------------- SSHD Begin ------------------------
Disconnecting after too many authentication failures for user: root : 149 Time(s)
Not good.
-- Tilman Schmidt Phoenix Software GmbH Bonn, Germany
Am 07.03.2013 19:07, schrieb Michael Krug:
You could deny all by default and only allow your locations in tcp_wrappers.
Can't do that. People must be able to ssh in from dynamic IPs.
On Fri, Mar 8, 2013 at 7:43 AM, Tilman Schmidt t.schmidt@phoenixsoftware.de wrote:
Am 07.03.2013 19:07, schrieb Michael Krug:
You could deny all by default and only allow your locations in tcp_wrappers.
Can't do that. People must be able to ssh in from dynamic IPs.
Sure, but as you've noticed, logging the reverse-DNS isn't particularly useful there. Turn it off and you should get the number in the logs.
On Thu, Mar 7, 2013 at 10:45 AM, Tilman Schmidt t.schmidt@phoenixsoftware.de wrote:
Any ideas how to remedy that situation?
As long as you get the IP address for failed logins, ignore reverse mapping failures.
Trouble is, I don't:
Does it work if you set UseDNS no in /etc/ssh/sshd_config?
I don't think it is that useful anyway.
Am 07.03.2013 19:49, schrieb Les Mikesell:
On Thu, Mar 7, 2013 at 10:45 AM, Tilman Schmidt t.schmidt@phoenixsoftware.de wrote:
Any ideas how to remedy that situation?
As long as you get the IP address for failed logins, ignore reverse mapping failures.
Trouble is, I don't:
Does it work if you set UseDNS no in /etc/ssh/sshd_config?
Not really. That seems to remove the "reverse mapping checking failed" messages (assuming there were the usual number of such attempts after I set that option), but IP addresses for failed logins to existing users are never logged. The log contains just:
sshd[27912]: Disconnecting: Too many authentication failures for root
In contrast, log entries for login attempts with non-existing user names do contain the source IP address:
sshd[30576]: Invalid user condor from 62.201.70.8
But this is true on both CentOS 5 and 6, so it's apparently the way OpenSSH decided to do things, and cannot be remedied by the distribution.
On 03/07/2013 08:45 AM, Tilman Schmidt wrote:
As long as you get the IP address for failed logins, ignore reverse mapping failures.
Trouble is, I don't:
Are you watching the messages or secure log?
# cat /etc/redhat-release CentOS release 5.8 (Final) # tail -f /var/log/secure Mar 8 11:46:54 firewall sshd[27455]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=173-xx-xx-xx-washington.hfc.comcastbusiness.net user=root Mar 8 11:46:56 firewall sshd[27455]: Failed password for root from 173.xx.xx.xx port 51437 ssh2
The standard configuration should be logging the IP address of failed logins.
I don't think I have access to any hosts where the reverse lookup is broken, so I'm not sure if what you're seeing is a result of a logging bug related to PTR mismatch, or what.
Am 08.03.2013 20:51, schrieb Gordon Messmer:
# tail -f /var/log/secure Mar 8 11:46:54 firewall sshd[27455]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=173-xx-xx-xx-washington.hfc.comcastbusiness.net user=root Mar 8 11:46:56 firewall sshd[27455]: Failed password for root from 173.xx.xx.xx port 51437 ssh2
I think I see what's happening now.
The machines in question all have password authentication disabled, so they obviously never log "Failed password". If someone tries to log in to an existing user account with password authentication, she gets the message "no supported authentication methods available" or something like that. In that case /var/log/secure does not log a failure message. The only trace of that attempt is a "Received disconnect", like here after the message I cited in my original posting:
Mar 3 04:44:48 gimli sshd[12870]: reverse mapping checking getaddrinfo for hn.ly.kd.adsl failed - POSSIBLE BREAK-IN ATTEMPT! Mar 3 04:44:49 gimli sshd[12871]: Received disconnect from 61.163.113.72: 11: Bye Bye
If I set "UseDNS no" the first message disappears and only the second one remains.
So it seems there is no way to identify password bruteforcing attempts on servers which don't accept password authentication in the first place.
On 03/09/2013 09:57 AM, Tilman Schmidt wrote:
So it seems there is no way to identify password bruteforcing attempts on servers which don't accept password authentication in the first place.
Yes... you can't detect what you don't receive. If you want to block hosts that are scanning for vulnerabilities, you could set up a honeypot. Watch its logs and ship off the firewall rules to a device at the edge of your network.
On Sat, Mar 9, 2013 at 11:57 AM, Tilman Schmidt t.schmidt@phoenixsoftware.de wrote:
Mar 3 04:44:48 gimli sshd[12870]: reverse mapping checking getaddrinfo for hn.ly.kd.adsl failed - POSSIBLE BREAK-IN ATTEMPT! Mar 3 04:44:49 gimli sshd[12871]: Received disconnect from 61.163.113.72: 11: Bye Bye
If I set "UseDNS no" the first message disappears and only the second one remains.
So it seems there is no way to identify password bruteforcing attempts on servers which don't accept password authentication in the first place.
Can't you pick some reasonable number of 'received disconnect' messages to allow from a single IP?
Am 10.03.2013 03:01, schrieb Les Mikesell:
On Sat, Mar 9, 2013 at 11:57 AM, Tilman Schmidt t.schmidt@phoenixsoftware.de wrote:
Mar 3 04:44:48 gimli sshd[12870]: reverse mapping checking getaddrinfo for hn.ly.kd.adsl failed - POSSIBLE BREAK-IN ATTEMPT! Mar 3 04:44:49 gimli sshd[12871]: Received disconnect from 61.163.113.72: 11: Bye Bye
If I set "UseDNS no" the first message disappears and only the second one remains.
So it seems there is no way to identify password bruteforcing attempts on servers which don't accept password authentication in the first place.
Can't you pick some reasonable number of 'received disconnect' messages to allow from a single IP?
Yes, I think that should work. I was worried that "received disconnect" messages might also appear for legitimate connections, but looking through my logs it seems that they don't.
I have set it up as a test on one of my servers with a threshold of 15 attempts in 1000 secs now to see how it will fare.
Thanks, Tilman
-
Gordon Messmer -
Les Mikesell -
Michael Krug -
Tilman Schmidt