Message-ID: 479F2A63.2070408@centos.org
On: Tue, 29 Jan 2008 07:30:11 -0600, Johnny Hughes johnny@centos.org Subject Was: [CentOS] Unknown rootkit causes compromised servers
SOME of the script kiddies check higher ports for SSH *_BUT_* I only see 4% of the brute force attempts to login on ports other than 22.
I would say that dropping brute force login attempts by 96% is quite a good reason to move the SSH port from 22 to something else.
I am not a fan of security through obscurity. If a port is open to the internet then it must be secured whether it is well known or not and if it is properly secured then changing the port number customarily assigned provides no measurable benefit. In my opinion, arbitrarily switching port numbers for well known services provides only the illusion of security while often inconveniencing the legitimate users in unpredictable, and sometimes expensively resolved, fashions.
To deal with brute force attacks (not just on ssh) I spent some time tracking down how others had dealt with the problem. I discovered thereby that one can use the simple linux firewall iptables to restrict the number of connections to a given port from a single source over a specified interval. I therefore added these rules to my /etc/sysconfig/iptables file:
... # This is usually present in all setups but, you never know.... # Established connections go right through. -A RH-Firewall-1-INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT ...
# Block brute force attacks # Drop repeated ssh connection attempts within 20 seconds interval -A RH-Firewall-1-INPUT -p tcp -m tcp -m state -m recent -i eth0 --dport 22 --state NEW -j DROP --rcheck --seconds 20 --name THROTTLE --rsource
# Accept ssh connection if not attempted within past 20 sec. -A RH-Firewall-1-INPUT -p tcp -m tcp -m state -m recent -i eth0 --dport 22 --state NEW -j ACCEPT --set --name THROTTLE --rsource
You can change the interval from 20 seconds to whatever you feel represents a decent compromise between user satisfaction and security. Many authorities considered a value between 3 and 6 seconds sufficient to render brute force attacks impractical. These rules can be trivially modified to protect any destination port (-dport 21 for ftp for instance) or protocol (-p udp).
I hope this information is of use to some of you. I find this list and its archives very helpful myself.
Regards,
I use this one, works great and easy to setup
http://rfxnetworks.com/bfd.php
On Jan 30, 2008 11:54 AM, James B. Byrne byrnejb@harte-lyne.ca wrote:
Message-ID: 479F2A63.2070408@centos.org
On: Tue, 29 Jan 2008 07:30:11 -0600, Johnny Hughes johnny@centos.org Subject Was: [CentOS] Unknown rootkit causes compromised servers
SOME of the script kiddies check higher ports for SSH *_BUT_* I only see 4% of the brute force attempts to login on ports other than 22.
I would say that dropping brute force login attempts by 96% is quite a good reason to move the SSH port from 22 to something else.
I am not a fan of security through obscurity. If a port is open to the internet then it must be secured whether it is well known or not and if it is properly secured then changing the port number customarily assigned provides no measurable benefit. In my opinion, arbitrarily switching port numbers for well known services provides only the illusion of security while often inconveniencing the legitimate users in unpredictable, and sometimes expensively resolved, fashions.
To deal with brute force attacks (not just on ssh) I spent some time tracking down how others had dealt with the problem. I discovered thereby that one can use the simple linux firewall iptables to restrict the number of connections to a given port from a single source over a specified interval. I therefore added these rules to my /etc/sysconfig/iptables file:
... # This is usually present in all setups but, you never know.... # Established connections go right through. -A RH-Firewall-1-INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT ...
# Block brute force attacks # Drop repeated ssh connection attempts within 20 seconds interval -A RH-Firewall-1-INPUT -p tcp -m tcp -m state -m recent -i eth0 --dport 22 --state NEW -j DROP --rcheck --seconds 20 --name THROTTLE --rsource
# Accept ssh connection if not attempted within past 20 sec. -A RH-Firewall-1-INPUT -p tcp -m tcp -m state -m recent -i eth0 --dport 22 --state NEW -j ACCEPT --set --name THROTTLE --rsource
You can change the interval from 20 seconds to whatever you feel represents a decent compromise between user satisfaction and security. Many authorities considered a value between 3 and 6 seconds sufficient to render brute force attacks impractical. These rules can be trivially modified to protect any destination port (-dport 21 for ftp for instance) or protocol (-p udp).
I hope this information is of use to some of you. I find this list and its archives very helpful myself.
Regards,
-- *** E-Mail is NOT a SECURE channel *** James B. Byrne mailto:ByrneJB@Harte-Lyne.ca Harte & Lyne Limited http://www.harte-lyne.ca 9 Brockley Drive vox: +1 905 561 1241 Hamilton, Ontario fax: +1 905 561 0757 Canada L8E 3C3
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On Wed, Jan 30, 2008 at 12:17 PM, Ed Donahue liberaled@gmail.com wrote:
On Jan 30, 2008 11:54 AM, James B. Byrne byrnejb@harte-lyne.ca wrote:
Message-ID: 479F2A63.2070408@centos.org
On: Tue, 29 Jan 2008 07:30:11 -0600, Johnny Hughes johnny@centos.org Subject Was: [CentOS] Unknown rootkit causes compromised servers
SOME of the script kiddies check higher ports for SSH *_BUT_* I only see 4% of the brute force attempts to login on ports other than 22.
I would say that dropping brute force login attempts by 96% is quite a good reason to move the SSH port from 22 to something else.
I am not a fan of security through obscurity. If a port is open to the internet then it must be secured whether it is well known or not and if it
is
properly secured then changing the port number customarily assigned
provides
no measurable benefit. In my opinion, arbitrarily switching port numbers
for
well known services provides only the illusion of security while often inconveniencing the legitimate users in unpredictable, and sometimes expensively resolved, fashions.
To deal with brute force attacks (not just on ssh) I spent some time
tracking
down how others had dealt with the problem. I discovered thereby that one
can
use the simple linux firewall iptables to restrict the number of
connections
to a given port from a single source over a specified interval. I
therefore
added these rules to my /etc/sysconfig/iptables file:
... # This is usually present in all setups but, you never know.... # Established connections go right through. -A RH-Firewall-1-INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT ...
# Block brute force attacks # Drop repeated ssh connection attempts within 20 seconds interval -A RH-Firewall-1-INPUT -p tcp -m tcp -m state -m recent -i eth0 --dport 22 --state NEW -j DROP --rcheck --seconds 20 --name THROTTLE --rsource
# Accept ssh connection if not attempted within past 20 sec. -A RH-Firewall-1-INPUT -p tcp -m tcp -m state -m recent -i eth0 --dport 22 --state NEW -j ACCEPT --set --name THROTTLE --rsource
You can change the interval from 20 seconds to whatever you feel
represents a
decent compromise between user satisfaction and security. Many
authorities
considered a value between 3 and 6 seconds sufficient to render brute
force
attacks impractical. These rules can be trivially modified to protect any destination port (-dport 21 for ftp for instance) or protocol (-p udp).
I hope this information is of use to some of you. I find this list and
its
archives very helpful myself.
Regards,
James B. Byrne mailto:ByrneJB@Harte-Lyne.ca
I use this one, works great and easy to setup
Log parsing scripts often don't provide the immediacy that rate limiting does when under attack. You'd have to run the script constantly parsing logs, since most ssh scans come in bursts.
@James: As for the "security through obscurity" post, you are missing the point. Changing the port number that SSH runs on is not "security through obscurity". Moving an already highly secure service to a different port so scanners don't hit it automatically is a different thing. This type of move is purely to reduce the amount of garbage in one's log file due to automated scans. However, I do agree that there are probably better ways to handle the situation, such as using rate limiting.
Security through obscurity would be something like leaving a shell that requires no login running on some random port, and hoping nobody finds it.
Brian Mathis wrote:
@James: As for the "security through obscurity" post, you are missing the point. Changing the port number that SSH runs on is not "security through obscurity". Moving an already highly secure service to a different port so scanners don't hit it automatically is a different thing. This type of move is purely to reduce the amount of garbage in one's log file due to automated scans. However, I do agree that there are probably better ways to handle the situation, such as using rate limiting.
Not to mention that if there is a lot less "garbage", it is much easier to catch something trying to sneak in. So it does have an element of security to it.
Patrick
Good security is like an onion. The users' think it smells... No, it's layered.
Changing the the sshd port from the default does add a layer, a thin layer, but a layer all the same.
The rate limiting is a somewhat thicker layer.
I personally prefer to block all ssh traffic from the internet and have my customers vpn to my server which let's me ssh over the vpn to their machines. If they happen to have dynamic addresses, it doesn't matter to me.
Patrick wrote:
Brian Mathis wrote:
@James: As for the "security through obscurity" post, you are missing the point. Changing the port number that SSH runs on is not "security through obscurity". Moving an already highly secure service to a different port so scanners don't hit it automatically is a different thing. This type of move is purely to reduce the amount of garbage in one's log file due to automated scans. However, I do agree that there are probably better ways to handle the situation, such as using rate limiting.
Not to mention that if there is a lot less "garbage", it is much easier to catch something trying to sneak in. So it does have an element of security to it.
Patrick
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On Wed, Jan 30, 2008, Brian Mathis wrote: ...
Log parsing scripts often don't provide the immediacy that rate limiting does when under attack. You'd have to run the script constantly parsing logs, since most ssh scans come in bursts.
We use swatch for this and othter interesting events (e.g. NICs being put in promiscuous mode). It continually monitors one or more log files using gnu-tail in a perl script, and can do various things depending on a configuration file. It can send e-mail notifications and/or execute scripts which can do anything your heart desires.
The fail2ban program has similar capabilities, and can block IP addresses attempting multiple connections using iptables. Personally I prefer swatch, but that's largely because I found it first and understand its configuration.
We generally restrict ssh access to using authorized_keys, and use tcp_wrappers to further limit access by IP address.
Roaming users can first establish a VPN connection using OpenVPN, then make any ssh connections vis the private VPN tunnel.
Bill -- INTERNET: bill@celestial.com Bill Campbell; Celestial Software LLC URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way FAX: (206) 232-9186 Mercer Island, WA 98040-0820; (206) 236-1676
Many citizens because of their respect for what only appears to be a law are cunningly coerced into waiving their rights due to ignorance. -- U.S. v. Minker
On Wed, 2008-01-30 at 13:11 -0800, Bill Campbell wrote:
On Wed, Jan 30, 2008, Brian Mathis wrote: ...
Log parsing scripts often don't provide the immediacy that rate limiting does when under attack. You'd have to run the script constantly parsing logs, since most ssh scans come in bursts.
We use swatch for this and othter interesting events (e.g. NICs being put in promiscuous mode). It continually monitors one or more log files using gnu-tail in a perl script, and can do various things depending on a configuration file. It can send e-mail notifications and/or execute scripts which can do anything your heart desires.
Hello,
Do you have any specific swatch config lines for detecting ssh brute-force attacks? If so would you care to share them? (off-list if you prefer). Likewise we use swatch for general log monitoring, and have it report back anything unusual to our central monitoring system (Big Brother).
John.
On Mon, Feb 04, 2008, John Horne wrote:
On Wed, 2008-01-30 at 13:11 -0800, Bill Campbell wrote:
On Wed, Jan 30, 2008, Brian Mathis wrote: ...
Log parsing scripts often don't provide the immediacy that rate limiting does when under attack. You'd have to run the script constantly parsing logs, since most ssh scans come in bursts.
We use swatch for this and othter interesting events (e.g. NICs being put in promiscuous mode). It continually monitors one or more log files using gnu-tail in a perl script, and can do various things depending on a configuration file. It can send e-mail notifications and/or execute scripts which can do anything your heart desires.
Hello,
Do you have any specific swatch config lines for detecting ssh brute-force attacks? If so would you care to share them? (off-list if you prefer). Likewise we use swatch for general log monitoring, and have it report back anything unusual to our central monitoring system (Big Brother).
Here's part of the swatchrc file from one of our public servers. We get many more reports from sshd via tcp_wrappers (libwrap) which have been modified to use DNSRBLs to white and black list various hosts and IP ranges.
perlcode 0 use Sys::Hostname::Long; perlcode 0 my $host_long = hostname_long; perlcode 0 my $email=qq(support@$host_long); perlcode 0 my $secmail = qq(security@$host_long); perlcode my ($month, $day, $time, $host_name, @message) = split(/\s+/);
watchfor /device (\S+) entered promiscuous mode/ mail addresses=$secmail, subject=[swatch] $host_name promiscuous $1
watchfor /File name too long/ mail addresses=$email, subject=[swatch] BufferOverflow_attempt
watchfor /DHCPREQUEST/ mail addresses=postmaster, subject=[swatch] $host_name@message
watchfor /Failed password for.*from\s+(\S+)/ threshold track_by=$1,type=both,count=3,seconds=60 mail addresses=$secmail, subject=[swatch] $host_name@message
watchfor /Accepted password for root.*from\s+(\S+)/ mail addresses=$secmail, subject=[swatch] $host_name ssh password $1
watchfor /Accepted publickey for root.*from\s+(\S+)/ mail addresses=$secmail, subject=[swatch] $host_name ssh publickey $1
watchfor /Invalid login as admin/ mail addresses=$secmail, subject=[swatch] $host_name@message
watchfor /Invalid login as mainadmin/ mail addresses=$secmail, subject=[swatch] $host_name@message
watchfor /Successful login as mainadmin/ mail addresses=$secmail, subject=[swatch] $host_name@message
watchfor /DeliveryErrors/ mail addresses=postmaster, subject=[swatch] Postfix_Delivery_Errors
watchfor /file system full/ mail addresses=$email, subject=[swatch] $host_name@message
watchfor /refused connect from\s+(\S+)/ threshold track_by=$1,type=both,count=3,seconds=60 mail addresses=$secmail, subject=[swatch] $host_name@message
# end of file
Bill -- INTERNET: bill@celestial.com Bill Campbell; Celestial Software LLC URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way FAX: (206) 232-9186 Mercer Island, WA 98040-0820; (206) 236-1676
there is nothing more difficult to take in hand, more perilous to conduct, or more uncertain in its success, than to take the lead in the introduction of a new order of things. Because the innovator has for enemies all those who have done well under the old conditions, and lukewarm defenders in those who may do well under the new. -- Machiavelli
On Wed, Jan 30, 2008 at 12:17:22PM -0500, Ed Donahue wrote:
I use this one, works great and easy to setup http://rfxnetworks.com/bfd.php
This is how I deal with them: deny by default unless you know the "secret handshake".
http://wiki.xdroop.com/space/Linux/Limited+SSH+Access
James B. Byrne wrote:
Message-ID: 479F2A63.2070408@centos.org
On: Tue, 29 Jan 2008 07:30:11 -0600, Johnny Hughes johnny@centos.org Subject Was: [CentOS] Unknown rootkit causes compromised servers
SOME of the script kiddies check higher ports for SSH *_BUT_* I only see 4% of the brute force attempts to login on ports other than 22.
I would say that dropping brute force login attempts by 96% is quite a good reason to move the SSH port from 22 to something else.
I am not a fan of security through obscurity. If a port is open to the internet then it must be secured whether it is well known or not and if it is properly secured then changing the port number customarily assigned provides no measurable benefit.
If you consider this security through obscurity, then why not publish the list of your users on a public web page? after all, you should use strong passwords, so why hide usernames? and how about also publishing the list of your files and directories, of package versions, ... etc. Relying on the secrecy of such infos is security through obscurity too ;-p
Of course one must secure the setup and not rely solely on a port number. but using a different port:
- reduces the noise, and the stress level, so one can audit logs quietly instead of trying to separate kiddie attempts from serious attacks...
- an attacker needs to find the port. In general, this means some form of port scanning. and before he finds the port, there is a chance that he gets caught. Not certain, but still. There is the case of an attacker who guesses the port at once or an attacker using N machines to do the scanning, which is why one must not rely on the port choice, but this will happen less. better fight few ennemies than the full jungle.
In my opinion, arbitrarily switching port numbers for well known services provides only the illusion of security while often inconveniencing the legitimate users in unpredictable, and sometimes expensively resolved, fashions.
What I would I like to do is:
- allow 22 from specific IPs - allow another port (redirected) from anywhere. this port is then redirected to 22.
I guess this requires marking the redirected packets so they can be allowed to go to port 22? anyone having a working ruleset for this?
This way, users and programs that connect from specific machines do not need to use a different port (which becomes quickly annoying if you use rsync or other tasks over ssh and you don't want to spend your times setting a .ssh/config).
[snip]
What I would I like to do is:
- allow 22 from specific IPs
- allow another port (redirected) from anywhere. this port is then
redirected to 22.
I do exactly this with a combination of SSH config options and iptables rules. In your /etc/ssh/sshd_config file, find the "Port 22" statement and add a "Port" statement for the desired port, something like:
<snip> Port 22 Port 20022 Protocol 2 <snip>
Then, in iptables, add the appropriate rules to let incoming connections to port 22 from only specific addresses and to allow port 20022 (or whatever you pick) to be available worldwide. Assuming you wanted port 22 access for a local subnet like 192.169.1.0/24, add the following to the /etc/sysconfig/iptables file before the REJECT statement at the end of the file:
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp -s 192.168.1.0/24 --dport 22 -j ACCEPT -A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 20022 -j ACCEPT
After restarting SSH and reloading iptables you should have just what you want. I use this, in addition to blockhosts (http://www.aczoom.com/cms/blockhosts/), on several production systems and the result has been almost total elimination of brute-force attacks. on those systems.
Another possibility is a variation on port-knocking using PKI authentication or a shared secret. The project is called fwknop (http://www.cipherdyne.org/fwknop/) and has the potential to almost completely eliminate brute-force attacks.
Essentially, the target port (22 in the case of SSH) is not open at all normally, but a daemon monitors the network interface for a specific packet signed using either a shared secret or a pre-authorized PGP key. When it sees the packet, it opens up the appropriate port for a specified time (usually just a few seconds) to the IP address the packet comes from. This allows a very short time window for the client system to complete its connection before the port gets closed down. I've set this up on a couple of systems so far with excellent results.
Your mileage may vary!
Jay Leafey wrote:
What I would I like to do is:
- allow 22 from specific IPs
- allow another port (redirected) from anywhere. this port is then
redirected to 22.
I do exactly this with a combination of SSH config options and iptables rules. In your /etc/ssh/sshd_config file, find the "Port 22" statement and add a "Port" statement for the desired port, something like:
<snip> Port 22 Port 20022 Protocol 2 <snip>
Then, in iptables, add the appropriate rules to let incoming connections to port 22 from only specific addresses and to allow port 20022 (or whatever you pick) to be available worldwide. Assuming you wanted port 22 access for a local subnet like 192.169.1.0/24, add the following to the /etc/sysconfig/iptables file before the REJECT statement at the end of the file:
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp -s 192.168.1.0/24 --dport 22 -j ACCEPT -A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 20022 -j ACCEPT
After restarting SSH and reloading iptables you should have just what you want. I use this, in addition to blockhosts (http://www.aczoom.com/cms/blockhosts/), on several production systems and the result has been almost total elimination of brute-force attacks. on those systems.
Another possibility is a variation on port-knocking using PKI authentication or a shared secret. The project is called fwknop (http://www.cipherdyne.org/fwknop/) and has the potential to almost completely eliminate brute-force attacks.
Essentially, the target port (22 in the case of SSH) is not open at all normally, but a daemon monitors the network interface for a specific packet signed using either a shared secret or a pre-authorized PGP key. When it sees the packet, it opens up the appropriate port for a specified time (usually just a few seconds) to the IP address the packet comes from. This allows a very short time window for the client system to complete its connection before the port gets closed down. I've set this up on a couple of systems so far with excellent results.
Your mileage may vary!
I didn't think about adding the port in ssh. Thanks for the hint.
I was however looking for a way to implement this without touching sshd ports. After playing a little, I found the following to work
iptables -t nat -A PREROUTING -p tcp -m tcp --dport 12345 -j REDIRECT --to-ports 22 iptables -t mangle -A PREROUTING -p tcp -m tcp --dport 12345 -j MARK --set-mark 0x22 iptables -A INPUT -m mark --mark 0x22 -j ACCEPT
seems to do it. (12345 is not the real port).
James B. Byrne wrote:
I am not a fan of security through obscurity.
You're diluting a useful phrase.
It originally referred to practices where obscurity was the _only_ source of security. As soon as you saw through the obscurity, there was no security. Of course, this means that there was no real security to begin with. Hiding telnet by changing its listening port would be security through obscurity.
Changing ssh's service port is not at all the same thing. If you get past this bit of obscurity, security remains. To employ another popular phrase, it's defense in depth. The more walls you can throw up, the less stress on the final wall that stops the attack.
If you don't like those arguments, the CPU usage difference should matter to you. Your system can reject (or ignore) a connection to an unused port with a lot less CPU power than it takes to reject a login attempt, especially to a cryptographically protected service. And, the post that started this says there's a 96% chance that the port rejection runs the attacker off completely, whereas a password rejection just invites another attempt. So, one TCP RST packet vs. several thousand login attempts. Which would you rather allocate CPU resources for?
-
Bill Campbell -
Brian Mathis -
David Mackintosh -
Ed Donahue -
James B. Byrne -
Jay Leafey -
John Horne -
Milton Calnek -
mouss -
Patrick -
Warren Young