New subject: Fail2ban & logrotate [was: Update on spam, postfix, fail2ban, centos 6]

15 Jun 2012


      I have been using centos 6 in a virtualized system for a few months now.
Took a while to batten down the hatches with postfix, rbls, and to use 
fail2ban correctly.
The mailserver for my website(s) are located on the http server as 
well..an 'all in one' server.
DNS servers are separated.
My two sites, and their emails addresses (1 for each) have been around 
for 10 and 15 years respectively.
One site was a business site, one was news and politics...both were very 
busy at one point, thus 'on the radar'
of hackers and spammers.
I decided to see what I could do with my system to prevent hacks and 
spams in regards to email and brute force attacks
on all systems except for my web apps (which are down right now and in 
development).
Fail2ban is really good at the brute force, assuming it is just one ip 
and not all attempts are at once. Thus it works on script kiddies
but I do not think it would work well on a dedicated hack attempt by a 
serious individual or group.
But I am using fail2ban to auto ban ips regarding spam.
As far as spam, very little gets through now. A few a day. Between 
blacklists, my own blacklist of commercial spammers, stringent
settings of postfix the actual spam that gets through is small. But it 
still gets through.
I was using fail2ban on attempts that numbered 3 or more that ended in 
5xx replies from my server. I would block for 10 minutes.
I found I was blocking about 800 ips a day on one server, half that on 
the other.
I did notice that there were a ton of attempts that were under 3. Lots 
of 2's and a ton of 1's.
So a couple weeks ago (not sure when I started) I decided to try 
blocking any 5xx reply by IP.
This is a private server and just my own mail comes to it, so I am not 
worried too much about false positives or other effects.
------------------------------------------------
So what happened?
The ips jumped up considerably, to 1,500 to 1,700 a day banned on one 
server, about 1000 on the other.
What is interesting in those numbers is they are constant. Every week 
day I can count on about 1500 banned ips on one, 1000 on
the other, give or take.
What really changed was the mail servers sending mail that got through 
the restrictions, but were sending to non existent addresses.
A majority (like 80%) were from yahoo. This was a sudden change. It was 
not like this before.
Yahoo spammed like crazy. And they got the mailserver ip banned.
10 to 20 emails a day from yahoo mail servers, going to non existent 
emails. Where before it would be one or two.
The yahoo mails got bigger every day until they started waning (probably 
due to ip banning).
The mail that actually got through all of this was 50% free mail (yahoo, 
msn/live, some aol, etc) Yahoo being the biggest.
Another thing I noticed. When I started adding domains to my 'blacklist 
of commercial senders', legitimate or not, I started to get yahoo
mails with references inside the mails to many of the illegitimate sites 
that were coming from the UCE's I had blocked.
It is quite interesting to watch this process. More interesting that no 
matter how strict or lax I make the system there will be the same
number of attempted mails sent to my server. (give or take a few hundred).
If I unban all the ips, which I did once, there was a one day bump up, 
then it leveled off to the same amount of individual attemtps
(not counting the same attempt being tried again).
I have 35,000 ips blocked right now and nothing changed...except yahoo spam.
Spamassassin I use, but only for level 10 or more spam...it is deleted. 
I found all of these over the last few months to be the kind
with attachments, probably viruses.
-------------------------------------------------------------------------
What Have I learned?
I have learned a large number of attempts are from ISP's and not websites.
I have learned that ISP's will not do anything at all, ever, about this. 
(someone trying to send 1 million mails a day might be suspicious,
but they ignore it)
I have learned a large majority of 'hosts' are technically challenged 
small business owners who have no sys admin knowledge.
Those hosts spew spam bots
I have learned the chinese have really taken a liking to play with my 
server, possibly for training purposes. My server is a hit in beijing
and some other province I cannot spell.
----------------------------------------------------------
What can be done?
Not much. If the isp's do nothing, and the technology is not available 
to datacenters and hosts, there is not much I can do at all.
Complaining to an isp or host would take 24 hours a day of messages, 99% 
which would be ignored.
There is a consideration for the scumbags that call themselves 
legitimate mailers, like vocus.com. They are in the US, as I am.
I am considering going to small claims for some of these spam attempts. 
I cannot use the can-spam act, since they are technically
not in violation.
However, I could use the logs and attempts, copies of emails and phone 
calls telling them to stop, and sue them
for a small dedicated denial of service attack, use of my bandwidth, 
harassment of my server and business.
Would I win? Probably. Would I ever get money from them? Most likely not.