[CentOS] Re: Kind of OT: internal imap server

Fri Sep 1 16:33:34 UTC 2006
Aleksandar Milivojevic <alex at milivojevic.org>

Quoting Feizhou <feizhou at graffiti.net>:

> So you are saying that mimedefang is not reliable if not provided
> enough resources. What happens to the mail then? temporary reject or
> let through?

Whatever way you configure Sendmail and MIMEDefang.

In Sendmail, you can configure it to either permanently reject,  
temporary reject or just pass through in case of filter failure and/or  
if filter gets stuck (there's configurable timeout).  These are  
per-filter settings, so if you for example have AV scan in one filter,  
and spam scan in another filter, you can configure Sendmail to  
temp-fail if AV filter fails, and to pass through if spam filter fails.

MIMEDefang also comes with its own set of configuartion direcitves.   
You can configure maximum number of slaves, the minimum number of  
slaves to keep around, and what to do if the slave misbehaves (for  
example, doesn't return in configurable amount of time).  And of  
course, in your filter script (the Perl code you are supposed to  
write) you can take whatever action you like in case your script  
detects an error condition.

Some of this settings you need to coordinate a bit.  For example,  
there is no point in telling MIMEDefang to wait 1 minute for a slave  
to finish processing, and than telling Sendmail to wait only 10  
seconds for MIMEDefang (you want timeout in Sendmail to be longer than  
timeout in MIMEDefang).

Another word of warnning if using SpamAssassin from MIMEDefang (or  
from anything else).  MIMEDefang alone is relatively lightweight and  
fast (for a Perl thingie).  However, SpamAssassin is a memory pig.   
Very fat pig (to put it politely).  The kind of a pig you would take  
to a farm show.  Like any other fat boy, it can't run fast either.   
Even if you are scanning only small emails with it (never attempt to  
pass something larger than 100-200k through SpamAssassin).   
SpamAssassin can easily take anywhere between 10 to 20 megs of memory  
just to process relatively small message (or even more, depending how  
large are emails that you pass through it).  When calculating how many  
slaves you allow MIMEDefang to start, keep this in mind and make sure  
you don't exceed the amount of physical memory in your server.  You do  
not want SpamAssassin processes fighting for memory.  It can kill your  
email server.  Imagine ten really fat pigs (the real live pigs, flash  
and blood, curly tails, covered in mud and flies, burping, farting and  
everything) stacked on your server.  You've got an idea.  If running  
on a slower machine, you'll also want to adjust timeouts in Sendmail  
and MIMEDefang to give SpamAssassin enough time to chew the message.

If using SpamAssassin from MIMEDefang, I usually set maximum number of  
requests per slave in MIMEDefang low.  Even if SpamAssassin allocates  
too much memory in some slave when processing some nasty email, the  
slave will be replaced by a new one relatively quickly and memory will  
be released back to the system.  It is kind of a tradeoff setting.   
You don't want slaves being terminated and resetarted too often.  And  
you don't want fat ones lingering around for too long either.

You'd also tell MIMEDefang to use embeded Perl, so the Perl  
initializes only once.  This can help performance dramatically.

When using SpamAssassin, you might also want to look into Sendmail's  
throttle and alike option(s).  What it does is that it controlls the  
maximum number of connections per second.  If the number is exceeded,  
it will simply postpone the processing of incomming connection (it  
will not reject anything, just postpone the handling of incomming  
connection).  However, when using this option, if you experience the  
storm of incomming connections, the postponing might cause the clients  
to time out (howerver, if it gets that bad that clients start timing  
out, you are most likely under the DoS attack anyhow).  Older  
Sendmails have only global throttle option, and you can specify only  
integer amount of connections per second.  Newer Sendmails can do it  
per-IP address (which helps if you are dealing with only single bad  
host), and you can specify number of messages per time interval (so  
you'r more flexible here too).  This might help prevent having too  
many SpamAssassin scans occuring simultaniously and fighting for CPU  
and memory.

NOTICE: If you are not intended recipient, you are hereby notified
that by reading this message you agreed not to disturb frogs during
mating season.  For more info, visit http://www.8-P.ca/