[CentOS] Re: How to size an email server to handle 5 million emails per day

[Bill Campbell]

On Thu, Jan 03, 2008, Joshua Gimer wrote:
>I can only talk from experience; we are currently doing spam and anti- 
>virus checks in our inbound flow of around 600,000 messages per day.  
>To do this we have three inbound SMTP gateways running Sophos  
>Puremessage with Sendmail as the MTA.. These systems are quad proc  
>systems with 6 to 8 GB of ram. This is still not enough to handle the  
>inbound flow efficiently at our organization.

We have a system that handles similar quantities of incoming mail with a
single incoming MX server running postfix, amavisd, and clamav to do anti-
virus checking only, passing clean messages to a cluster of five machines
which do spamassassin checking and delivery into Maildir folders NFS
mounted on a central machine using LDAP authentication on the cluster
machines.

The incoming MX server has an Intel(R) Pentium(R) 4 CPU 3.20GHz with 2GB
RAM running SLES9, and rarely has a load average above 1.00.

The cluster servers have similar processors with 1GB RAM, running SLES9 and
SLES10 (new ones will be CentOS :-).

The main file server that has all the home directories is rather ancient by
comparison, running SuSE 9.2 Pro on an Intel(R) Pentium(R) 4 CPU 3.00GHz
with 2GB RAM and lots of hard disk space.

>We are currently looking into Ironport, which should be able to handle  
>our entire inbound and outbound flow on one system. They say that they  
>have the ability to drop around 98% of traffic that is coming in using  
>reputation filtering, anti-spam checks and anti-virus checks. We have  
>been demoing the device for a couple of months and I am really happy  
>with it, it has been doing what was promised.

The border server rejects several million attempts a day using a
combination of DNSRBLs, and other checks.  It also has no users, accepting
mail for valid users with rather large postfix virtual tables that map all
incoming addresses to the internal servers.

I like this distributed architecture as all the machines in the cluster are
pretty much vanilla boxes that are easily built and replaced if necessary.
The only machine that's critical is the one containing all the user's home
directories.  Even that one has been replaced with a new machine with
minimal down time by bringing up a replacement, syncing the users from the
old machine to the new one, doing a bit of DNS editing to point to the new
machine, then rsync'ing the user's Maildir folders as new mail is delivered
to the new machine.  Each of the cluster machines needs to remount the home
directories with the new DNS.  We were able to make the switch with less
than 15 minutes of down time while making the DNS changes and remounting
cluster machines.  It took about an hour to complete the home rsyncs with
about 10,000 users.

Even considering the relatively puny public MX server, it would be able to
handle quite a bit more mail easily.  The cluster machines scale close to
linearly.  They're also running on a 10/100 switch, and going to a gigabit
switch should speed up mail delivery.

Bill
--
INTERNET:   bill at celestial.com  Bill Campbell; Celestial Software LLC
URL: http://www.celestial.com/  PO Box 820; 6641 E. Mercer Way
FAX:            (206) 232-9186  Mercer Island, WA 98040-0820; (206) 236-1676

Government is actually the worst failure of civilized man. There has
never been a really good one, and even those that are most tolerable
are arbitrary, cruel, grasping and unintelligent.  -- H. L. Mencken