On Fri, Nov 30, 2007, Christopher Chan wrote: >Bill Campbell wrote: >>On Thu, Nov 29, 2007, Rodrigo Barbosa wrote: >>>On Wed, Nov 28, 2007 at 08:51:25AM -0800, Bill Campbell wrote: >>>>We haven't had any notable performance problems using this at a regional >>>>ISP customer's site with about 10,000 e-mail users and several machines >>>>in >>>>a cluster delivering mail to Maildir folders that are NFS mounted to the >>>>central server. >>>I've been using ext3 on server with 20000+ boxes for quite some time now, >>>without any performance problems. >>> >>>I'm using the same kind of setup you use. Cluster, Maildir, NFS. >>> >>>Works quite nice, doesn't it ? >> >>Very. We have a single Linux box facing the Internet which >>runs everything through postfix, amavisd, and clamav to weed out >>the phishing and worms that attack the Microsoft virus, Windows, >>then hands off messages that pass to the internal cluster using >>round-robin DNS as the poor-mans load balancer. This box runs >>with a load average less than 1.00 most of the time, rejects >>close to 2 million messages a day on IP related tests, passing >>about a half-million through to the internal servers which do the >>spamassassin checking and delivery to the user's mail stores. >> > >What processing and i/o power do you have on that box and how much RAM? >For the front end boxes, I had about 20-30 dual PIII 800Mhz boxes with >two SCSI disks and 1GB worth of RAM. They reject close to 180 million >messages based on access and ip rules, header and body checks (so >nothing cpu heavy) and they pass on about 3 million for routing or >further processing. The border MX machine is running a Intel(R) Pentium(R) 4 CPU 3.20GHz, seen as two processors in /proc/cpuinfo with 6389.76 bogomips. It has 2GB RAM, and currently has a load average of 0.24 reported by top. The hard drive is a 40GB WDC WD400JD-19LS SATA which isn't anything special by any means. It's running SLES9, installed in February 2006. Uptime is only 356 days as it had to be rebooted to move things around in the rack. The machines handling mail deliver in the cluster vary. The first one I checked has an Intel(R) Celeron(R) CPU 2.66GHz with 1GB of RAM. These too have pretty vanilla SATA drives. The main server with the home directories has an Intel(R) Pentium(R) 4 CPU 3.00GHz with no SMP, 2GB RAM, and several SATA drives. The border MX isn't beginning to breath hard handling the IP access rules, postfix, amavisd, and clamav. We have seen very even distribution amongst the delivery machines in the cluster using nothing more for load balancing than dnscache from djbdns for a single hostname on the private internal 10/100 LAN. The attached image shows the size of the mail queues on each of the 4 machines every fifteen minutes since midnight yesterday. This peaks shortly after midnight when daily security scans and other maintenance jobs are running. The load averages on these cluster machines rarely gets over 1.00. The primary limiting factor seems to be the time spamassassin takes to process messages. This is typically measured in seconds per message on commodity hardware. Bill -- INTERNET: bill at celestial.com Bill Campbell; Celestial Software LLC URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way FAX: (206) 232-9186 Mercer Island, WA 98040-0820; (206) 236-1676 Do not meddle in the affairs of dragons, for you are crunchy and taste good with ketchup. -------------- next part -------------- A non-text attachment was scrubbed... Name: qstats.png Type: image/png Size: 5666 bytes Desc: not available URL: <http://lists.centos.org/pipermail/centos/attachments/20071129/09abb601/attachment-0005.png>
