I have no idea as to how to size an email server. I was approached by a customer that wanted a single server with RAID 1 disks to handle about 5 million emails a day. In general terms, what parameters should I take into account to size the hardware specs when the average email is about 10kb, the smalles email is 2kb and the largest email is about 5meg (with attachment)
thanks,
Erick Perez wrote:
I have no idea as to how to size an email server. I was approached by a customer that wanted a single server with RAID 1 disks to handle about 5 million emails a day. In general terms, what parameters should I take into account to size the hardware specs when the average email is about 10kb, the smalles email is 2kb and the largest email is about 5meg (with attachment)
5 million emails a day? Would your customer be involved in marketing? I am not sure I want to give advice that will be used to aid a spammer.
I think I would do RAID5 over RAID1 since you'll get more spindles. I'd also consider multiple cores and LOTS of RAM.
Mike
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Erick Perez Sent: Thursday, August 23, 2007 9:52 PM To: CentOS mailing list Subject: [CentOS] How to size an email server to handle 5 million emails perday
I have no idea as to how to size an email server. I was approached by a customer that wanted a single server with RAID 1 disks to handle about 5 million emails a day. In general terms, what parameters should I take into account to size the hardware specs when the average email is about 10kb, the smalles email is 2kb and the largest email is about 5meg (with attachment)
thanks,
--
Erick Perez _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On second thought, are mails being delivered locally or are you relaying to Exchange (or similar)?
Mike
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Erick Perez Sent: Thursday, August 23, 2007 9:52 PM To: CentOS mailing list Subject: [CentOS] How to size an email server to handle 5 million emails perday
I have no idea as to how to size an email server. I was approached by a customer that wanted a single server with RAID 1 disks to handle about 5 million emails a day. In general terms, what parameters should I take into account to size the hardware specs when the average email is about 10kb, the smalles email is 2kb and the largest email is about 5meg (with attachment)
thanks,
--
Erick Perez _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Mike Kercher wrote:
On second thought, are mails being delivered locally or are you relaying to Exchange (or similar)?
It cannot possibly be used for local delivery. Do you have any idea what it takes to handle 5 million local deliveries daily?
I spent over three years managing a system that delivers more than 2 million emails and handles on average 200 million smtp transactions on a daily basis and you do not use a single box for this sort of thing.
Delivering 5 million emails daily with a single box has got to be an outgoing box.
Well, of the 5M, how many would be real emails? I handle over 1M on a quad xeon, but only a fraction of those are good.
Mike
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Feizhou Sent: Thursday, August 23, 2007 10:43 PM To: CentOS mailing list Subject: Re: [CentOS] How to size an email server to handle 5 million emailsperday
Mike Kercher wrote:
On second thought, are mails being delivered locally or are you relaying to Exchange (or similar)?
It cannot possibly be used for local delivery. Do you have any idea what it takes to handle 5 million local deliveries daily?
I spent over three years managing a system that delivers more than 2 million emails and handles on average 200 million smtp transactions on a daily basis and you do not use a single box for this sort of thing.
Delivering 5 million emails daily with a single box has got to be an outgoing box. _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Mike Kercher wrote:
Well, of the 5M, how many would be real emails? I handle over 1M on a quad xeon, but only a fraction of those are good.
Heh. Yeah, I count emails as stuff that will be delivered, stuff that will hit the queue. I guess my definitions have got in the way of this one.
On 8/23/07, Feizhou feizhou@graffiti.net wrote:
Mike Kercher wrote:
Well, of the 5M, how many would be real emails? I handle over 1M on a quad xeon, but only a fraction of those are good.
Heh. Yeah, I count emails as stuff that will be delivered, stuff that will hit the queue. I guess my definitions have got in the way of this one. _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Right, actually the box will run an AV engine with antispam then delivered to a ms exchange 2003 server (local lan), so no local mailbox is being used.
The barracuda spam firewall 400 appliance handles my specs but i cannot get info on what hardware they run, it's a 1u raid1, linux hardened presentation but no idea of the cpu or ram.
And no, it won't be used for marketing, it's inbound only.
Right, actually the box will run an AV engine with antispam then delivered to a ms exchange 2003 server (local lan), so no local mailbox is being used.
oh okay. Do you plan to build the system? (the software involved)
You needs will vary on how emails are dwelt with at the smtp level. How did you come by the 5 million figure? If you are really going to scan that many, you will need lots of cpu power besides really good disk i/o if you plan to use a single box to handle everything.
The barracuda spam firewall 400 appliance handles my specs but i cannot get info on what hardware they run, it's a 1u raid1, linux hardened presentation but no idea of the cpu or ram.
They most probably make heavy use of NVRAM or whatever they use for the memory cache of the RAID system. The service provider was once providing with two boxes from F5 for testing and its i/o was fantastic and they somehow used the RAID cache to do their guarantee of not losing any email under any circumstances.
And no, it won't be used for marketing, it's inbound only.
If you do get a barracuda, please be sure to bin crap that you do not reject at the smtp level. Otherwise, you will be marked as an outscatter 'spammer'.
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Erick Perez Sent: Thursday, August 23, 2007 11:59 PM To: CentOS mailing list Subject: Re: [CentOS] How to size an email server to handle 5 millionemailsperday
On 8/23/07, Feizhou feizhou@graffiti.net wrote:
Mike Kercher wrote:
Well, of the 5M, how many would be real emails? I handle
over 1M on a
quad xeon, but only a fraction of those are good.
Heh. Yeah, I count emails as stuff that will be delivered,
stuff that
will hit the queue. I guess my definitions have got in the
way of this one.
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Right, actually the box will run an AV engine with antispam then delivered to a ms exchange 2003 server (local lan), so no local mailbox is being used.
The barracuda spam firewall 400 appliance handles my specs but i cannot get info on what hardware they run, it's a 1u raid1, linux hardened presentation but no idea of the cpu or ram.
And no, it won't be used for marketing, it's inbound only.
I wouldn't try to do it. 5 mil on a single box is too much with AV and SA going. Look into getting a cluster of spam appliances, say Ironports going, or using a MX service to do it for you.
-Ross
______________________________________________________________________ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof.
--- Feizhou feizhou@graffiti.net wrote:
Mike Kercher wrote:
On second thought, are mails being delivered
locally or are you relaying
to Exchange (or similar)?
It cannot possibly be used for local delivery. Do you have any idea what it takes to handle 5 million local deliveries daily?
I spent over three years managing a system that delivers more than 2 million emails and handles on average 200 million smtp transactions on a daily basis and you do not use a single box for this sort of thing.
Delivering 5 million emails daily with a single box has got to be an outgoing box. _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
are you trying to say spam bot? ;-)
Steven
Get your Art Supplies @ www.littleartstore.com
Erick Perez wrote:
I have no idea as to how to size an email server. I was approached by a customer that wanted a single server with RAID 1 disks to handle about 5 million emails a day. In general terms, what parameters should I take into account to size the hardware specs when the average email is about 10kb, the smalles email is 2kb and the largest email is about 5meg (with attachment)
you need to define what you mean by 'handle' email. that could mean anything from a sending 1000s of copies of the same message over and over to a email hosting server at a corporation with 5000 busy users with sox requirements.
On 8/24/07, Erick Perez eaperezh@gmail.com wrote:
I have no idea as to how to size an email server. I was approached by a customer that wanted a single server with RAID 1 disks to handle about 5 million emails a day.
pls see below URL. It may help you.
http://wiki.mailscanner.info/doku.php?id=maq:index#setup_examples
Erick Perez wrote on Thu, 23 Aug 2007 21:52:20 -0500:
5 million emails a day
so, 4 million of that are spam? The ressources you need depend very much on how you deal with spam. If you take good measures at MTA level the system backend with SA etc. has only to deal with 1 million, otherwise it has to deal with five times of that. Quite a difference.
Kai
Kai Schaetzl napsal(a):
Erick Perez wrote on Thu, 23 Aug 2007 21:52:20 -0500:
5 million emails a day
so, 4 million of that are spam? The ressources you need depend very much on how you deal with spam. If you take good measures at MTA level the system backend with SA etc. has only to deal with 1 million, otherwise it has to deal with five times of that. Quite a difference.
Kai
Some hints: - if there are 4 Mil of spam consider greylisting, http://postgrey.schweikert.ch/ helps a lot SA - use local DNS cache David Hrbáč
Erick Perez wrote:
I have no idea as to how to size an email server. I was approached by a customer that wanted a single server with RAID 1 disks to handle about 5 million emails a day. In general terms, what parameters should I take into account to size the hardware specs when the average email is about 10kb, the smalles email is 2kb and the largest email is about 5meg (with attachment)
thanks,
I don't know if you have done it yet, but Fort System's new offering, BarricadeMX, could help you cut spam.
http://www.fsl.com/barricademx.html
It is closed-source, but FSL gives a lot to open-source communities, especially MailScanner's.
Ugo
I can only talk from experience; we are currently doing spam and anti- virus checks in our inbound flow of around 600,000 messages per day. To do this we have three inbound SMTP gateways running Sophos Puremessage with Sendmail as the MTA.. These systems are quad proc systems with 6 to 8 GB of ram. This is still not enough to handle the inbound flow efficiently at our organization.
We are currently looking into Ironport, which should be able to handle our entire inbound and outbound flow on one system. They say that they have the ability to drop around 98% of traffic that is coming in using reputation filtering, anti-spam checks and anti-virus checks. We have been demoing the device for a couple of months and I am really happy with it, it has been doing what was promised.
Josh G.
On Jan 3, 2008, at 9:21 AM, Ugo Bellavance wrote:
Erick Perez wrote:
I have no idea as to how to size an email server. I was approached by a customer that wanted a single server with RAID 1 disks to handle about 5 million emails a day. In general terms, what parameters should I take into account to size the hardware specs when the average email is about 10kb, the smalles email is 2kb and the largest email is about 5meg (with attachment) thanks,
I don't know if you have done it yet, but Fort System's new offering, BarricadeMX, could help you cut spam.
http://www.fsl.com/barricademx.html
It is closed-source, but FSL gives a lot to open-source communities, especially MailScanner's.
Ugo
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On Thu, Jan 03, 2008, Joshua Gimer wrote:
I can only talk from experience; we are currently doing spam and anti- virus checks in our inbound flow of around 600,000 messages per day. To do this we have three inbound SMTP gateways running Sophos Puremessage with Sendmail as the MTA.. These systems are quad proc systems with 6 to 8 GB of ram. This is still not enough to handle the inbound flow efficiently at our organization.
We have a system that handles similar quantities of incoming mail with a single incoming MX server running postfix, amavisd, and clamav to do anti- virus checking only, passing clean messages to a cluster of five machines which do spamassassin checking and delivery into Maildir folders NFS mounted on a central machine using LDAP authentication on the cluster machines.
The incoming MX server has an Intel(R) Pentium(R) 4 CPU 3.20GHz with 2GB RAM running SLES9, and rarely has a load average above 1.00.
The cluster servers have similar processors with 1GB RAM, running SLES9 and SLES10 (new ones will be CentOS :-).
The main file server that has all the home directories is rather ancient by comparison, running SuSE 9.2 Pro on an Intel(R) Pentium(R) 4 CPU 3.00GHz with 2GB RAM and lots of hard disk space.
We are currently looking into Ironport, which should be able to handle our entire inbound and outbound flow on one system. They say that they have the ability to drop around 98% of traffic that is coming in using reputation filtering, anti-spam checks and anti-virus checks. We have been demoing the device for a couple of months and I am really happy with it, it has been doing what was promised.
The border server rejects several million attempts a day using a combination of DNSRBLs, and other checks. It also has no users, accepting mail for valid users with rather large postfix virtual tables that map all incoming addresses to the internal servers.
I like this distributed architecture as all the machines in the cluster are pretty much vanilla boxes that are easily built and replaced if necessary. The only machine that's critical is the one containing all the user's home directories. Even that one has been replaced with a new machine with minimal down time by bringing up a replacement, syncing the users from the old machine to the new one, doing a bit of DNS editing to point to the new machine, then rsync'ing the user's Maildir folders as new mail is delivered to the new machine. Each of the cluster machines needs to remount the home directories with the new DNS. We were able to make the switch with less than 15 minutes of down time while making the DNS changes and remounting cluster machines. It took about an hour to complete the home rsyncs with about 10,000 users.
Even considering the relatively puny public MX server, it would be able to handle quite a bit more mail easily. The cluster machines scale close to linearly. They're also running on a 10/100 switch, and going to a gigabit switch should speed up mail delivery.
Bill -- INTERNET: bill@celestial.com Bill Campbell; Celestial Software LLC URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way FAX: (206) 232-9186 Mercer Island, WA 98040-0820; (206) 236-1676
Government is actually the worst failure of civilized man. There has never been a really good one, and even those that are most tolerable are arbitrary, cruel, grasping and unintelligent. -- H. L. Mencken
Bill Campbell wrote:
On Thu, Jan 03, 2008, Joshua Gimer wrote:
I can only talk from experience; we are currently doing spam and anti- virus checks in our inbound flow of around 600,000 messages per day. To do this we have three inbound SMTP gateways running Sophos Puremessage with Sendmail as the MTA.. These systems are quad proc systems with 6 to 8 GB of ram. This is still not enough to handle the inbound flow efficiently at our organization.
We have a system that handles similar quantities of incoming mail with a single incoming MX server running postfix, amavisd, and clamav to do anti- virus checking only, passing clean messages to a cluster of five machines which do spamassassin checking and delivery into Maildir folders NFS mounted on a central machine using LDAP authentication on the cluster machines.
I wonder if you have tried postfix + clamav via clamav-milter in any testing for potential system upgrades?
-
Bill Campbell -
Christopher Chan -
David Hrbáč -
Erick Perez -
Feizhou -
Indunil Jayasooriya -
Jean-Yves Avenard -
John R Pierce -
Joshua Gimer -
Kai Schaetzl -
Mike Kercher -
Ross S. W. Walker -
Steven Vishoot -
Ugo Bellavance