Hi Everyone,
I'm running a Postfix+Dovecot+SpamAssassin box in a DMZ. Everything is honkey dorey. Lately I've been thinking about moving Dovecot (for IMAP) into the internal network - I'd rather not store my mail on the CentOS 4 host in the DMZ.
Not having done this before, I'm not quite sure what options I have. Also, I don't know if this is a good idea at all. So:
1. Should I just leave mail storage on the same box in the DMZ? 2. If the answer to 1 is no, what's the best way to get mail from the SMTP server in the DMZ to an IMAP server in the internal network? Here's what I've briefly considered:
DMZ Postfix+SpamAssassin -> Internal Postfix+Dovecot DMZ Postfix+SpamAssassin -> Internal Fetchmail+Dovecot
3. Any tutorials for this out there, or even articles, etc., discussing using Postfix as a gateway? So far, I haven't found any that I've liked.
Thanks in advance for any help.
Regards,
Ranbir
Kanwar Ranbir Sandhu wrote:
Hi Everyone,
I'm running a Postfix+Dovecot+SpamAssassin box in a DMZ. Everything is honkey dorey. Lately I've been thinking about moving Dovecot (for IMAP) into the internal network - I'd rather not store my mail on the CentOS 4 host in the DMZ.
Why?
Not having done this before, I'm not quite sure what options I have. Also, I don't know if this is a good idea at all. So:
- Should I just leave mail storage on the same box in the DMZ?
Tell us why you want to move first.
- If the answer to 1 is no, what's the best way to get mail from the
SMTP server in the DMZ to an IMAP server in the internal network? Here's what I've briefly considered:
DMZ Postfix+SpamAssassin -> Internal Postfix+Dovecot DMZ Postfix+SpamAssassin -> Internal Fetchmail+Dovecot
Forget this. Answer the question first and then you can come to this after you also consider your hardware.
- Any tutorials for this out there, or even articles, etc., discussing
using Postfix as a gateway? So far, I haven't found any that I've liked.
...
It is a little bit involved. But first answer the question of why you want to move before we explore this.
Feizhou wrote:
Kanwar Ranbir Sandhu wrote:
Lately I've been thinking about moving Dovecot (for IMAP) into the internal network - I'd rather not store my mail on the CentOS 4 host in the DMZ.
Why?
Because you don't want to have sensitive data in the demilitarized zone? I know that I don't want to.
- If the answer to 1 is no, what's the best way to get mail from the
SMTP server in the DMZ to an IMAP server in the internal network? Here's what I've briefly considered:
DMZ Postfix+SpamAssassin -> Internal Postfix+Dovecot DMZ Postfix+SpamAssassin -> Internal Fetchmail+Dovecot
The first one. Pinch a hole in your firewall which *only* allows smtp from that *one* host to the internal host.
- Any tutorials for this out there, or even articles, etc., discussing
using Postfix as a gateway? So far, I haven't found any that I've liked.
Look at the relaydomains and the transports tables from postfix. Make sure that your domain isn't in $mydestinations. Make sure that your domain gets relayed (and transported) to the internal mailserver.
It is a little bit involved. But first answer the question of why you want to move before we explore this.
I wonder why that should be necessary - it's his decision, and I can really understand, why he's making it.
Ralph
Ralph Angenendt wrote:
Feizhou wrote:
Kanwar Ranbir Sandhu wrote:
Lately I've been thinking about moving Dovecot (for IMAP) into the internal network - I'd rather not store my mail on the CentOS 4 host in the DMZ.
Why?
Because you don't want to have sensitive data in the demilitarized zone? I know that I don't want to.
Well, if the mails are sensitive data then maybe he should consider having them all encrypted then rather than letting them flow around the Internet in plain text.
- If the answer to 1 is no, what's the best way to get mail from the
SMTP server in the DMZ to an IMAP server in the internal network? Here's what I've briefly considered:
DMZ Postfix+SpamAssassin -> Internal Postfix+Dovecot DMZ Postfix+SpamAssassin -> Internal Fetchmail+Dovecot
The first one. Pinch a hole in your firewall which *only* allows smtp from that *one* host to the internal host.
Yeah, if he does not have to serve his mails outside the office that should suffice.
- Any tutorials for this out there, or even articles, etc., discussing
using Postfix as a gateway? So far, I haven't found any that I've liked.
Look at the relaydomains and the transports tables from postfix. Make sure that your domain isn't in $mydestinations. Make sure that your domain gets relayed (and transported) to the internal mailserver.
I guess you are also going to teach him how to reject mails to non-existent users at the smtp level and not become an outscatter host.
It is a little bit involved. But first answer the question of why you want to move before we explore this.
I wonder why that should be necessary - it's his decision, and I can really understand, why he's making it.
I am glad that you can read his mind and learn about his environment.
Feizhou wrote:
Look at the relaydomains and the transports tables from postfix. Make sure that your domain isn't in $mydestinations. Make sure that your domain gets relayed (and transported) to the internal mailserver.
I guess you are also going to teach him how to reject mails to non-existent users at the smtp level and not become an outscatter host.
True, there is more to it. http://www.postfix.org/postconf.5.html#relay_recipient_maps for starters.
I wonder why that should be necessary - it's his decision, and I can really understand, why he's making it.
I am glad that you can read his mind and learn about his environment.
See >:)
Ralph
True, there is more to it.
I said it was a bit involved.
http://www.postfix.org/postconf.5.html#relay_recipient_maps for starters.
I guess he has enough information to pick his poison.
I wonder why that should be necessary - it's his decision, and I can really understand, why he's making it.
I am glad that you can read his mind and learn about his environment.
See >:)
=)
That's why I insisted on knowing his reason. There could have been other angles that he has not considered.
On Tue, 2006-22-08 at 13:48 +0800, Feizhou wrote:
I'm running a Postfix+Dovecot+SpamAssassin box in a DMZ. Everything is honkey dorey. Lately I've been thinking about moving Dovecot (for IMAP) into the internal network - I'd rather not store my mail on the CentOS 4 host in the DMZ.
Why?
Well, I'd rather keep all of my (I have a small consulting company) mail on the inside, making it harder to get to. So, I'd say it's to secure the mail storage some more.
I realize email is whizzing around the net in plain text (for the most part) and can be easily sniffed. I just think keeping mail in the internal network would be a little safer than leaving it on a Internet facing box.
Regards,
Ranbir
Well, I'd rather keep all of my (I have a small consulting company) mail on the inside, making it harder to get to. So, I'd say it's to secure the mail storage some more.
If you don't have to make it accessible outside then you just have to worry only about getting the mails in.
I realize email is whizzing around the net in plain text (for the most part) and can be easily sniffed. I just think keeping mail in the internal network would be a little safer than leaving it on a Internet facing box.
Of course and if you don't have to make it accessible on the Internet then by all means.
If you don't have an awful lot of email addresses you could just use a simple Berkeley DB table of them to prevent your mx box from becoming an outscatter box and saving bandwidth. All configuration will be done by editing files. It also means less integration work to do.
The alternatives are ldap, mysql or postgresql for storage of certain information such as authentication data, mail store path and either building and integrating the pieces of the system yourself or trying to integrate available pieces.
Quoting Kanwar Ranbir Sandhu m3freak@rogers.com:
- Should I just leave mail storage on the same box in the DMZ?
- If the answer to 1 is no, what's the best way to get mail from the
SMTP server in the DMZ to an IMAP server in the internal network? Here's what I've briefly considered:
The decision on having mail storage in the DMZ or not is up to you and depends on your actuall needs and security considerations (how sensitive is the content of the emails and how disciplined is the user population and/or 3rd parties in using encryption is just one thing to think about). I've read a previous response saying something along the lines "if emails are sensitive, encrypt them". Easy to say and explain to the tech person. Try it with non-tech people who don't even work at your company (since those would be emails stored on your server).
If you decide you want storage inside, here's couple of tips. Note that I'm mostly sendmail guy, so you'll have to find postfix equivalents yourself. Generally, I'd use SMTP to get emails from DMZ into internal network. Not a big fan of fetchmail for this kind of stuff. Fetchmail is nice tool for individual users. But not for this kind of stuff.
In the DMZ, make sure you accept email only for existing email addresses. Any rejections you make, you want to make on your border mail server. This includes non-existing email addresses, as well as rejecting spam and virus infected messages. It will also save you some bandwith, since (a) body of messages is not transmitted (non-existing users case) and (b) your border mail server doesn't need to generate delivery notifications.
You can do this in many ways. At least with sendmail. I'll describe some. I'm not saying they are the best. It all depends on your local configuration and preferences.
For example, you can configure border system to accept email for foobar.com. Than use virtusertables to map to some internal address so mails get pushed to the inside:
user@foobar.com user@internal.foobar.com @foobar.com error:nouser No such user here
Note that this will be rewriting envelope address, the one users don't see. The addresses in To/Cc/Whatever headers remain as it was.
On the inside system, you'd configure it to accept email for foobar.com and internal.foobar.com. This is to avoid sending internal mail to DMZ, and than having it come back inside. Than you can use virtusertable again (optional) to map addresses to user mailboxes:
user@foobar.com user user@internal.foobar.com user @foobar.com error:nouser No such user here @internal.foovar.com error:nouser No such user here
Another, maybe simpler, way to do it would be using LDAP mail routing. I've no idea if postfix can do this. That way, all the information needed for mail delivery is centralized in one place, and you don't need to keep information on what email addresses exist and what mailboxes they correspond to on both internal and external server.
Basically, you'd use LDAP to store information where the hack user's mailbox is. You would set mailHost attribute to point to your internal email server. You would not set mailRoutingAddress attribute. This would cause your external mail server to forward all email for existing email addresses to internal host. Your internal host will figure out that mailHost points to itself, and deliver email to the mailbox. So you don't need to rewrite email addresses like when using virtusertables. There's a lot of options when configuring LDAP routing, so if you go that way, best is to first read and fully understand documentation. Or you'll get unexpected results and will be generally dissapointed.
Now, the remaining problem is, what to do for people who want to access their email from outside. You probably don't want to allow direct POP3/IMAP connections from outside to your internal mail server. You may consider here several options. Webmail would be very nice approach in many cases. If you have lots of roaming laptop users that insist on using their favorite email client from home or when on road, you might consider setting VPN for them. It kind of adds to the complexity. Especially if you don't need VPN for other stuff. On the other hand, if you already have VPN, than you have the solution for accessing email from outside, right? Another solution might be setting IMAP proxy in the DMZ. But it is almost as allowing direct connections from the outside. So I'd leave it as last resort.
It's kind of longer answer. Just giving you couple of hints. At the end, you might find some solution that better fits your needs. But at least it will give you couple of ideas to explore.
Another, maybe simpler, way to do it would be using LDAP mail routing. I've no idea if postfix can do this. That way, all the information needed for mail delivery is centralized in one place, and you don't need to keep information on what email addresses exist and what mailboxes they correspond to on both internal and external server.
postfix can do ldap, mysql and pgsql. I, for one, install postfix 2.2 over RHEL's postfix package and disable all updates for postfix.
Basically, you'd use LDAP to store information where the hack user's mailbox is. You would set mailHost attribute to point to your internal email server. You would not set mailRoutingAddress attribute. This would cause your external mail server to forward all email for existing email addresses to internal host. Your internal host will figure out that mailHost points to itself, and deliver email to the mailbox. So you don't need to rewrite email addresses like when using virtusertables. There's a lot of options when configuring LDAP routing, so if you go that way, best is to first read and fully understand documentation. Or you'll get unexpected results and will be generally dissapointed.
postfix is a bit more involved. You have to use the right maps...like the mx postfix should use relay_domains and relay_recipient_maps if there is no address rewriting and the mail store postfix needs to use virtual_mailbox_domains and virtual_mailbox_maps (or maybe not needed...since the mx postfix should have ensured the recipient exists) if you are interested in ditching sendmail.
Now, the remaining problem is, what to do for people who want to access their email from outside. You probably don't want to allow direct POP3/IMAP connections from outside to your internal mail server. You may consider here several options. Webmail would be very nice approach in many cases. If you have lots of roaming laptop users that insist on using their favorite email client from home or when on road, you might consider setting VPN for them. It kind of adds to the complexity. Especially if you don't need VPN for other stuff. On the other hand, if you already have VPN, than you have the solution for accessing email from outside, right? Another solution might be setting IMAP proxy in the DMZ. But it is almost as allowing direct connections from the outside. So I'd leave it as last resort.
Hence my question why did he want to move his emails which would have been followed by questions about whether he needs to grant access from outside to the mail store or not.
On Tue, 2006-08-22 at 00:50 -0400, Kanwar Ranbir Sandhu wrote:
Hi Everyone,
I'm running a Postfix+Dovecot+SpamAssassin box in a DMZ. Everything is honkey dorey. Lately I've been thinking about moving Dovecot (for IMAP) into the internal network - I'd rather not store my mail on the CentOS 4 host in the DMZ.
Not having done this before, I'm not quite sure what options I have. Also, I don't know if this is a good idea at all. So:
- Should I just leave mail storage on the same box in the DMZ?
- If the answer to 1 is no, what's the best way to get mail from the
SMTP server in the DMZ to an IMAP server in the internal network? Here's what I've briefly considered:
A simple solution if you have an extra machine.. install qmail on a new box... put it into your DMZ to collect mail. You then set a simple smtproute to forward all mail to your inner mail server's ip.
There are no user accounts/passwords on the DMZ mail gateway and no mail stored (sensitive data) on the DMZ mail gateway machine.
It simply accepts all email for your domain, and simply forwards it through the DMZ pinhole to your internal mail server. If you want you could also have it handle antivirus, spam and rblsmtpd listing.
A simple solution if you have an extra machine.. install qmail on a new box... put it into your DMZ to collect mail. You then set a simple smtproute to forward all mail to your inner mail server's ip.
qmail is secure, bug free and the programs are efficient but it needs updating.
There are no user accounts/passwords on the DMZ mail gateway and no mail stored (sensitive data) on the DMZ mail gateway machine.
It simply accepts all email for your domain, and simply forwards it through the DMZ pinhole to your internal mail server. If you want you could also have it handle antivirus, spam and rblsmtpd listing.
The prime recipe for an outscatter host.
You will have to patch qmail to get any form of recipient address checking to reject at the smtp level.
Queue management can become a nightmare. With your proposal, if some spammer stuffs the queue with a load of spam (send spam to qmail box, set sender address to spam victim and voila! almost filter proof spamming) you have to stop the queue manager to do any deletes.
qmail is the best choice for an outgoing mail queue in its current state. Or a second stage mta if you want to make use of its great dot-qmail delivery behaviour. But as an mx, it won't cut it with today's Internet.
Check out qpsmtpd and use it as an SMTP relay or even run it on your public MX sever. You could even run it under apache if you'd like.
http://www.oreillynet.com/pub/a/sysadmin/2005/09/15/qpsmtpd.html
peter
Peter Eisch wrote:
Check out qpsmtpd and use it as an SMTP relay or even run it on your public MX sever. You could even run it under apache if you'd like.
http://www.oreillynet.com/pub/a/sysadmin/2005/09/15/qpsmtpd.html
That thing is perl based but I guess for a small site it won't really matter. Performance is not an issue.
On 8/22/06 10:54 PM, "Feizhou" feizhou@graffiti.net wrote:
Peter Eisch wrote:
Check out qpsmtpd and use it as an SMTP relay or even run it on your public MX sever. You could even run it under apache if you'd like.
http://www.oreillynet.com/pub/a/sysadmin/2005/09/15/qpsmtpd.html
That thing is perl based but I guess for a small site it won't really matter. Performance is not an issue.
Yes, so long as you define "small site" as something less than 250,000 emails per day.
Jaymz Ringler wrote:
A simple solution if you have an extra machine.. install qmail on a new box... put it into your DMZ to collect mail. You then set a simple smtproute to forward all mail to your inner mail server's ip.
qmail is outdated. qmail doesn't check for available users before accepting a mail and thus always sends bounces. qmail isn't available for centos. qmail hasn't seen any updates since way back when. qmail can only be made into a more modern MTA with tons of patches.
Ralph
On Wed, 2006-08-23 at 04:33, Ralph Angenendt wrote:
Jaymz Ringler wrote:
A simple solution if you have an extra machine.. install qmail on a new box... put it into your DMZ to collect mail. You then set a simple smtproute to forward all mail to your inner mail server's ip.
qmail is outdated. qmail doesn't check for available users before accepting a mail and thus always sends bounces. qmail isn't available for centos. qmail hasn't seen any updates since way back when. qmail can only be made into a more modern MTA with tons of patches.
On the other hand a very nice approach to an outside DMZ relay is to use sendmail with MimeDefang (http://www.mimedefang.org) to coordinate scanning with clamav, spamassassin, etc. It has an option to check for valid users via smtp to the delivery server so you don't have to set up anything extra for this other than the snippet of perl that controls it all. There is a moderately active mail list with helpful advice.
Les Mikesell wrote:
On Wed, 2006-08-23 at 04:33, Ralph Angenendt wrote:
Jaymz Ringler wrote:
A simple solution if you have an extra machine.. install qmail on a new box... put it into your DMZ to collect mail. You then set a simple smtproute to forward all mail to your inner mail server's ip.
qmail is outdated. qmail doesn't check for available users before accepting a mail and thus always sends bounces. qmail isn't available for centos. qmail hasn't seen any updates since way back when. qmail can only be made into a more modern MTA with tons of patches.
On the other hand a very nice approach to an outside DMZ relay is to use sendmail with MimeDefang (http://www.mimedefang.org) to coordinate scanning with clamav, spamassassin, etc. It has an option to check for valid users via smtp to the delivery server so you don't have to set up anything extra for this other than the snippet of perl that controls it all. There is a moderately active mail list with helpful advice.
i'd rather use postfix in tandem with clamav and spamassassin. sendmail sucks. milter sucks.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Thu, Aug 24, 2006 at 11:27:50AM +0800, Feizhou wrote:
On the other hand a very nice approach to an outside DMZ relay is to use sendmail with MimeDefang (http://www.mimedefang.org) to coordinate scanning with clamav, spamassassin, etc. It has an option to check for valid users via smtp to the delivery server so you don't have to set up anything extra for this other than the snippet of perl that controls it all. There is a moderately active mail list with helpful advice.
i'd rather use postfix in tandem with clamav and spamassassin. sendmail sucks. milter sucks.
YEAH ! Flamewar ! Yohooooooooooooo!
i'd rather use exim in tandem with clamav and spamassassin. postfix sucks.
But only when editing the configuration file with VI. Emacs sucks.
But only when running on Linux. FreeBSD sucks.
But only when using AMD processors. Intel sucks.
But only on ext3 filesystems. Reiserfs sucks.
But only when the server is located on earth. Mars sucks.
Did I miss anything ?
- -- Rodrigo Barbosa "Quid quid Latine dictum sit, altum viditur" "Be excellent to each other ..." - Bill & Ted (Wyld Stallyns)
Rodrigo Barbosa wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Thu, Aug 24, 2006 at 11:27:50AM +0800, Feizhou wrote:
On the other hand a very nice approach to an outside DMZ relay is to use sendmail with MimeDefang (http://www.mimedefang.org) to coordinate scanning with clamav, spamassassin, etc. It has an option to check for valid users via smtp to the delivery server so you don't have to set up anything extra for this other than the snippet of perl that controls it all. There is a moderately active mail list with helpful advice.
i'd rather use postfix in tandem with clamav and spamassassin. sendmail sucks. milter sucks.
YEAH ! Flamewar ! Yohooooooooooooo!
haha. well, i guess having had to immediately upgrade more than a dozen boxes running sendmail whenever a remote exploit was found kind of made me rather unwilling to deal with it anymore after the peace and quiet I get from running postfix. Also, not having to decipher them sendmail rulesets and create new ones were a relief. Then there was the instability of milter to contend with...
I guess that is all long past now is it?
But only when editing the configuration file with VI. Emacs sucks.
/rotfl
But only when running on Linux. FreeBSD sucks.
Still? Haven't they improved performance yet? ;)
But only when using AMD processors. Intel sucks.
YEAH! :D
But only on ext3 filesystems. Reiserfs sucks.
...it would appear all Linux fs suck :(. I haven't tried JFS though.
But only when the server is located on earth. Mars sucks.
But you get super cooling there ;)
Did I miss anything ?
OpenBSD and Solaris. :D
On Wed, 2006-08-23 at 23:22, Feizhou wrote:
YEAH ! Flamewar ! Yohooooooooooooo!
haha. well, i guess having had to immediately upgrade more than a dozen boxes running sendmail whenever a remote exploit was found kind of made me rather unwilling to deal with it anymore after the peace and quiet I get from running postfix. Also, not having to decipher them sendmail rulesets and create new ones were a relief. Then there was the instability of milter to contend with...
I guess that is all long past now is it?
Unless typing 'yum update' to pick up fixes that have been needed less often than in the Linux kernel for the last few years is a problem for you...
No one edits sendmail.cf directly anymore and milter has been stable for about as long as postfix has existed at all. Postfix still doesn't have a way to let you hook user defined scanners running under a different uid to run in realtime during the smtp conversation, does it? MimeDefang lets you do anything you can describe in perl and return the response through the milter interface for various operations before the mail has been acknowledged as accepted.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Wed, Aug 23, 2006 at 11:38:14PM -0500, Les Mikesell wrote:
On Wed, 2006-08-23 at 23:22, Feizhou wrote:
I guess that is all long past now is it?
No one edits sendmail.cf directly anymore and milter has been stable for about as long as postfix has existed at all. Postfix still doesn't have a way to let you hook user defined scanners running under a different uid to run in realtime during the smtp conversation, does it? MimeDefang lets you do anything you can describe in perl and return the response through the milter interface for various operations before the mail has been acknowledged as accepted.
Well, this is obviously just my personal option, but:
sendmail - No longer the security nightmare it used to be, but it is the slower of the bunch.
postfix - The easiest to configure. Has enough features and speed to keep most people happy.
exim - Fast and powerful. Lots of features and highly configurable. But ONLY if you know very well what you are going. Not as hard as sendmail to configure (think sendmail.cf), but much harder than postfix. Expecially recomended for people (like me) that enjoy micromanaging and customizing every small detail. ACL features are a god given.
qmail - (Take with a grain of salt, since I dislike dbj) A pain. It is fast and secure, as long as you use the stock package. But you will need to include several external packages to get even basic functionality, and everything about it (except SMTP protocol itself) is non standard, and the configuration is spread between a lot of files and directories. But it really is fast (slightly faster than exim in some cases, about the same in others).
As you can see, I use Exim so, again, take another grain of salt. My current exim setup is, as far as I know, impossible to reproduce with any other MTA, unless you use several external hacks (multiple instances with different configurations, with port redirection, to name just one).
But most of the time, when recomending a MTA to others, I say postfix.
- -- Rodrigo Barbosa "Quid quid Latine dictum sit, altum viditur" "Be excellent to each other ..." - Bill & Ted (Wyld Stallyns)
sendmail - No longer the security nightmare it used to be, but it is the slower of the bunch.
are we talking about sendmail X? ;)
postfix - The easiest to configure. Has enough features and speed to keep most people happy.
hmm...i don't know...qmail appears to me to be the easiest of them all due to its utter simplicity (and lack of features/necessary behavior)
exim - Fast and powerful. Lots of features and highly configurable. But ONLY if you know very well what you are going. Not as hard as sendmail to configure (think sendmail.cf), but much harder than postfix. Expecially recomended for people (like me) that enjoy micromanaging and customizing every small detail. ACL features are a god given.
I must take a look at exim one of these days.
qmail - (Take with a grain of salt, since I dislike dbj) A pain. It is fast and secure, as long as you use the stock package. But you will need to include several external packages to get even basic functionality, and everything about it (except SMTP protocol itself) is non standard, and the configuration is spread between a lot of files and directories. But it really is fast (slightly faster than exim in some cases, about the same in others).
You could run qmail-smtpd out of xinetd...and run qmail without daemontools and log to syslog. But if you are into djb's stuff you do end up using the rest of his good stuff.
As you can see, I use Exim so, again, take another grain of salt. My current exim setup is, as far as I know, impossible to reproduce with any other MTA, unless you use several external hacks (multiple instances with different configurations, with port redirection, to name just one).
When you say multiple instances, I assume multiple queues too?
But most of the time, when recomending a MTA to others, I say postfix.
:D - qmail not suitable for today's Internet as an MX mta without patching and sendmail a pain to learn/understand if you are not familiar with m4
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Thu, Aug 24, 2006 at 03:03:39PM +0800, Feizhou wrote:
qmail - (Take with a grain of salt, since I dislike dbj) A pain. It is fast and secure, as long as you use the stock package. But you will need to include several external packages to get even basic functionality, and everything about it (except SMTP protocol itself) is non standard, and the configuration is spread between a lot of files and directories. But it really is fast (slightly faster than exim in some cases, about the same in others).
You could run qmail-smtpd out of xinetd...and run qmail without daemontools and log to syslog. But if you are into djb's stuff you do end up using the rest of his good stuff.
All of which has a definitive slackware flavor.
As you can see, I use Exim so, again, take another grain of salt. My current exim setup is, as far as I know, impossible to reproduce with any other MTA, unless you use several external hacks (multiple instances with different configurations, with port redirection, to name just one).
When you say multiple instances, I assume multiple queues too?
Has to be, unless you want your queues crashing into each other.
But most of the time, when recomending a MTA to others, I say postfix.
:D - qmail not suitable for today's Internet as an MX mta without patching and sendmail a pain to learn/understand if you are not familiar with m4
True, and exim, if not configured correctly, can be very slow.
- -- Rodrigo Barbosa "Quid quid Latine dictum sit, altum viditur" "Be excellent to each other ..." - Bill & Ted (Wyld Stallyns)
Les Mikesell wrote:
Postfix still doesn't have a way to let you hook user defined scanners running under a different uid to run in realtime during the smtp conversation, does it? MimeDefang
I just implemented a greylisting app called GPS with postfix on an embedded ARM board here, and it runs as "nobody" and is active during the smtp conversation.
You add something like this to /etc/postfix/master.cf and you're away:
policy unix - n n - - spawn user=nobody argv=/usr/bin/gps /etc/gps.conf
I realize often the skills one acquires managing a particular setup can outweigh moving to another platform even if it is better, but one look at the need for a Makefile to translate one incomprehensible config format into a config format that sends grown men insane convinced me to back the Postfix horse :-)
-Andy
On Thu, 2006-08-24 at 00:22, Andy Green wrote:
Les Mikesell wrote:
Postfix still doesn't have a way to let you hook user defined scanners running under a different uid to run in realtime during the smtp conversation, does it? MimeDefang
I just implemented a greylisting app called GPS with postfix on an embedded ARM board here, and it runs as "nobody" and is active during the smtp conversation.
You add something like this to /etc/postfix/master.cf and you're away:
policy unix - n n - - spawn user=nobody argv=/usr/bin/gps /etc/gps.conf
^^^^
Does that mean it starts a new process for every message? The milter interface chats over a socket to a long-running process so you don't have to initialize it every time. MimeDefang also multiplexes to several slaves that do the scanning work so you don't have to serialize everything either.
I realize often the skills one acquires managing a particular setup can outweigh moving to another platform even if it is better, but one look at the need for a Makefile to translate one incomprehensible config format into a config format that sends grown men insane convinced me to back the Postfix horse :-)
I'm still missing the 'better' part. And Makefiles have always been a useful way to automate repetitive tasks - even better when someone else has written them and embedded the execution in the program startup script.
Les Mikesell wrote:
policy unix - n n - - spawn user=nobody argv=/usr/bin/gps /etc/gps.conf
^^^^Does that mean it starts a new process for every message? The milter interface chats over a socket to a long-running process so you don't have to initialize it every time. MimeDefang also multiplexes to several slaves that do the scanning work so you don't have to serialize everything either.
I think spawn acts like an inittab entry, it hangs around itself and if the thing it spawned ever dies it cooks a new one. I believe that postfix is also using Unix sockets to communicate with external apps. Pass on the muxing aspect I only know enough to get gps working.
I realize often the skills one acquires managing a particular setup can outweigh moving to another platform even if it is better, but one look at the need for a Makefile to translate one incomprehensible config format into a config format that sends grown men insane convinced me to back the Postfix horse :-)
I'm still missing the 'better' part. And Makefiles have always
Fair enough, I did stick that cart before the horse.
been a useful way to automate repetitive tasks - even better when someone else has written them and embedded the execution in the program startup script.
That must be why so much other software manages their config by converting it into gibberish by build tools, instead of the naive and unprofessional method of parsing a single level of config straight.
-Andy
BTW just saw this fly by
Package : sendmail Vulnerability : programming error Problem type : remote Debian-specific: no CVE ID : CVE-2006-1173 CERT advisory : VU#146718 BugTraq ID : 18433 Debian Bug : 373801 380258
Frank Sheiness discovered that a MIME conversion routine in sendmail, a powerful, efficient, and scalable mail transport agent, could be tricked by a specially crafted mail to perform an endless recursion.
For the stable distribution (sarge) this problem has been fixed in version 8.13.4-3sarge2.
For the unstable distribution (sid) this problem has been fixed in version 8.13.7-1.
We recommend that you upgrade your sendmail package.
On Thu, 2006-08-24 at 01:42, Andy Green wrote:
been a useful way to automate repetitive tasks - even better when someone else has written them and embedded the execution in the program startup script.
That must be why so much other software manages their config by converting it into gibberish by build tools, instead of the naive and unprofessional method of parsing a single level of config straight.
Yes, at some point just about every piece of software you use had a Makefile create it with the options someone chose. It is the right way to do things. Sendmail just adds another level so you don't have to rebuild from source to get custom programming capability in case you want to do something that no one else has done before.
Les Mikesell wrote:
On Wed, 2006-08-23 at 23:22, Feizhou wrote:
YEAH ! Flamewar ! Yohooooooooooooo!
haha. well, i guess having had to immediately upgrade more than a dozen boxes running sendmail whenever a remote exploit was found kind of made me rather unwilling to deal with it anymore after the peace and quiet I get from running postfix. Also, not having to decipher them sendmail rulesets and create new ones were a relief. Then there was the instability of milter to contend with...
I guess that is all long past now is it?
Unless typing 'yum update' to pick up fixes that have been needed less often than in the Linux kernel for the last few years is a problem for you...
sorry, it was a patched sendmail and those boxes were FreeBSD then.
No one edits sendmail.cf directly anymore and milter has been stable for about as long as postfix has existed at all.
Oh yeah? Hit SPAM-L for a bunch of great sendmail admins that do just that and I contest milter being stable as long as postfix has existed since we had major problems with sendmail + milter all through 8.11.x to 8.12.10 and finally replaced it with postfix at my previous place of work.
Postfix still doesn't have a way to let you hook user defined scanners running under a different uid to run in realtime during the smtp conversation, does it? MimeDefang lets you do anything you can describe in perl and return the response through the milter interface for various operations before the mail has been acknowledged as accepted.
Okay, it does not provide a hook but it can be done via a smtp passthrough proxy. milter is coming soon.
postfix does provide content inspection before queueing via perl regex and if needed, you stuff the mail through smtp to a filter that can either run on the same box or run on another box under whatever uid you wish and return appropriate response before queueing the mail.
Let's get one thing straight. I have not used exim yet but I dare say that sendmail is the most flexible mta program available thanks to its ruleset feature. However, this power is limited to those who can think in sendmail rulesets and given your comment about nobody edits sendmail.cf anymore, I guess it shows how hard it is to get mastery of sendmail's power.
As for mimedefang, qmail lets you do anything that can be described in perl, shell, C, python, whatever you fancy in fact and reject at the smtp level too since you can replace qmail-queue or put a filter before qmail-queue.
I am sorry, but one can get the functionalily of sendmail sans the neverending list of security updates and that is on two other mta software.
On Thu, 2006-08-24 at 01:51, Feizhou wrote:
Let's get one thing straight. I have not used exim yet but I dare say that sendmail is the most flexible mta program available thanks to its ruleset feature. However, this power is limited to those who can think in sendmail rulesets and given your comment about nobody edits sendmail.cf anymore, I guess it shows how hard it is to get mastery of sendmail's power.
But you are missing the point that once something has been done for sendmail via the included m4 macros, no one else ever has to understand it again. You just edit a line in the .mc file to activate the feature/option following the comments in the file or some documentation and the right thing happens. As shipped in Centos you can do pretty much anything you would want a mailer to do by changing a few lines in sendmail.mc. It doesn't make any more sense to talk about the difficulty of understanding sendmail.cf than it does to talk about source code changes. It is nice that both are available for those who might want to tackle changes at that level but it is not necessary for ordinary use.
Then when you add MimeDefang, you also get the ability to add in any other operations you want to happen in parallel with the smtp chat and control it all with a bit of perl.
As for mimedefang, qmail lets you do anything that can be described in perl, shell, C, python, whatever you fancy in fact and reject at the smtp level too since you can replace qmail-queue or put a filter before qmail-queue.
Another way of saying that is that qmail is so bad you have to completely replace components to make it usable at all.
I am sorry, but one can get the functionalily of sendmail sans the neverending list of security updates and that is on two other mta software.
Sendmail is probably the most heavily audited code available today, and none of the other MTA+addons are as well integrated or designed for efficiency as sendmail+MimeDefang with its multiplexed pool of backend slaves. Qpsmtpd is promising but the project is still in the process of reinventing things MimeDefang has had down for years.
Sendmail vs. Postfix. Use whichever you like and don't bother telling the other campus that your choice is better. Sendmail fits some needs better. Postfix fits other needs better. That's why we still have both of them. So save some bandwith and continue using whatever works for you.
Hello Les,
But you are missing the point that once something has been done for sendmail via the included m4 macros, no one else ever has to understand it again. You just edit a line in the .mc file to activate the feature/option following the comments in the file or some documentation and the right thing happens. As shipped in Centos you can do pretty much anything you would want a mailer to do by changing a few lines in sendmail.mc.
Actually, what I want to get at is if you need anything beyond the rulesets that are provided, you cannot do anything unless you understand sendmail rulesets and that is the same if you need to change something too.
It doesn't make any more sense to talk about the difficulty of understanding sendmail.cf than it does to talk about source code changes. It is nice that both are available for those who might want to tackle changes at that level but it is not necessary for ordinary use.
I guess that is the whole point. Ordinary users will probably have no clue how sendmail works and when problems arise, good luck trying to fix or get around them. So unless they know how to build other stuff and use procmail or maildrop, they are pretty much stuck to system mailboxes.
Then when you add MimeDefang, you also get the ability to add in any other operations you want to happen in parallel with the smtp chat and control it all with a bit of perl.
Which makes it no different from your dissing of qmail. qmail provides very little that you can do before DATA in the smtp chat and mimedefang does not kick in until after SMTP DATA in sendmail and they are therefore the same.
As for mimedefang, qmail lets you do anything that can be described in perl, shell, C, python, whatever you fancy in fact and reject at the smtp level too since you can replace qmail-queue or put a filter before qmail-queue.
Another way of saying that is that qmail is so bad you have to completely replace components to make it usable at all.
Which is no different from sendmail + mimedefang. You have no idea how qmail works so I shall ignore your ignorance on this. sendmail gave you milter so that you can get at the headers and message body. DJB's multiple program approach to separate the different functions automatically provides you the ability to get at what you want by either modifying that particular qmail program or by replacing like qpsmtpd or by adding another program like qmail-qfilter. For this same reason, people don't get any trouble from postfix and qmail with regard to security issues and sendmail X following the same design principles says a lot.
I am sorry, but one can get the functionalily of sendmail sans the neverending list of security updates and that is on two other mta software.
Sendmail is probably the most heavily audited code available today, and none of the other MTA+addons are as well integrated or designed for efficiency as sendmail+MimeDefang with its multiplexed pool of backend slaves. Qpsmtpd is promising but the project is still in the process of reinventing things MimeDefang has had down for years.
yeah, sendmail is probably the most heavily audited code and people still find issues just exim also had/has issues due to their monolithic design. Ever wonder why sendmail X is following the footsteps of qmail and postfix, two mtas that were written by two security different experts?
Multiplexed pool of backend slave? sendmail + mysql anyone?
Let's see, postfix supports mysql, ldap and postgresql out of the box. qmail's design allows people to add mysql/postgresql/ldap support such that we have qmail-ldap and qmail-sql. So you can use qmail-ldap or qmail-sql instead of doing your own. exim also comes with mysql, ldap support out of the box.
methinks it is sendmail that is behind here regarding backend slaves. You need to add mysql table support and then you have to write the rulesets to be able to use those tables.
Well integrated and designed for efficiency eh? I'd like to see benchmarks between sendmail + mimedefang versus postfix + amavisd. In fact, I'd like to see sendmail + mimedefang integrated with a mysql backend versus postfix + amavisd integrated with a mysql backend. But the whole thing would be unfair since postfix does connection pooling to its backends while sendmail will probably beat the crap out of its backend if it supported any sql database at all.
You can diss all other mtas + their addons all you like Les, but sendmail X is following the design principles of qmail and postfix which says something.
On Thu, 2006-08-24 at 22:49, Feizhou wrote:
As shipped in Centos you can do pretty much anything you would want a mailer to do by changing a few lines in sendmail.mc.
Actually, what I want to get at is if you need anything beyond the rulesets that are provided, you cannot do anything unless you understand sendmail rulesets and that is the same if you need to change something too.
Yes, if you want to do something that no one else has needed to do in sendmail's 3-decade history, you'll probably have to read the bat book first. But that seems unlikely to me and with MimeDefang in the picture you can control many operations in your perl filter instead of modifying sendmail macros.
I guess that is the whole point. Ordinary users will probably have no clue how sendmail works and when problems arise, good luck trying to fix or get around them. So unless they know how to build other stuff and use procmail or maildrop, they are pretty much stuck to system mailboxes.
As shipped in Centos, procmail is used for local delivery.
Then when you add MimeDefang, you also get the ability to add in any other operations you want to happen in parallel with the smtp chat and control it all with a bit of perl.
Which makes it no different from your dissing of qmail. qmail provides very little that you can do before DATA in the smtp chat and mimedefang does not kick in until after SMTP DATA in sendmail and they are therefore the same.
I think you've been misinformed. My MimeDefang checks the inbound recipients via smtp to the final delivery server before DATA is ever mentioned. If there are no valid recipients (which the outside relay wouldn't know directly) the message is rejected before DATA.
As for mimedefang, qmail lets you do anything that can be described in perl, shell, C, python, whatever you fancy in fact and reject at the smtp level too since you can replace qmail-queue or put a filter before qmail-queue.
Another way of saying that is that qmail is so bad you have to completely replace components to make it usable at all.
Which is no different from sendmail + mimedefang. You have no idea how qmail works so I shall ignore your ignorance on this.
I've experienced the pain of using qmail. I know more than I want to know about how it accepts everything, then generates millions of bounces for the things it can't deliver then lets the bounce delivery attempts throttle your real deliverable outbound mail.
sendmail gave you milter so that you can get at the headers and message body. DJB's multiple program approach to separate the different functions
But he separates the functions so you can't do the thing you need at the time you need to do it...
automatically provides you the ability to get at what you want by either modifying that particular qmail program
Wait - you think modifying a program is easier than configuring one that works in the first place?
or by replacing like qpsmtpd or by adding another program like qmail-qfilter. For this same reason, people don't get any trouble from postfix and qmail with regard to security issues and sendmail X following the same design principles says a lot.
The milter interface is a more extreme solution, since it gives you separation of permissions for the parts on the other end of the socket but you get its response when you need it.
Well integrated and designed for efficiency eh? I'd like to see benchmarks between sendmail + mimedefang versus postfix + amavisd. In fact, I'd like to see sendmail + mimedefang integrated with a mysql backend versus postfix + amavisd integrated with a mysql backend. But the whole thing would be unfair since postfix does connection pooling to its backends while sendmail will probably beat the crap out of its backend if it supported any sql database at all.
If you run spamassassin, all of the other operations are pretty much meaningless time-wise. The piece you need to keep things going is a multiplexer that allows some maximum number of backend scanners and some larger number of other MTA operations to run concurrently. That is, you don't want to serialize everything around a few scanners, and you also don't want to be allowed to spawn off enough instances of them to kill the machine. I don't think any of the other systems understand this concept - but I'd be happy to hear that I'm wrong about that.
You can diss all other mtas + their addons all you like Les, but sendmail X is following the design principles of qmail and postfix which says something.
The only one I'll really diss is qmail, and I'll concede that qpsmtpd will solve a few of it's problems as the project slowly re-invents the things that MimeDefang has been doing for years. My point is just that sendmail does a good job and has some unique advantages when used with MimeDefang.
Les Mikesell wrote:
You can diss all other mtas + their addons all you like Les, but sendmail X is following the design principles of qmail and postfix which says something.
The only one I'll really diss is qmail, and I'll concede that qpsmtpd will solve a few of it's problems as the project slowly re-invents the things that MimeDefang has been doing for years. My point is just that sendmail does a good job and has some unique advantages when used with MimeDefang.
I guess it comes down to this: if you are handling a ton of mail and mail is a big part of what you do, the rules are different. You can consider to look past the investment needed to get sendmail and qmail[1] to perform.
If you are handling relatively low volumes of mail, say the low tens of thousands a day, and "mail guy" is not a shout you respond to, then I strongly recommend not becoming a white-coated acolyte to these and to make the smaller brain-investment needed to get Postfix working great.
-Andy
[1] qmail's license used to be source distribution only, because that locked out anyone unable to compile it and its dependent packages and "killed the weak". Mail isn't that hard! Mortals can get Postfix going!
I guess it comes down to this: if you are handling a ton of mail and mail is a big part of what you do, the rules are different. You can consider to look past the investment needed to get sendmail and qmail[1] to perform.
or anything else :)
If you are handling relatively low volumes of mail, say the low tens of thousands a day, and "mail guy" is not a shout you respond to, then I strongly recommend not becoming a white-coated acolyte to these and to make the smaller brain-investment needed to get Postfix working great.
heh, so you don't recommend sendmail for a small site? I had trouble imagining using sendmail for anything else!
-Andy
[1] qmail's license used to be source distribution only, because that locked out anyone unable to compile it and its dependent packages and "killed the weak". Mail isn't that hard! Mortals can get Postfix going!
mail wasn't that hard i would say. Now, putting a mail server on the Internet means getting a handle on a whole lot of things.
On Fri, 2006-08-25 at 01:59, Andy Green wrote:
You can diss all other mtas + their addons all you like Les, but sendmail X is following the design principles of qmail and postfix which says something.
The only one I'll really diss is qmail, and I'll concede that qpsmtpd will solve a few of it's problems as the project slowly re-invents the things that MimeDefang has been doing for years. My point is just that sendmail does a good job and has some unique advantages when used with MimeDefang.
I guess it comes down to this: if you are handling a ton of mail and mail is a big part of what you do, the rules are different. You can consider to look past the investment needed to get sendmail and qmail[1] to perform.
If you are handling relatively low volumes of mail, say the low tens of thousands a day, and "mail guy" is not a shout you respond to, then I strongly recommend not becoming a white-coated acolyte to these and to make the smaller brain-investment needed to get Postfix working great.
Unfortunately the amount of real mail you intend to handle doesn't relate much to what can happen when you plug into the internet. Sites frequently get hammered with spam or virus 'dictionary attacks' where messages with near-random user names are sent by the millions to a domain. For the last few years this seems to be coordinated over a vast network of virus-compromized zombie machines in a way that can make the messages appear from many different IP addresses but at a controlled rate that doesn't completely swap the receiver. An unmodified qmail will accept all these messages then in a later operation realize that the recipient doesn't exist, construct a bounce message for each, and try to return them, generally to a forged sender address which may in fact be the real target of the scheme. You have a similar problem with other mailers if you try to accept internet mail through a DMZ relay that doesn't have the user base, forwarding it to a firewalled machine for delivery unless you work out some way to reject invalid users at the relay.
[1] qmail's license used to be source distribution only, because that locked out anyone unable to compile it and its dependent packages and "killed the weak". Mail isn't that hard! Mortals can get Postfix going!
That's not the worst part of the license. The real problem is that qmail as written has several logical flaws, the above-mentioned being the most obvious, and the license states that no one is allowed to distribute modified versions so it can't be fixed without completely replacing components.
[1] qmail's license used to be source distribution only, because that locked out anyone unable to compile it and its dependent packages and "killed the weak". Mail isn't that hard! Mortals can get Postfix going!
That's not the worst part of the license. The real problem is that qmail as written has several logical flaws, the above-mentioned being the most obvious, and the license states that no one is allowed to distribute modified versions so it can't be fixed without completely replacing components.
There is a slight workaround. One can distribute the original + source + patches and a script that does the patching/installing. Hence the netqmail-1.05 version.
Les, quit making false statements about qmail. qmail does not have any logical flaws otherwise Wietse would not follow the same design principles in postfix after his spat with DJB. qmail's problem is that the author has not done any updates to it since 1998 to handle the changed needs required of a MTA software since then. SMTP is broken which is why spammers and virus writers can create so much trouble. Both SMTP and qmail were designed in an environment very different from what we have now. That is why we have ESMTP and what not updates to the SMTP protocol.
qmail is a fine piece of software and it also introduced the maildir format which is now widely supported and used by anyone who cares about the integrity of their mailboxes. I cannot understand what you have against qmail to go around bad mouthing it. Don't tell me a stuffed queue and perhaps a listing in some over zealous RBL was all it took.
Feizhou wrote:
[1] qmail's license used to be source distribution only, because that locked out anyone unable to compile it and its dependent packages and "killed the weak". Mail isn't that hard! Mortals can get Postfix going!
That's not the worst part of the license. The real problem is that qmail as written has several logical flaws, the above-mentioned being the most obvious, and the license states that no one is allowed to distribute modified versions so it can't be fixed without completely replacing components.
There is a slight workaround. One can distribute the original + source
- patches and a script that does the patching/installing. Hence the
netqmail-1.05 version.
Les, quit making false statements about qmail. qmail does not have any logical flaws otherwise Wietse would not follow the same design principles in postfix after his spat with DJB. qmail's problem is that the author has not done any updates to it since 1998 to handle the changed needs required of a MTA software since then. SMTP is broken which is why spammers and virus writers can create so much trouble. Both SMTP and qmail were designed in an environment very different from what we have now. That is why we have ESMTP and what not updates to the SMTP protocol.
qmail is a fine piece of software and it also introduced the maildir format which is now widely supported and used by anyone who cares about the integrity of their mailboxes. I cannot understand what you have against qmail to go around bad mouthing it. Don't tell me a stuffed queue and perhaps a listing in some over zealous RBL was all it took.
And what exactly does any of this have to do with CentOS? Personally, I dumped qmail back in the 90's when it became apparent what a flaming horse's hiney DJB was. Wietse has been a lot easier to deal with, quick with patches, and just an overall nice fellow. So I've used postfix since 98-ish without incident and still use it today. It's fast, easy to configure, and it is well supported by Redhat (and by extension) and CentOS (see, this is sorta on topic now). So while I could go through all these gyrations to make qmail work...blah blah blah....why should I? If you'd like to continue this, I'd be happy to do it off list.
Cheers,
Chris
On Fri, 2006-08-25 at 22:30 +0800, Feizhou wrote:
Les, quit making false statements about qmail.
Nothing I've said about qmail has been false.
qmail does not have any logical flaws otherwise Wietse would not follow the same design principles in postfix after his spat with DJB.
That's a funny interpretation - if qmail had been done correctly there would have been no need for postfix.
qmail's problem is that the author has not done any updates to it since 1998 to handle the changed needs required of a MTA software since then.
In the open source world that is not a problem. If something needs to be fixed, someone will fix it because they are allowed to. That has happened with sendmail. What the original author does or doesn't do is mostly irrelevant.
qmail is a fine piece of software and it also introduced the maildir format which is now widely supported and used by anyone who cares about the integrity of their mailboxes.
I have no problem with maildir - or the format cyrus uses for that matter, but that's not relevant to a discussion of mail transports. Sendmail doesn't perform local delivery. If you want maildir, tell procmail to use that and you get maildir. If you want cyrus, use the delivery agent it provides.
I cannot understand what you have against qmail to go around bad mouthing it. Don't tell me a stuffed queue and perhaps a listing in some over zealous RBL was all it took.
The first thing that turned me off about qmail was its insistence on sending separate copies of messages to multiple recipients at the same remote host. The machines I manage have always had a usage pattern of most email being to distribution groups along the lines of departmental memos with many recipients grouped at remote offices, and at the time delivery happened over slow and expensive private lines where this kind of stupid behavior just didn't work. I've never expected software to be perfect but I do expect it to improve over time. With anything written by DJB, this doesn't happen and the best thing to do is run away and don't look back. When qmail also demonstrated its inability to process inbound mail it was just additional proof that my first impression had been right.
Les Mikesell wrote:
If you are handling relatively low volumes of mail, say the low tens of thousands a day, and "mail guy" is not a shout you respond to, then I strongly recommend not becoming a white-coated acolyte to these and to make the smaller brain-investment needed to get Postfix working great.
Unfortunately the amount of real mail you intend to handle doesn't relate much to what can happen when you plug into the internet.
Hm well I run my own MX that is "on the Internet" and have done for a couple of years or more, and I do it with Postfix on a residential cable modem. I have never had these spamfloods, Every day my daily logs for this and other machines show one or more attempts to relay which fail during SMTP time, so they go somewhere else. Often the recipient on the relaying attempt is undeliverable, they're just interested if you'll take it. I guess if you take their probes, then you get the Zombie army hammering at the door.
If you set your MTA (whatever it is) up with
- reject unknown usernames (much virus mail and a fair amount of spam: gone)
- reduce the stock usernames in /etc/aliases, keep the RFC ones
- greylist one way or another (10 mins seems to work fine)
- reject non-FQDN HELO
- optionally reject "unknown" HELOs, ie, alleged mailservers that lack reverse DNS
you will knock out the vast bulk of your enemies before you spend any real CPU or bandwidth on them. So far I did not need to look at the next step, doing a fake DNS lookup on one of the realtime blackhole lists.
Because all of these operate at SMTP transaction time the problems you point out don't result in dodgy bounces that are sent to the alleged From guy. Anything that can't be talked out of sending dodgy bounces to the alleged From guy would indeed be evil.
That's not the worst part of the license. The real problem is that qmail as written has several logical flaws, the above-mentioned being the most obvious, and the license states that no one is allowed to distribute modified versions so it can't be fixed without completely replacing components.
he he what a nonsense license. It's up there with Creative Commons Non-commercial stopping radio stations playing liberally licensed music as needing a shooting yourself in the foot award.
-Andy
Andy Green wrote:
Les Mikesell wrote:
If you are handling relatively low volumes of mail, say the low tens of thousands a day, and "mail guy" is not a shout you respond to, then I strongly recommend not becoming a white-coated acolyte to these and to make the smaller brain-investment needed to get Postfix working great.
Unfortunately the amount of real mail you intend to handle doesn't relate much to what can happen when you plug into the internet.
Hm well I run my own MX that is "on the Internet" and have done for a couple of years or more, and I do it with Postfix on a residential cable modem. I have never had these spamfloods, Every day my daily logs for this and other machines show one or more attempts to relay which fail during SMTP time, so they go somewhere else. Often the recipient on the relaying attempt is undeliverable, they're just interested if you'll take it. I guess if you take their probes, then you get the Zombie army hammering at the door.
If you set your MTA (whatever it is) up with
- reject unknown usernames (much virus mail and a fair amount of
spam: gone)
reduce the stock usernames in /etc/aliases, keep the RFC ones
greylist one way or another (10 mins seems to work fine)
reject non-FQDN HELO
optionally reject "unknown" HELOs, ie, alleged mailservers that
lack reverse DNS
you will knock out the vast bulk of your enemies before you spend any real CPU or bandwidth on them. So far I did not need to look at the next step, doing a fake DNS lookup on one of the realtime blackhole lists.
Because all of these operate at SMTP transaction time the problems you point out don't result in dodgy bounces that are sent to the alleged From guy. Anything that can't be talked out of sending dodgy bounces to the alleged From guy would indeed be evil.
I use similar tactics on my postfix setups and have not had any DoS or other successful attacks against any of the servers under my care in the last 8 years or so. And they're all dangling out on the Internet with a big bullseye painted on them. So I think the risk is manageable and not terribly relevant for me. I've got a few servers that are rather busy and have had servers in the past that were handling a few tens of thousands of users.
Understanding and managing risks associated with being plugged in to the Internet is not a MTA-specific problem. But I daresay that some MTA's are a bit more difficult to understand than others. ;-)
Cheers,
On Fri, 2006-08-25 at 18:52 +0100, Andy Green wrote:
Unfortunately the amount of real mail you intend to handle doesn't relate much to what can happen when you plug into the internet.
Hm well I run my own MX that is "on the Internet" and have done for a couple of years or more, and I do it with Postfix on a residential cable modem. I have never had these spamfloods, Every day my daily logs for this and other machines show one or more attempts to relay which fail during SMTP time, so they go somewhere else.
Do you want some? My maillog shows 625856 rejects in the last 5 days. We have had some employee turnover so some are to previously valid addresses, but most are to things like seg04_831@domain and segark862@domain, and so on.
Often the recipient on the relaying attempt is undeliverable, they're just interested if you'll take it. I guess if you take their probes, then you get the Zombie army hammering at the door.
Yes, I suppose this is still a lingering after effect of long ago having a qmail box answering for that domain (it was an appliance-like SME server - I wouldn't have set one up like that otherwise...). But they've been getting rejected at that rate for a couple of years now and still coming.
If you set your MTA (whatever it is) up with
- reject unknown usernames (much virus mail and a fair amount of spam:
gone)
The difficulty here is that my internet-reachable relays don't actually have any users.
Because all of these operate at SMTP transaction time the problems you point out don't result in dodgy bounces that are sent to the alleged From guy.
MimeDefang allows checking for valid addresses at the delivery host during the SMTP transaction before accepting at the relay. I know there are ways to propagate all of your usernames and aliases in LDAP or other network database form so other MTAs could have the same functionality, but MimeDefang lets you use the real thing in real time without setting up other copies.
Les Mikesell wrote:
Do you want some? My maillog shows 625856 rejects in the last 5 days.
Wow that is awesome, 100K+ attempts a day is totally beyond what I experience, not even 1K attempts a day. Even so I quite like less-ing through my maillog and smiling at the NOQUEUEs.
they've been getting rejected at that rate for a couple of years now and still coming.
I guess you made it on to some list that never gets cleaned, no wonder you have it in for qmail ;-)
- reject unknown usernames (much virus mail and a fair amount of spam:
gone)
The difficulty here is that my internet-reachable relays don't actually have any users.
Ah well the advice is still good. Deciding you don't want the mail at SMTP time is the golden solution everyone agrees, comparing against a domainwide userlist is the way whether it is done with mimedefang or some script that collates usernames. My mail setup is so small that I have /etc/aliases to control users and that's the end of it. I guess that's at the heart of why sendmail complexity has no payback for me but does for you.
-Andy
Yes, if you want to do something that no one else has needed to do in sendmail's 3-decade history, you'll probably have to read the bat book first. But that seems unlikely to me and with MimeDefang in the picture you can control many operations in your perl filter instead of modifying sendmail macros.
hmm...i wonder why there are sendmail + mysql patches floating around then. Being able to something natively is obviously desired over passing it out to perl.
I guess that is the whole point. Ordinary users will probably have no clue how sendmail works and when problems arise, good luck trying to fix or get around them. So unless they know how to build other stuff and use procmail or maildrop, they are pretty much stuck to system mailboxes.
As shipped in Centos, procmail is used for local delivery.
Ah, i forgot the part about procmail don't support virtual mailboxes and therefore wasn't really what I want to put in there. postfix, qmail and maybe exim all do that out of the box without having to resort to using maildrop.
Then when you add MimeDefang, you also get the ability to add in any other operations you want to happen in parallel with the smtp chat and control it all with a bit of perl.
Which makes it no different from your dissing of qmail. qmail provides very little that you can do before DATA in the smtp chat and mimedefang does not kick in until after SMTP DATA in sendmail and they are therefore the same.
I think you've been misinformed. My MimeDefang checks the inbound recipients via smtp to the final delivery server before DATA is ever mentioned. If there are no valid recipients (which the outside relay wouldn't know directly) the message is rejected before DATA.
right, sorry, i have not actually used milter myself so I was not aware that it was not limited to content filtering. In any case, this shows one shortcoming of sendmail. sendmail processes each the helo, sender, recipient and client ip/rdns in separate rulesets and you cannot create (at least I have not managed it yet) any rulesets that can make decisions based on two or more of these and so you need to run a perl program via milter to deal with these or perhaps in your case, you need it to lookup a mysql table for the recipient. You call this efficient?
As for mimedefang, qmail lets you do anything that can be described in perl, shell, C, python, whatever you fancy in fact and reject at the smtp level too since you can replace qmail-queue or put a filter before qmail-queue.
Another way of saying that is that qmail is so bad you have to completely replace components to make it usable at all.
Which is no different from sendmail + mimedefang. You have no idea how qmail works so I shall ignore your ignorance on this.
I've experienced the pain of using qmail. I know more than I want to know about how it accepts everything, then generates millions of bounces for the things it can't deliver then lets the bounce delivery attempts throttle your real deliverable outbound mail.
Right, so because qmail does not do recipient checking at the smtp level you write it off although there are options in qmail that you can take to get the same effect and which, in fact, is what you are also doing with sendmail + mimedefang from what you posted. It appears to me you are being unfair in your treatment then of qmail nevermind the rest.
sendmail gave you milter so that you can get at the headers and message body. DJB's multiple program approach to separate the different functions
But he separates the functions so you can't do the thing you need at the time you need to do it...
Yes, you are right that unless you get into the qmail-smtpd code, you cannot do any before SMTP DATA checking which is what qmail-ldap and qmail-sql have done and what various patches have done too...hence why I say qmail alone is not suitable as an mx mta in today's Internet.
automatically provides you the ability to get at what you want by either modifying that particular qmail program
Wait - you think modifying a program is easier than configuring one that works in the first place?
Normally no. qmail is an exception because it is the code is so simple it is plain to see where you can add extra checking stuff. Of course, that is the case if writing your own mysql lookup code and what not is possible for you :D. Otherwise, yes, qmail is not suitable as an mx mta.
or by replacing like qpsmtpd or by adding another program like qmail-qfilter. For this same reason, people don't get any trouble from postfix and qmail with regard to security issues and sendmail X following the same design principles says a lot.
The milter interface is a more extreme solution, since it gives you separation of permissions for the parts on the other end of the socket but you get its response when you need it.
sendmail runs as root in its entirety (others like qmail and postfix only have one or two running as root and the rest run with other uids) so the separation of permissions is necessary for sendmail. I don't see anything extreme about this....this is standard elsewhere.
Well integrated and designed for efficiency eh? I'd like to see benchmarks between sendmail + mimedefang versus postfix + amavisd. In fact, I'd like to see sendmail + mimedefang integrated with a mysql backend versus postfix + amavisd integrated with a mysql backend. But the whole thing would be unfair since postfix does connection pooling to its backends while sendmail will probably beat the crap out of its backend if it supported any sql database at all.
If you run spamassassin, all of the other operations are pretty much meaningless time-wise. The piece you need to keep things going is a multiplexer that allows some maximum number of backend scanners and some larger number of other MTA operations to run concurrently. That is, you don't want to serialize everything around a few scanners, and you also don't want to be allowed to spawn off enough instances of them to kill the machine. I don't think any of the other systems understand this concept - but I'd be happy to hear that I'm wrong about that.
Okay, since you are talking about mimedefang being in action before SMTP DATA mode, qmail is already out unless you patch it. I have not used exim so I cannot say anything for it.
postfix, however, can already do a fair amount of checks on the helo, recipient, sender, client ip/rdns separately and/or in combination out of the box. You can apply lookup checks against sql databases, flat file databases, regex, pcre (a perl regex C library) and for the helo and client ip/rdns you now get some interesting dns checks available too. If rejecting, you can also give custom responses too. I don't know what checks you are doing with mimedefang but I think the chances are high they can be done within the postfix provided framework without ever hitting an external filter. postfix will handle far more traffic that sendmail + mimedefang could have hope to in regard to before SMTP DATA checks unless of course you do something in mimedefang that cannot be done with the postfix framework. What do you do in mimedefang anyway?
You can diss all other mtas + their addons all you like Les, but sendmail X is following the design principles of qmail and postfix which says something.
The only one I'll really diss is qmail, and I'll concede that qpsmtpd will solve a few of it's problems as the project slowly re-invents the things that MimeDefang has been doing for years. My point is just that sendmail does a good job and has some unique advantages when used with MimeDefang.
I will concede that sendmail does a good job for what it is designed to do (read no virtual mailboxes and only supporting system account mbox format mailboxes) which is suitable for small sites but I highly doubt that it has any unique advantages in combination with mimedefang versus postfix or exim out of the box or against qmail + patches + external (qpsmtpd don't cut it).
On Fri, 2006-08-25 at 15:06 +0800, Feizhou wrote:
I think you've been misinformed. My MimeDefang checks the inbound recipients via smtp to the final delivery server before DATA is ever mentioned. If there are no valid recipients (which the outside relay wouldn't know directly) the message is rejected before DATA.
right, sorry, i have not actually used milter myself so I was not aware that it was not limited to content filtering.
Why are knocking something you don't understand?
In any case, this shows one shortcoming of sendmail.
No, it show how the shortcoming has been fixed in the current version.
sendmail processes each the helo, sender, recipient and client ip/rdns in separate rulesets and you cannot create (at least I have not managed it yet) any rulesets that can make decisions based on two or more of these and so you need to run a perl program via milter to deal with these or perhaps in your case, you need it to lookup a mysql table for the recipient. You call this efficient?
Yes it is efficient because you don't have to start a perl program for every message. You let a pre-initialized program do the processing and respond in real time. It's like the difference between a web server running cgi scripts vs. mod_perl or fastcgi - that is, hundreds of times faster.
Right, so because qmail does not do recipient checking at the smtp level you write it off although there are options in qmail that you can take to get the same effect and which, in fact, is what you are also doing with sendmail + mimedefang from what you posted. It appears to me you are being unfair in your treatment then of qmail nevermind the rest.
No, I write off qmail because circumstances and experience show that it isn't going to get fixed.
The milter interface is a more extreme solution, since it gives you separation of permissions for the parts on the other end of the socket but you get its response when you need it.
sendmail runs as root in its entirety (others like qmail and postfix only have one or two running as root and the rest run with other uids) so the separation of permissions is necessary for sendmail. I don't see anything extreme about this....this is standard elsewhere.
Since this is the centos list, perhaps we could talk about sendmail as included in the distribution... Sendmail runs as root long enough to open port 25. And separate components write into the queue and deliver from it. Things running as milters can use whatver uid/permissions are appropriate. Talking about anything else would be much like elaborating on the 2 gig file size limit in Linux.
I don't know what checks you are doing with mimedefang but I think the chances are high they can be done within the postfix provided framework without ever hitting an external filter. postfix will handle far more traffic that sendmail + mimedefang could have hope to in regard to before SMTP DATA checks unless of course you do something in mimedefang that cannot be done with the postfix framework. What do you do in mimedefang anyway?
I have a pair of internet relays in a DMZ that forward for firewalled local mail servers. MimeDefang runs virus scans on all mail and spamassassin on inbound mail, first checking that the user(s) actually exist on the destination host with the included md_check_against_smtp_server function. I don't think postfix offers that option, but I'd be pleasantly surprised if it can.
I will concede that sendmail does a good job for what it is designed to do (read no virtual mailboxes and only supporting system account mbox format mailboxes)
Sendmail has nothing to do with local delivery beyond starting the selected delivery agent. I've always provided additional services like home directories, cvs, wiki's etc. on the same machine as used for mail delivery so the users are real rather than virtual, although different domains have different servers (with a recent exception of using a vmware virtual server for one as it is being phased out). But others are using it that way.
which is suitable for small sites but I highly doubt that it has any unique advantages in combination with mimedefang versus postfix or exim out of the box or against qmail + patches + external (qpsmtpd don't cut it).
It's real unique advantage is decades-long consistency and improvement. With the milter interface you get the ability to add any additional processing you want in the language of your choice - and with MimeDefang pretty much all of that is already done for you. If you are interested in why it is a good approach, there is a very detailed description if you follow the 'slides' link at the top of http://www.mimedefang.com.
I have a pair of internet relays in a DMZ that forward for firewalled local mail servers. MimeDefang runs virus scans on all mail and spamassassin on inbound mail, first checking that the user(s) actually exist on the destination host with the included md_check_against_smtp_server function. I don't think postfix offers that option, but I'd be pleasantly surprised if it can.
not the RHEL4 (therefore Centos 4) one. postfix does offer this option now.
It's real unique advantage is decades-long consistency and improvement. With the milter interface you get the ability to add any additional processing you want in the language of your choice - and with MimeDefang pretty much all of that is already done for you. If you are interested in why it is a good approach, there is a very detailed description if you follow the 'slides' link at the top of http://www.mimedefang.com.
I'll probably wait for postfix + milter support to become stable and take the advantages of postfix over sendmail if I need the features provided by mimedefang.
Feizhou wrote:
right, sorry, i have not actually used milter myself so I was not aware that it was not limited to content filtering. In any case, this shows one shortcoming of sendmail. sendmail processes each the helo, sender, recipient and client ip/rdns in separate rulesets and you cannot create (at least I have not managed it yet) any rulesets that can make decisions based on two or more of these and so you need to run a perl program via milter to deal with these or perhaps in your case, you need it to lookup a mysql table for the recipient. You call this efficient?
I don't have anything in particular against Postfix and Sendmail. I'm not advocating using Postfix over Sendmail here. I'm not advocating using Sendmail over Postfix either. Frankly, couldn't care less what MTA you are using or advocating. Just feeling a bit educational.
I'm using a custom ruleset in Local_check_rcpt that checks the sender part for some heavily locked down accounts. Among other things, it checks if local recipient is allowed to receive mail from particular sender (for inbound mail), and if the local sender is allowed to send to particular recipient (for outbound mail). Obviously, that ruleset is making decions based on both sender and recipient. If you'll need it in the future, sender's address is stored in $f macro, as nicely documented in Sendmail documentation (Sendmail Installation and Operation Guide). There's also bunch of other macros you can use throughout your rulesets. So you see, it is possible, well documented, and simple.
If you are using Milter interface to implement something like above in an external filter (say it is C program), you'd simply use Milter API to store information between callback function calls. You do not need to store data anywhere externally (like in database, or in files, or whatever). In the above (simple) example, in callback function for mail command you'd allocate some memory for a structure to hold your data (for example, sender's adress and whatever else you want to keep track of), and use smfi_setpriv() to let Milter API know you want to keep track of that data for this particular connection. In callback function for rcpt command, you'd use smfi_getpriv(), which would return you a pointer to your data (sender's address and whatever else you are keeping track of between callback function calls). Note that smfi_setpriv() and smfi_getpriv() do not move or copy or store the data anywhere. The data stays where it was, in the memory. Those two functions accept and return the pointer to the location in memory (that your program allocated). You'd also register a callback to a cleanup function, which would be called at the end of transaction or connection (depending what callback you register) to free up the allocated memory (and/or any other resources you allocated/used).
It is done this way since filters in Sendmail are multithreaded. There is only one instance of filter running. It's kind of more complicated to code multithreaded program, but on the other hand it is more efficient than having single filter process running for each current transaction (on busy email server you can have dozens or even hundreds of transactions running in parallel). It is very efficient, and very well designed filtering API for email transactions, and it would be really great to see this API implemented in Postfix too. I just hope it will be exactly the same API as in Sendmail so that all existing Sendmail filters can be used in Postfix, and that all future Postfix filters could be used in Sendmail.
If you are using MIMEDefang, you don't even need to bother with all that. MIMEDefang is doing all that behind the scenes for you. Your filter_recipient function will get the sender's address in second argument (as well as bunch of other info from previous stages in the remaining arguments).
I will concede that sendmail does a good job for what it is designed to do (read no virtual mailboxes and only supporting system account mbox format mailboxes).
Reading the above, one would conclude that you can't have Sendmail and Cyrus on the same box. Cyrus does not store mail in mbox format. Cyrus does not use system user accounts (all users are virtual, as far as Cyrus is concerned). But somehow, Sendmail just works with Cyrus.
Thing is. Sendmail doesn't know about mailboxes. Period. Sendmail will never attempt to write a mail into a mailbox. That's the job of local delivery agent (mailer). You tell sendmail (M lines in sendmail.cf file) how to talk with the mailer. Mailer will than write the email into the mailbox. Sendmail doesn't care if there is corresponding system account for that mailbox, or what format the mailbox is.
The only way to get Sendmail to write email directly into the file on the disk (note that I wrote "file on the disk", not "mailbox") is using aliases file or dot forward file in user's home directory (if there are system accounts on the mail server). But this is not considered to be delivery into mailbox. It's just "append this to that file" quick and dirty feature, and yes, you are doomed to mbox format if you are lazy enough to actually use it.
Of course, if there are system accounts, there's a feature or two extra that Sendmail will offer (like dot forward file). But you do not need to have system accounts on the mail server for Sendmail to work.
I'm using a custom ruleset in Local_check_rcpt that checks the sender part for some heavily locked down accounts. Among other things, it checks if local recipient is allowed to receive mail from particular sender (for inbound mail), and if the local sender is allowed to send to particular recipient (for outbound mail). Obviously, that ruleset is making decions based on both sender and recipient. If you'll need it in the future, sender's address is stored in $f macro, as nicely documented in Sendmail documentation (Sendmail Installation and Operation Guide). There's also bunch of other macros you can use throughout your rulesets. So you see, it is possible, well documented, and simple.
??? The documentation:
The following macros are defined and/or used internally by sendmail for interpolation into argv's for mailers or for other contexts. The ones marked * are information passed into sendmail[16], the ones marked # are information passed both in and out of sendmail, and the unmarked macros are passed out of sendmail but are not otherwise used internally. These macros are:
$f is unmarked.
I only see $f being used in:
sendmail-cf]# grep -r '$f' ./ ./README:FAX_MAILER_ARGS [mailfax $u $h $f] The arguments passed to the FAX ./README:PROCMAIL_MAILER_ARGS [procmail -Y -m $h $f $u] The arguments passed to ./README: setreuid() call, you may need to add -f $f to the procmail ./ostype/uxpds.m4:define(`UUCP_MAILER_ARGS', `uux - -r -a$f -gmedium $h!rmail ($u)')dnl ./ostype/a-ux.m4:ifdef(`LOCAL_MAILER_ARGS',, `define(`LOCAL_MAILER_ARGS', `mail -d -r $f $u')')dnl ./ostype/qnx.m4:define(`UUCP_MAILER_ARGS', `uux - -r -z -a$f $h!rmail ($u)')dnl ./mailer/fax.m4: `define(`FAX_MAILER_ARGS', faxmail -d $u@$h $f)') ./mailer/procmail.m4: `define(`PROCMAIL_MAILER_ARGS', `procmail -Y -m $h $f $u')')
If you can, please let me see the relevant section in the Local_check_rcpt ruleset where the sender env is being used. I guess the helo and client ip/rdns are not stored anywhere...
It is done this way since filters in Sendmail are multithreaded. There is only one instance of filter running. It's kind of more complicated to code multithreaded program, but on the other hand it is more efficient than having single filter process running for each current transaction (on busy email server you can have dozens or even hundreds of transactions running in parallel). It is very efficient, and very well designed filtering API for email transactions, and it would be really great to see this API implemented in Postfix too. I just hope it will be exactly the same API as in Sendmail so that all existing Sendmail filters can be used in Postfix, and that all future Postfix filters could be used in Sendmail.
Wietse designed postfix to replace sendmail...so some sendmail quirks have also hit postfix. (undocumented sendmail behaviour but replicated nonetheless) I just noticed that postfix 2.3 is now the stable series. milter is now standard but I'd wait a bit more since the recent changelogs indicated some stabilizing of milter code.
If you are using MIMEDefang, you don't even need to bother with all that. MIMEDefang is doing all that behind the scenes for you. Your filter_recipient function will get the sender's address in second argument (as well as bunch of other info from previous stages in the remaining arguments).
I will concede that sendmail does a good job for what it is designed to do (read no virtual mailboxes and only supporting system account mbox format mailboxes).
Reading the above, one would conclude that you can't have Sendmail and Cyrus on the same box. Cyrus does not store mail in mbox format. Cyrus does not use system user accounts (all users are virtual, as far as Cyrus is concerned). But somehow, Sendmail just works with Cyrus.
You missed this part of my mail:
"Ah, i forgot the part about procmail don't support virtual mailboxes and therefore wasn't really what I want to put in there. postfix, qmail and maybe exim all do that out of the box without having to resort to using maildrop." I did not say that you cannot make any virtual mail LDA work with sendmail. sendmail does not somehow just work with Cyrus. Somebody wrote the necessary ruleset to be the cyrus LDA working with sendmail.
Thing is. Sendmail doesn't know about mailboxes. Period. Sendmail will never attempt to write a mail into a mailbox. That's the job of local delivery agent (mailer). You tell sendmail (M lines in sendmail.cf file) how to talk with the mailer. Mailer will than write the email into the mailbox. Sendmail doesn't care if there is corresponding system account for that mailbox, or what format the mailbox is.
Okay, sendmail does not come with a virtual mailbox capable LDA then. I did not very clearly highlight the MTA and LDA parts. It does not change the truth of what I said.
The only way to get Sendmail to write email directly into the file on the disk (note that I wrote "file on the disk", not "mailbox") is using aliases file or dot forward file in user's home directory (if there are system accounts on the mail server). But this is not considered to be delivery into mailbox. It's just "append this to that file" quick and dirty feature, and yes, you are doomed to mbox format if you are lazy enough to actually use it.
Hmm...okay, so most sendmail users are probably not only ignorant of how sendmail works, they are lazy too now I guess.
Of course, if there are system accounts, there's a feature or two extra that Sendmail will offer (like dot forward file). But you do not need to have system accounts on the mail server for Sendmail to work.
Yes and dot qmail offers much more over dot forward.
Wietse has done a great job. Probably a good thing he and DJB had a major spat since DJB has not done a thing since 1998 about qmail. Wished he write a dot-qmail compatible LDA though.
Feizhou wrote:
??? The documentation:
The following macros are defined and/or used internally by sendmail for interpolation into argv's for mailers or for other contexts. The ones marked * are information passed into sendmail[16], the ones marked # are information passed both in and out of sendmail, and the unmarked macros are passed out of sendmail but are not otherwise used internally. These macros are:$f is unmarked.
So what? Sendmail isn't using it internally. But you can still use it in the configuration file, as long as you use it after senders address is assigned to it.
If you can, please let me see the relevant section in the Local_check_rcpt ruleset where the sender env is being used. I guess the helo and client ip/rdns are not stored anywhere...
The beginning of my Local_check_rcpt looks something like:
SLocal_check_rcpt R$* $: $>3 $1 R$* $: $1 $| $>3 $&f
The input workspace for Local_check_rcpt contains only the recipient. This two rules rewrites the workspace to look something like "recipient $| sender". The remaining rules (not quoted above) than work on this pair, and based on some map lookups the ruleset returns either OK or an error. Very simple.
You missed this part of my mail:
[snip]
Okay, sendmail does not come with a virtual mailbox capable LDA then. I did not very clearly highlight the MTA and LDA parts. It does not change the truth of what I said.
And you haven't really understood Sendmail's design. Sendmail does not ship with an LDA. Well, to be correct, there is this simple LDA included in the source, however it is not compiled and installed by default (you have to do it by hand). Even the readme file for it discourages its use. It is not part of Red Hat's sendmail RPM either.
Procmail is not part of Sendmail. It is completely separate component. It was nothing more than Red Hat's decision to use procmail as default LDA in their default Sendmail configuration. Well Red Hat's and pretty much every other Linux distro. Solaris for example uses completely different LDA by default.
Sendmail is purely and only MTA implementation. Moving email around. Not storing it. It has nothing to do with actually delivering mail into the mailboxes. There is not a single line of code in Sendmail itself that deals with storing the email into the mailbox. Any kind of mailbox. System or virtual. Berkeley format or maildir. The simple LDA that I mentioned is in completely different tree inside Sendmail sources, clearly separate from Sendmail, and I really never looked at it as part of Sendmail as such. If it was removed from the sources, there would be very few people to actually notice it.
If you are using Sendmail, you must have an external LDA, and you must configure Sendmail to pass email to it. To make things simple, Sendmail distribution comes with predefined configuration directives for most common cases. Almost all operating system distributions that use Sendmail as default MTA come with preconfigured Sendmail that "just works". And to complete the circle, for most common operating system distributions, Sendmail sources come with predefined configuratin files that use default LDA for that system. So the users that do not need anything better, they simply will never notice this important detail.
So again, don't blame Sendmail for limitations of particular LDA. If LDA doesn't do what you need, get another LDA that does. Nothing to do with Sendmail.
Hmm...okay, so most sendmail users are probably not only ignorant of how sendmail works, they are lazy too now I guess.
I'm lazy too. From time to time I alias something to /dev/null. However, I don't really care in what format /dev/null mailbox is.
Wietse has done a great job.
Yes Wietse did a great job. However there is no universal tool that is best for everything and everybody. Let people use whatever MTA works for them. Having a choice is marvelous thing.
The beginning of my Local_check_rcpt looks something like:
SLocal_check_rcpt R$* $: $>3 $1 R$* $: $1 $| $>3 $&f
The input workspace for Local_check_rcpt contains only the recipient. This two rules rewrites the workspace to look something like "recipient $| sender". The remaining rules (not quoted above) than work on this pair, and based on some map lookups the ruleset returns either OK or an error. Very simple.
Thanks.
I probably might have missed them if they exist but are there any for the helo string and client ip/rdns?
On Wed, 2006-08-30 at 00:02, Feizhou wrote:
The beginning of my Local_check_rcpt looks something like:
SLocal_check_rcpt R$* $: $>3 $1 R$* $: $1 $| $>3 $&f
The input workspace for Local_check_rcpt contains only the recipient. This two rules rewrites the workspace to look something like "recipient $| sender". The remaining rules (not quoted above) than work on this pair, and based on some map lookups the ruleset returns either OK or an error. Very simple.
Thanks.
I probably might have missed them if they exist but are there any for the helo string and client ip/rdns?
They are available to milters. Here's an example of the perl you might use in mimedefang: http://www.mimedefang.org/node.php?id=18
I probably might have missed them if they exist but are there any for the helo string and client ip/rdns?
They are available to milters. Here's an example of the perl you might use in mimedefang: http://www.mimedefang.org/node.php?id=18
Oh well, at least that confirms I cannot get the helo string to use later with just sendmail rulesets.
On Wed, 2006-08-30 at 00:37, Feizhou wrote:
I probably might have missed them if they exist but are there any for the helo string and client ip/rdns?
They are available to milters. Here's an example of the perl you might use in mimedefang: http://www.mimedefang.org/node.php?id=18
Oh well, at least that confirms I cannot get the helo string to use later with just sendmail rulesets.
No, it just means I'm too lazy to do it that way when it is easy in perl. Sendmail obviously has it in a macro or it couldn't be instructed to pass it to milters. Look around page SMM:08-46 of this document for the list of macros: http://sendmail.org/doc/sendmail-current/doc/op/op.pdf
Les Mikesell wrote:
On Wed, 2006-08-30 at 00:37, Feizhou wrote:
I probably might have missed them if they exist but are there any for the helo string and client ip/rdns?
They are available to milters. Here's an example of the perl you might use in mimedefang: http://www.mimedefang.org/node.php?id=18
Oh well, at least that confirms I cannot get the helo string to use later with just sendmail rulesets.
No, it just means I'm too lazy to do it that way when it is easy in perl. Sendmail obviously has it in a macro or it couldn't be instructed to pass it to milters. Look around page SMM:08-46 of this document for the list of macros: http://sendmail.org/doc/sendmail-current/doc/op/op.pdf
i did...there is none for helo but i missed the client_$$$ bunch somehow.
Feizhou wrote:
i did...there is none for helo but i missed the client_$$$ bunch somehow.
As I wrote already, the argument of HELO/EHLO is in $s (well, you'd use delayed expansion in rulesets to get what you really want, so you would actually use $&s). When sendmail reads its configuration file, $s contains the name of the local host. After HELO/EHLO it will be assigned whatever was the argument of HELO/EHLO.
Aleksandar Milivojevic wrote:
Feizhou wrote:
i did...there is none for helo but i missed the client_$$$ bunch somehow.
As I wrote already, the argument of HELO/EHLO is in $s (well, you'd use delayed expansion in rulesets to get what you really want, so you would actually use $&s). When sendmail reads its configuration file, $s contains the name of the local host. After HELO/EHLO it will be assigned whatever was the argument of HELO/EHLO.
OH, so that was what the $s represented. Thanks.
On Wed, 2006-08-30 at 22:09, Feizhou wrote:
I probably might have missed them if they exist but are there any for the helo string and client ip/rdns?
They are available to milters. Here's an example of the perl you might use in mimedefang: http://www.mimedefang.org/node.php?id=18
Oh well, at least that confirms I cannot get the helo string to use later with just sendmail rulesets.
No, it just means I'm too lazy to do it that way when it is easy in perl. Sendmail obviously has it in a macro or it couldn't be instructed to pass it to milters. Look around page SMM:08-46 of this document for the list of macros: http://sendmail.org/doc/sendmail-current/doc/op/op.pdf
i did...there is none for helo but i missed the client_$$$ bunch somehow.
The way I read it, $s holds the helo response when receiving. You could always peek at the source code to be sure.
Feizhou wrote:
The beginning of my Local_check_rcpt looks something like:
SLocal_check_rcpt R$* $: $>3 $1 R$* $: $1 $| $>3 $&f
The input workspace for Local_check_rcpt contains only the recipient. This two rules rewrites the workspace to look something like "recipient $| sender". The remaining rules (not quoted above) than work on this pair, and based on some map lookups the ruleset returns either OK or an error. Very simple.
Thanks.
I probably might have missed them if they exist but are there any for the helo string and client ip/rdns?
There's set of macros that deal with that (${client_addr}, ${client_name}, ${client_ptr}, $s). Not sure about HELO/EHLO argument. Check what is exactly in $s or definition of Received header (it digs it from somewhere).
Aleksandar Milivojevic schrieb:
Feizhou wrote:
The beginning of my Local_check_rcpt looks something like:
SLocal_check_rcpt R$* $: $>3 $1 R$* $: $1 $| $>3 $&f
The input workspace for Local_check_rcpt contains only the recipient. This two rules rewrites the workspace to look something like "recipient $| sender". The remaining rules (not quoted above) than work on this pair, and based on some map lookups the ruleset returns either OK or an error. Very simple.
Thanks.
I probably might have missed them if they exist but are there any for the helo string and client ip/rdns?
There's set of macros that deal with that (${client_addr}, ${client_name}, ${client_ptr}, $s). Not sure about HELO/EHLO argument. Check what is exactly in $s or definition of Received header (it digs it from somewhere).
To those of you dealing with Sendmail the name "Neil W. Rickert" should say something.
http://www.cs.niu.edu/~rickert/cf/bad-ehlo.html -->
http://www.cs.niu.edu/~rickert/cf/hack/block_bad_helo.m4
rDNS tests do too much damage: please see http://www.cs.niu.edu/~rickert/cf/
"HACK(`require_rdns') http://www.cs.niu.edu/%7Erickert/cf/hack/require_rdns.m4 -- reject mail from sites without valid reverse DNS. Access entries allow individual override. I don't recommend this. The amount of collateral damage is excessive."
Alexander
Les Mikesell spake the following on 8/23/2006 9:38 PM:
On Wed, 2006-08-23 at 23:22, Feizhou wrote:
YEAH ! Flamewar ! Yohooooooooooooo!
haha. well, i guess having had to immediately upgrade more than a dozen boxes running sendmail whenever a remote exploit was found kind of made me rather unwilling to deal with it anymore after the peace and quiet I get from running postfix. Also, not having to decipher them sendmail rulesets and create new ones were a relief. Then there was the instability of milter to contend with...
I guess that is all long past now is it?
Unless typing 'yum update' to pick up fixes that have been needed less often than in the Linux kernel for the last few years is a problem for you...
No one edits sendmail.cf directly anymore and milter has been stable for about as long as postfix has existed at all. Postfix still doesn't have a way to let you hook user defined scanners running under a different uid to run in realtime during the smtp conversation, does it? MimeDefang lets you do anything you can describe in perl and return the response through the milter interface for various operations before the mail has been acknowledged as accepted.
I think postfix has added milter support lately. I use sendmail, but only because it is what I know, and I don't want to muck about in a working mailserver. Someday I will learn postfix, right after I get those washboard abs and die out the grey! ;-) ('cause only the cool kids know postfix!!!!!)
Scott Silva wrote:
I think postfix has added milter support lately. I use sendmail, but only because it is what I know, and I don't want to muck about in a working mailserver. Someday I will learn postfix, right after I get those washboard abs and die out the grey! ;-) ('cause only the cool kids know postfix!!!!!)
dont waste time, pass postfix, go straight to exim...
- K
Karanbir Singh spake the following on 8/30/2006 11:30 AM:
Scott Silva wrote:
I think postfix has added milter support lately. I use sendmail, but only because it is what I know, and I don't want to muck about in a working mailserver. Someday I will learn postfix, right after I get those washboard abs and die out the grey! ;-) ('cause only the cool kids know postfix!!!!!)
dont waste time, pass postfix, go straight to exim...
- K
I see flames!!!
I wish I could find an UNBIASED comparison of the main MTA's. I just seem to find ones that slant to the one that just happens to be used by the writer.
Most of what I have seen says stay away from Qmail, but only because of lack of maintenance by the programmer. Everything else just seems to orbit around Exim, Postfix, and Sendmail. I don't want to learn all three, so ... Hello again sendmail!
-> Scott uttered: -> I wish I could find an UNBIASED comparison of the main MTA's. -> I just seem to find ones that slant to the one that just happens to be -> used by -> the writer. -> -> Most of what I have seen says stay away from Qmail, but only because of -> lack -> of maintenance by the programmer. Everything else just seems to orbit -> around -> Exim, Postfix, and Sendmail. I don't want to learn all three, so ... -> Hello again sendmail! -> ->
I was fixed on sendmail as an isp for 10 years. Worked reasonably well. I got away for a coupla years.
When I got back into it, I chose and switched to qmail because as a Linux person since pre Ver 1.x I wanted to know to the best of my ability what is going on and I wanted the ease of my clients admin'ing their own virtual mail servers etc blah.
The phone calls I like are sales calls!!! :-)
So, I was tired of doing of having employess do the work for clients when they can do it themselves.
qmail and some other packages did that in a excellent way and I only had to pay my time in research and implementation to get it where I wanted it i.e. roll your own and save. I didn't have *unlimited* funds to do it even though I could have purchased virtually any commercial solution within reason.
I watched www.qmailrocks.org Redhat version on the mailing list for a little less than a year before I wanted to totally jump away from sendmail. After a few months of implementation and testing live Centos servers were forged with these packages.
The thing is, it keeps getting better and better. I almost didn't use it until some huge changes in the qmail world started coming thru plus people on this list (and others) keep programming and adjusting and sharing and you name it etc.
I wanted to stay with sendmail yet couldn't find a good remote virtual server administration like qmailadmin.
Qmail personalities are not an issue.
Yes, there is a lot to think about and setup and admin yet it can be made extremely easy to setup and admin if you are willing to spend the time. There is a toaster site that can do it in a little time as well. You can make your setup a toaster too once you know how.
So, after you do it and understand, you can script your own www.Qmailrocks.org setup so that you go from a bare centos server to a awesome custom configured Centos qmail server in a coupla hours (less if you wish) with all the bells and whistles the way YOU want it and NOT the way SOMEONE else does that you have to adapt to.... forget that.
Mostly I roll my own RPMs for everything too. Why wait? :-)
There are other websites.
http://www.qmailrocks.org/ http://cr.yp.to/qmail.html http://www.inter7.com/vpopmail/ http://www.inter7.com/qmailadimin/ http://www.courier-mta.org/imap/ http://qmail-scanner.sourceforge.net/ http://spamassassin.apache.org/ http://www.clamav.net/ http://www.squirrelmail.org/
other useful sites:
http://www.qmail.org/ http://www.lifewithqmail.org/ http://qmail.jms1.net/ http://www.qmailtoaster.com/
thanks for your time...
- rh
-- Abba Communications Computer & Internet Services (509) 624-7159 - www.abbacomm.net
On Wed, 2006-08-30 at 12:13 -0700, Scott Silva wrote:
Karanbir Singh spake the following on 8/30/2006 11:30 AM:
Scott Silva wrote:
<snip>
dont waste time, pass postfix, go straight to exim...
- K
I see flames!!!
I wish I could find an UNBIASED comparison of the main MTA's. I just seem to find ones that slant to the one that just happens to be used by the writer.
Not possible? Those who bother to reply have probably had a variety of experience and settled on what they thought was best for their situation. For them to express anything other than recommendations (mostly) for their selection would be .. illogical ... Kirk!
And those who replied and had not selected one and/or had not practical experience but only analytical comparisons are not the ones you want to here from.
Ergo, it is not possible for you to get an unbiased recommendation in which you can have confidence.
Most of what I have seen says stay away from Qmail, but only because of lack of maintenance by the programmer. Everything else just seems to orbit around Exim, Postfix, and Sendmail. I don't want to learn all three, so ... Hello again sendmail!
However, as with any tool selection, they may all be the *right* one. It depends on the job to be done. And level of expertise and/or effort required is an important consideration, along with the more traditional technical considerations.
Often the answers are wrong only because the original question was incomplete. The question should include such phrases as "... level of expertise...", "... volume approaching ...", "... remotely administered or by the user ...", etc.
Of course, I found over the years that my greatest contribution was in discovering the right questions. Everybody and his brother had the right answers, but I was just plain ignorant.
MHO!
On Wed, 2006-08-30 at 17:04 -0400, William L. Maltby wrote:
I see flames!!!
I wish I could find an UNBIASED comparison of the main MTA's. I just seem to find ones that slant to the one that just happens to be used by the writer.
Not possible? Those who bother to reply have probably had a variety of experience and settled on what they thought was best for their situation. For them to express anything other than recommendations (mostly) for their selection would be .. illogical ... Kirk!
The problem is that MTA's typically have to mesh with a lot of other tools, so you'll see people raving about one vs. the other because of some almost-unrelated management wrapper or it's ability to do SMTP AUTH against some oddball password database. Also, if you are looking for long experience, sendmail is pretty much the only one that has been around for what I'd call a long time - but you can't really compare its capabilities before the milter interface stabilized with the current version. A lot of people got things working with qmail or postfix and haven't looked at the new capabilities of sendmail with MimeDefang as a milter.
Les Mikesell wrote:
On Wed, 2006-08-30 at 17:04 -0400, William L. Maltby wrote:
I see flames!!!
I wish I could find an UNBIASED comparison of the main MTA's. I just seem to find ones that slant to the one that just happens to be used by the writer.
Not possible? Those who bother to reply have probably had a variety of experience and settled on what they thought was best for their situation. For them to express anything other than recommendations (mostly) for their selection would be .. illogical ... Kirk!
The problem is that MTA's typically have to mesh with a lot of other tools, so you'll see people raving about one vs. the other because of some almost-unrelated management wrapper or it's ability to do SMTP AUTH against some oddball password database. Also, if you are looking for long experience, sendmail is pretty much the only one that has been around for what I'd call a long time - but you can't really compare its capabilities before the milter interface stabilized with the current version. A lot of people got things working with qmail or postfix and haven't looked at the new capabilities of sendmail with MimeDefang as a milter.
maybe because they don't like the idea of perl as an inbetween. how large are your sendmail processes?
sendmail's fork a process per connection + perl in some situations make people have nightmares whereas the same could be done in a much more efficient manner especially with postfix. Most probably won't think of mysql as an oddball password database. Maybe a cdb one.
sendmail + mimedefang is overrated. Now that postfix 2.3 is the stable line now...postfix + mimedefang will probably be interesting depending on how it is done...
On Wed, 2006-08-30 at 22:18, Feizhou wrote:
The problem is that MTA's typically have to mesh with a lot of other tools, so you'll see people raving about one vs. the other because of some almost-unrelated management wrapper or it's ability to do SMTP AUTH against some oddball password database. Also, if you are looking for long experience, sendmail is pretty much the only one that has been around for what I'd call a long time - but you can't really compare its capabilities before the milter interface stabilized with the current version. A lot of people got things working with qmail or postfix and haven't looked at the new capabilities of sendmail with MimeDefang as a milter.
maybe because they don't like the idea of perl as an inbetween. how large are your sendmail processes?
Sendmail chats on sockets with the milter process and thus doesn't become any larger. And the milters can be written in any language. Mimedefang uses a combination of C and perl. If you are going to run spamassassin you are stuck with perl regardless and that's going to be your slow step anyway so you might as well have it pre-loaded and use it as the parser for your control steps.
sendmail's fork a process per connection + perl in some situations make people have nightmares whereas the same could be done in a much more efficient manner especially with postfix.
Like I said, most people don't bother to understand how it currently works. Sendmail runs one milter process which all the sendmail processes use via socket connections. In MimeDefang's case the controlling milter is written in C and runs multi-threaded, multiplexing connections to some small number of slaves that do the work in single-threaded perl. There is no extra fork per sendmail involved and a much smaller number of perl processes than sendmails. Also, the multiplexer manages the slave processes by restarting them if they crash or exceed memory limits and will inform sendmail to retry on timeouts.
Any way you look at it, it is a much nicer design than making the receiving process handle everything itself or fork additional processes per message or run.
most probably won't think of mysql as an oddball password database. Maybe a cdb one.
If you are used to having normal users on a unix-like system you would.
sendmail + mimedefang is overrated. Now that postfix 2.3 is the stable line now...postfix + mimedefang will probably be interesting depending on how it is done...
We can pick up the discussion when/if some large volume users equivalent to those on the MimeDefang list start using it. A big value in using existing, working software is that you can take advantage of other's previous experience with it.
Sendmail chats on sockets with the milter process and thus doesn't become any larger. And the milters can be written in any language. Mimedefang uses a combination of C and perl. If you are going to run spamassassin you are stuck with perl regardless and that's going to be your slow step anyway so you might as well have it pre-loaded and use it as the parser for your control steps.
Hence why I have spamassassin used to filter after the mail is queued. Do you make mimedefang run spamassassin on all your mails before queueing?
sendmail's fork a process per connection + perl in some situations make people have nightmares whereas the same could be done in a much more efficient manner especially with postfix.
Like I said, most people don't bother to understand how it currently works. Sendmail runs one milter process which all the sendmail processes use via socket connections. In MimeDefang's case the controlling milter is written in C and runs multi-threaded, multiplexing connections to some small number of slaves that do the work in single-threaded perl. There is no extra fork per sendmail involved and a much smaller number of perl processes than sendmails. Also, the multiplexer manages the slave processes by restarting them if they crash or exceed memory limits and will inform sendmail to retry on timeouts.
Okay, I get it. lotsa sendmail processes < (one socket per sendmail process) > single milter process < ? > perl processes
Any way you look at it, it is a much nicer design than making the receiving process handle everything itself or fork additional processes per message or run.
postfix does it similarly. smtpd does not handle everything itself. You've got cleanup and trivial-rewrite to handle some stuff for smtpd and they also pass some stuff out to proxymap to handle. Really light stuff are handled by smtpd and the more heavy stuff is parceled out to cleanup, trivial-rewrite and proxymap to handle.
sendmail + mimedefang is overrated. Now that postfix 2.3 is the stable line now...postfix + mimedefang will probably be interesting depending on how it is done...
We can pick up the discussion when/if some large volume users equivalent to those on the MimeDefang list start using it. A big value in using existing, working software is that you can take advantage of other's previous experience with it.
sure.
On Fri, 2006-09-01 at 01:30, Feizhou wrote:
Sendmail chats on sockets with the milter process and thus doesn't become any larger. And the milters can be written in any language. Mimedefang uses a combination of C and perl. If you are going to run spamassassin you are stuck with perl regardless and that's going to be your slow step anyway so you might as well have it pre-loaded and use it as the parser for your control steps.
Hence why I have spamassassin used to filter after the mail is queued. Do you make mimedefang run spamassassin on all your mails before queueing?
Inbound only, and after the other faster checks (virus, etc.) that might cause rejection are done. The advantage of scanning during the SMTP conversation is that you can still reject with a message that would find its way back to a legitimate sender without having to construct the bounce yourself.
Okay, I get it. lotsa sendmail processes < (one socket per sendmail process) > single milter process < ? > perl processes
Actually you can have multiple milter processes if you want, but MimeDefang handles about everything. Also, sendmail has separate conversations with the milter(s) for each operation which MimeDefang might hand off to different slaves. The side effect is that you don't block on some other long-running process unless you are out of slaves but you also can't count on globals that you set in one step (checking the sender or recipients) to be available in later steps - but MimeDefang passes most of the information you need each time and has dropped a complete copy of the message broken out into its MIME components in files where the programs can find them (hence the name and the advantage of running multiple scanners under its control).
Hence why I have spamassassin used to filter after the mail is queued. Do you make mimedefang run spamassassin on all your mails before queueing?
Inbound only, and after the other faster checks (virus, etc.) that might cause rejection are done. The advantage of scanning during the SMTP conversation is that you can still reject with a message that would find its way back to a legitimate sender without having to construct the bounce yourself.
I don't bounce the mail. I just tag the mail as a possible spam. Different ways of handling this :)
Okay, I get it. lotsa sendmail processes < (one socket per sendmail process) > single milter process < ? > perl processes
Actually you can have multiple milter processes if you want, but MimeDefang handles about everything. Also, sendmail has separate conversations with the milter(s) for each operation which MimeDefang might hand off to different slaves. The side effect is that you don't block on some other long-running process unless you are out of slaves but you also can't count on globals that you set in one step (checking the sender or recipients) to be available in later steps - but MimeDefang passes most of the information you need each time and has dropped a complete copy of the message broken out into its MIME components in files where the programs can find them (hence the name and the advantage of running multiple scanners under its control).
So you are saying that mimedefang is not reliable if not provided enough resources. What happens to the mail then? temporary reject or let through?
Quoting Feizhou feizhou@graffiti.net:
So you are saying that mimedefang is not reliable if not provided enough resources. What happens to the mail then? temporary reject or let through?
Whatever way you configure Sendmail and MIMEDefang.
In Sendmail, you can configure it to either permanently reject, temporary reject or just pass through in case of filter failure and/or if filter gets stuck (there's configurable timeout). These are per-filter settings, so if you for example have AV scan in one filter, and spam scan in another filter, you can configure Sendmail to temp-fail if AV filter fails, and to pass through if spam filter fails.
MIMEDefang also comes with its own set of configuartion direcitves. You can configure maximum number of slaves, the minimum number of slaves to keep around, and what to do if the slave misbehaves (for example, doesn't return in configurable amount of time). And of course, in your filter script (the Perl code you are supposed to write) you can take whatever action you like in case your script detects an error condition.
Some of this settings you need to coordinate a bit. For example, there is no point in telling MIMEDefang to wait 1 minute for a slave to finish processing, and than telling Sendmail to wait only 10 seconds for MIMEDefang (you want timeout in Sendmail to be longer than timeout in MIMEDefang).
Another word of warnning if using SpamAssassin from MIMEDefang (or from anything else). MIMEDefang alone is relatively lightweight and fast (for a Perl thingie). However, SpamAssassin is a memory pig. Very fat pig (to put it politely). The kind of a pig you would take to a farm show. Like any other fat boy, it can't run fast either. Even if you are scanning only small emails with it (never attempt to pass something larger than 100-200k through SpamAssassin). SpamAssassin can easily take anywhere between 10 to 20 megs of memory just to process relatively small message (or even more, depending how large are emails that you pass through it). When calculating how many slaves you allow MIMEDefang to start, keep this in mind and make sure you don't exceed the amount of physical memory in your server. You do not want SpamAssassin processes fighting for memory. It can kill your email server. Imagine ten really fat pigs (the real live pigs, flash and blood, curly tails, covered in mud and flies, burping, farting and everything) stacked on your server. You've got an idea. If running on a slower machine, you'll also want to adjust timeouts in Sendmail and MIMEDefang to give SpamAssassin enough time to chew the message.
If using SpamAssassin from MIMEDefang, I usually set maximum number of requests per slave in MIMEDefang low. Even if SpamAssassin allocates too much memory in some slave when processing some nasty email, the slave will be replaced by a new one relatively quickly and memory will be released back to the system. It is kind of a tradeoff setting. You don't want slaves being terminated and resetarted too often. And you don't want fat ones lingering around for too long either.
You'd also tell MIMEDefang to use embeded Perl, so the Perl initializes only once. This can help performance dramatically.
When using SpamAssassin, you might also want to look into Sendmail's throttle and alike option(s). What it does is that it controlls the maximum number of connections per second. If the number is exceeded, it will simply postpone the processing of incomming connection (it will not reject anything, just postpone the handling of incomming connection). However, when using this option, if you experience the storm of incomming connections, the postponing might cause the clients to time out (howerver, if it gets that bad that clients start timing out, you are most likely under the DoS attack anyhow). Older Sendmails have only global throttle option, and you can specify only integer amount of connections per second. Newer Sendmails can do it per-IP address (which helps if you are dealing with only single bad host), and you can specify number of messages per time interval (so you'r more flexible here too). This might help prevent having too many SpamAssassin scans occuring simultaniously and fighting for CPU and memory.
On Fri, 2006-09-01 at 22:48 +0800, Feizhou wrote:
Hence why I have spamassassin used to filter after the mail is queued. Do you make mimedefang run spamassassin on all your mails before queueing?
Inbound only, and after the other faster checks (virus, etc.) that might cause rejection are done. The advantage of scanning during the SMTP conversation is that you can still reject with a message that would find its way back to a legitimate sender without having to construct the bounce yourself.
I don't bounce the mail. I just tag the mail as a possible spam. Different ways of handling this :)
I mostly do that too, but I've tweaked some of the spamassassin rules (vi*gra, etc.) to extremely high values, then MimeDefang rejects on values that can only be reached that way with lower values passed in a header tag for user processing.
Actually you can have multiple milter processes if you want, but MimeDefang handles about everything. Also, sendmail has separate conversations with the milter(s) for each operation which MimeDefang might hand off to different slaves. The side effect is that you don't block on some other long-running process unless you are out of slaves but you also can't count on globals that you set in one step (checking the sender or recipients) to be available in later steps - but MimeDefang passes most of the information you need each time and has dropped a complete copy of the message broken out into its MIME components in files where the programs can find them (hence the name and the advantage of running multiple scanners under its control).
So you are saying that mimedefang is not reliable if not provided enough resources.
No, I didn't say anything even close to suggesting that - and I'm curious as to why you imagined I did. What I said was that different phases of the filtering are likely to be handled in different processes so your programming should be done accordingly.
What happens to the mail then? temporary reject or let through?
Sendmail will tempfail if it can't connect to it's configured milter(s) or it gets a socket error and there are 4 timeouts that can be set per-milter if you don't like the defaults. And of course sendmail has its own configurable limit on the number of sendmail children.
MimeDefang will tell sendmail to tempfail if a slave runs out of resources or crashes - like a virus scanner unpacking one of those zip-of-death archives might if it is configured incorrectly. The mimedefang-multiplexor process controls the slaves with a lot of options that I've never had to change from the defaults.
Actually you can have multiple milter processes if you want, but MimeDefang handles about everything. Also, sendmail has separate conversations with the milter(s) for each operation which MimeDefang might hand off to different slaves. The side effect is that you don't block on some other long-running process unless you are out of slaves but you also can't count on globals that you set in one step (checking the sender or recipients) to be available in later steps - but MimeDefang passes most of the information you need each time and has dropped a complete copy of the message broken out into its MIME components in files where the programs can find them (hence the name and the advantage of running multiple scanners under its control).
So you are saying that mimedefang is not reliable if not provided enough resources.
No, I didn't say anything even close to suggesting that - and I'm curious as to why you imagined I did. What I said was that different phases of the filtering are likely to be handled in different processes so your programming should be done accordingly.
Well, I did not quite get your 'can't count on globals' which you put after 'unless you are out of slaves'. That gave the impression that something was not on if you run out of resources. Are the globals you talk about ones you set or sendmail sets?
On Fri, 2006-09-01 at 18:34, Feizhou wrote:
No, I didn't say anything even close to suggesting that - and I'm curious as to why you imagined I did. What I said was that different phases of the filtering are likely to be handled in different processes so your programming should be done accordingly.
Well, I did not quite get your 'can't count on globals' which you put after 'unless you are out of slaves'. That gave the impression that something was not on if you run out of resources. Are the globals you talk about ones you set or sendmail sets?
I meant globals that you set in custom filters. It might seem like you could set some values when the sender or the recipient list is passed through the milter interface, then check them later when processing the content. However, due to the way mimedefang optimizes the processing, the different stages of a single message may be handled by different instances of perl slave processes.
I wish I could find an UNBIASED comparison of the main MTA's. I just seem to find ones that slant to the one that just happens to be used by the writer.
Not possible? Those who bother to reply have probably had a variety of experience and settled on what they thought was best for their situation. For them to express anything other than recommendations (mostly) for their selection would be .. illogical ... Kirk!
It is possible. The problem is whether others will allow their prejudices to cloud whatever is recommended. eg: some would never install qmail even if it is the best for that particular situation and tell everybody else so too.
And those who replied and had not selected one and/or had not practical experience but only analytical comparisons are not the ones you want to here from.
Oh yeah. For that reason I have nothing to say about exim.
Ergo, it is not possible for you to get an unbiased recommendation in which you can have confidence.
let's see.
On Wed, 2006-08-30 at 22:23, Feizhou wrote:
I wish I could find an UNBIASED comparison of the main MTA's. I just seem to find ones that slant to the one that just happens to be used by the writer.
Not possible? Those who bother to reply have probably had a variety of experience and settled on what they thought was best for their situation. For them to express anything other than recommendations (mostly) for their selection would be .. illogical ... Kirk!
It is possible. The problem is whether others will allow their prejudices to cloud whatever is recommended. eg: some would never install qmail even if it is the best for that particular situation and tell everybody else so too.
That's kind of the point of giving advice on mail lists. If you've had a bad experience with something you can spare other people similar pain. And it is pretty much guaranteed that you'll have a bad experience with a stock qmail system the first time you get a dictionary-type attack to your domain or if you send a lot of group messages over low-bandwidth links.
Les Mikesell wrote:
On Wed, 2006-08-30 at 22:23, Feizhou wrote:
I wish I could find an UNBIASED comparison of the main MTA's. I just seem to find ones that slant to the one that just happens to be used by the writer.
Not possible? Those who bother to reply have probably had a variety of experience and settled on what they thought was best for their situation. For them to express anything other than recommendations (mostly) for their selection would be .. illogical ... Kirk!
It is possible. The problem is whether others will allow their prejudices to cloud whatever is recommended. eg: some would never install qmail even if it is the best for that particular situation and tell everybody else so too.
That's kind of the point of giving advice on mail lists. If you've had a bad experience with something you can spare other people similar pain. And it is pretty much guaranteed that you'll have a bad experience with a stock qmail system the first time you get a dictionary-type attack to your domain or if you send a lot of group messages over low-bandwidth links.
and for the same reason I don't recommend qmail on an MX host without patches. There are other uses for an mta other than receiving mail for a domain.
Scott Silva wrote:
I think postfix has added milter support lately. I use sendmail, but only because it is what I know, and I don't want to muck about in a working mailserver. Someday I will learn postfix, right after I get those washboard abs and die out the grey! ;-) ('cause only the cool kids know postfix!!!!!)
dont waste time, pass postfix, go straight to exim...
I see flames!!!
I wish I could find an UNBIASED comparison of the main MTA's. I just seem to find ones that slant to the one that just happens to be used by the writer.
no flames at all - your statement was that you are familiar with sendmail and use that everywhere - and you were thinking of moving to Postfix.... I'd recommend, for someone used to sendmail and someone who likes the flexibility and power of sendmail - to not waste time with Postfix at all, and go straight to exim.
Exim is very powerfull, and has excellent docs as well. For a lot of people, moving from postfix, the setup and configure process is more involved than postfix might be. Think of it as being a midway between sendmail and postfix, its not as easy to setup as postfix - but once you get your head around it, its a breeze.
I admit it takes no longer to setup than postfix, and has a lot more flexibility and customisation options than postfix. Some even argue that its all of sendmail's good bits, without all the sendmail bad bits.
For the sake of disclosure, I use postfix and exim - postfix where things need to stay simple and small setup's - Exim when things get funky!
Ralph Angenendt wrote:
Jaymz Ringler wrote:
A simple solution if you have an extra machine.. install qmail on a new box... put it into your DMZ to collect mail. You then set a simple smtproute to forward all mail to your inner mail server's ip.
qmail is outdated. qmail doesn't check for available users before accepting a mail and thus always sends bounces. qmail isn't available for centos. qmail hasn't seen any updates since way back when. qmail can only be made into a more modern MTA with tons of patches.
nothing beats qmail's dot-qmail delivery mechanism. I would not put plain qmail on an mx host but I definitely would not write it off for other mail uses.
-
Aleksandar Milivojevic -
Alexander Dalloz -
Andy Green -
chrism＠imntv.com -
Email Lists -
Feizhou -
Jaymz Ringler -
Kanwar Ranbir Sandhu -
Karanbir Singh -
Les Mikesell -
Peter Eisch -
Ralph Angenendt -
Rodrigo Barbosa -
Scott Silva -
William L. Maltby