All,
I'm putting up a new email server, but I can't afford downtime when I switch servers. So, I've been writing a script that will take a CentOS install and turn it into a secure email server with Spam Assassin, Amavis-new, ClamAV and external relay using SASL authentication.
The script is still *very* raw, and I'm relatively new to bash script writing. I still have some areas of work to do:
- I need to incorporate the "alternatives" command to switch from sendmail to postfix, instead of using the redhat-switch-mail and changing it via gui.
- I need to figure out how to automate the certificate generation. Right now I still have to type in a password at the prompt, as well as hit <enter> to accept the defaults
- Some of my sed commands are kludgy. I'm still learning the ins and outs of sed
- I have *no* idea whether or not Vipul's Razor is working or not.
- I need to fix the yum "extras" section.
- I need to improve the iptables handling.
Anyway, if you guys'd like to take a look and make comments, I'm open to suggestions and whatnot.
It's at:
http://www.benjamin.weiss.name/mail-server.sh
Thanks!
Ben
On Wed, 8 Dec 2004, Benjamin J. Weiss wrote:
All,
I'm putting up a new email server, but I can't afford downtime when I switch servers. So, I've been writing a script that will take a CentOS install and turn it into a secure email server with Spam Assassin, Amavis-new, ClamAV and external relay using SASL authentication.
The script is still *very* raw, and I'm relatively new to bash script writing. I still have some areas of work to do:
If you are new to bash, or programming in general, I think you might find learning python is more specifically suited to the task, and more powerful.
However bash can work quite nicely for a lot of things ;-)
- I need to figure out how to automate the certificate generation. Right
now I still have to type in a password at the prompt, as well as hit <enter> to accept the defaults
have you looked at "expect"? It is designed for this sort of operation. There are modules for python or perl both, and I imagine you could also get it to work from the shell. We have scripts for apache ssl cert generation that use pyexpect.
- Some of my sed commands are kludgy. I'm still learning the ins and outs
of sed
If you are doing search/replace filters, many sed commands can be replaced from the shell with perl's re interpreter by doing: perl -pi.orig -e 's/search-this/replace-with-this/' filename which is more overhead, buts lets you make a backup file (named filename.orig), and gives you the power of perl's re engine, which is arguably more forgiving in syntax than sed.
I have *no* idea whether or not Vipul's Razor is working or not.
I need to fix the yum "extras" section.
I need to improve the iptables handling.
Anyway, if you guys'd like to take a look and make comments, I'm open to suggestions and whatnot.
It's at:
http://www.benjamin.weiss.name/mail-server.sh
Thanks!
Ben
CentOS mailing list CentOS@caosity.org http://lists.caosity.org/mailman/listinfo/centos
On Wed, 8 Dec 2004 09:14:48 -0600 (CST), Benjamin J. Weiss benjamin@birdvet.org wrote:
I'm putting up a new email server, but I can't afford downtime when I switch servers. So, I've been writing a script that will take a CentOS install and turn it into a secure email server with Spam Assassin, Amavis-new, ClamAV and external relay using SASL authentication.
Hi Benjamin. I'm sure your script will ultimately work OK, but your statement that you "can't afford downtime when you switch servers" suggests you might want to think about how exactly you're going to do the switch, rather than how quickly you can configure the new server.
It can be very helpful to write up a list of tests that you can perform after you cut over that will exercise all of the functionality you want to provide and will tell you whether you need to roll back or not. For example, "Send a mail locally to an external account - should succeed" "Send a mail from offsite to a local address - should get there" "Relay from offsite to a remote address - should be denied" "Relay from offsite after auth to a remote address - should succeed" Then once you do the cutover you have a very systematic way of knowing that everything is working OK.
If you have two physical machines, the old one and the new one, you should take your time and set up the new one by hand (rather than by script), and maybe plug it in side-by-side with the old one, without the public IP address that the world is talking to. When you're ready to cut over, 'ifdown' the old one, 'ifup' the new one, and run your tests. If your tests fail, depending on how bad the failure is, you can decide to roll back to the old server by reversing the process or fix things "live".
You really want to do everything possible to avoid cutting off your rollback path. The worst situation is when you're commited to the new server but it's not quite working, and you're frantically debugging shell (or worse, sed!) syntax trying to dig yourself out of the hole.
Sorry this is not specific to centos, but from reading your email I had flashbacks to some ugly mail server cutovers from the past... hopefully this helps you avoid some of the pitfalls -- good luck!
-=Eric
On Wed, 8 Dec 2004, Eric Sorenson wrote:
On Wed, 8 Dec 2004 09:14:48 -0600 (CST), Benjamin J. Weiss benjamin@birdvet.org wrote:
I'm putting up a new email server, but I can't afford downtime when I switch servers. So, I've been writing a script that will take a CentOS install and turn it into a secure email server with Spam Assassin, Amavis-new, ClamAV and external relay using SASL authentication.
Hi Benjamin. I'm sure your script will ultimately work OK, but your statement that you "can't afford downtime when you switch servers" suggests you might want to think about how exactly you're going to do the switch, rather than how quickly you can configure the new server.
It can be very helpful to write up a list of tests that you can perform after you cut over that will exercise all of the functionality you want to provide and will tell you whether you need to roll back or not. For example, "Send a mail locally to an external account - should succeed" "Send a mail from offsite to a local address - should get there" "Relay from offsite to a remote address - should be denied" "Relay from offsite after auth to a remote address - should succeed" Then once you do the cutover you have a very systematic way of knowing that everything is working OK.
Good idea. :) I hadn't formally written it down, just in my head.
If you have two physical machines, the old one and the new one, you should take your time and set up the new one by hand (rather than by script), and maybe plug it in side-by-side with the old one, without the public IP address that the world is talking to. When you're ready to cut over, 'ifdown' the old one, 'ifup' the new one, and run your tests. If your tests fail, depending on how bad the failure is, you can decide to roll back to the old server by reversing the process or fix things "live".
My testing methodology has been a bit different. I have an evaluation copy of VMWare that I downloaded, and I've been using that. I installed CentOS on it, then took a snapshot. I'll do part of the script, then go tot the VM, download and run the script. Then I'll see what went wrong, reset the VM to the snapshot, and do it again.
You really want to do everything possible to avoid cutting off your rollback path. The worst situation is when you're commited to the new server but it's not quite working, and you're frantically debugging shell (or worse, sed!) syntax trying to dig yourself out of the hole.
Yep, I agree! Not only am I testing with the VMWare, but I do have a new box. I'm going to set it up this weekend on a domain that I have but don't use much, and set it up as the mail server for that domain. Then I'll migrate my old email from mbox to Maildir, subscribe to a couple of lists, use the squirrelmail and thunderbird clients, etc. I plan on testing for about a week. Then, when I'm satisfied, I'll do your suggestion.
The main reason for the script was because I didn't want to have to type the same stuff over and over and over while figuring out the process of getting the server running. I just figured that it might be helpful to others, so I'm sharing it in the spirit of open source. :) Plus, I figure that if others take a look at what I'm writing, I might get some good suggetions on how to improve the code. I've been programming for awhile, but never in bash before.
Sorry this is not specific to centos, but from reading your email I had flashbacks to some ugly mail server cutovers from the past... hopefully this helps you avoid some of the pitfalls -- good luck!
Hey, no prob! I apreciate the input!
Thanks!
Ben
Eric Sorenson wrote:
If you have two physical machines, the old one and the new one, you should take your time and set up the new one by hand (rather than by script), and maybe plug it in side-by-side with the old one, without the public IP address that the world is talking to. When you're ready to cut over, 'ifdown' the old one, 'ifup' the new one, and run your tests. If your tests fail, depending on how bad the failure is, you can decide to roll back to the old server by reversing the process or fix things "live".
I can second this notion. If you have the extra hardware doing the network IP switch is the best way to go. I try to have every machine with two IP addresses. One for the physical machine and one for the "service". So 192.168.1.10 is the physical mail server but 192.168.1.20 is the "mail.domain.com" address. That way I can setup a new server, configure/test it. Then when it comes time to switching over, I just turn off the alias on the old server, and turn it on on the new server.
Our company went from a sendmail server to a postfix/clam/spamassassin setup over night using this method. The only difference to the staff was all their spam was filtered.
--Ajay
On Wed, Dec 08, 2004 at 09:14:48AM -0600, Benjamin J. Weiss wrote:
All,
I'm putting up a new email server, but I can't afford downtime when I switch servers. So, I've been writing a script that will take a CentOS install and turn it into a secure email server with Spam Assassin, Amavis-new, ClamAV and external relay using SASL authentication.
I'm also doing a new server at the moment. Postfix, spamassassin, courier-imap, ClamAV, ClamSMTP, Maildrop, Squirrelmail.
- I have *no* idea whether or not Vipul's Razor is working or not.
I suppose you use this at spamassassin time. Add '-D' to your spamd options (/etc/init.d/spamassassin) and see in your logfiles. I maded RPMs myself from the spamassassin tar and noticed the dependency on the perl-DB_File package was not specified in the spamassassin spec. You need that however (at least if you are doing spamassassin at local delivery time like I do)
Cheers,
On Thu, 9 Dec 2004, Henk van Lingen wrote:
On Wed, Dec 08, 2004 at 09:14:48AM -0600, Benjamin J. Weiss wrote:
All,
I'm putting up a new email server, but I can't afford downtime when I switch servers. So, I've been writing a script that will take a CentOS install and turn it into a secure email server with Spam Assassin, Amavis-new, ClamAV and external relay using SASL authentication.
I'm also doing a new server at the moment. Postfix, spamassassin, courier-imap, ClamAV, ClamSMTP, Maildrop, Squirrelmail.
- I have *no* idea whether or not Vipul's Razor is working or not.
I suppose you use this at spamassassin time. Add '-D' to your spamd options (/etc/init.d/spamassassin) and see in your logfiles. I maded RPMs myself from the spamassassin tar and noticed the dependency on the perl-DB_File package was not specified in the spamassassin spec. You need that however (at least if you are doing spamassassin at local delivery time like I do)
See, that's the problem. amavisd-new shows in the /var/log/maillog that it's picking up spam assassin, whether the spamd daemon is running or not. It seems to flag spam whether the spamd is running or not. And I don't see any spam assassin headers that I recognize from earlier versions of SA.
So, I don't know if amavis is just running the spam assassin code directly, or what. And I haven't been able to find much documentation on the subject.
On another note, I went with dovecot instead of courier, since dovecot supports Maildir, which I've been wanting to use. my mbox files are *huge*.
I'm not familiar with Maildrop, is it a procmail alternative?
Thanks!
Ben
On Thu, Dec 09, 2004 at 07:23:07AM -0600, Benjamin J. Weiss wrote:
- I have *no* idea whether or not Vipul's Razor is working or not.
I suppose you use this at spamassassin time. Add '-D' to your spamd options (/etc/init.d/spamassassin) and see in your logfiles. I maded RPMs myself from the spamassassin tar and noticed the dependency on the perl-DB_File package was not specified in the spamassassin spec. You need that however (at least if you are doing spamassassin at local delivery time like I do)
See, that's the problem. amavisd-new shows in the /var/log/maillog that it's picking up spam assassin, whether the spamd daemon is running or not. It seems to flag spam whether the spamd is running or not. And I don't see any spam assassin headers that I recognize from earlier versions of SA.
So, I don't know if amavis is just running the spam assassin code directly, or what. And I haven't been able to find much documentation on the subject.
I've no experience with amavis but if I understand you correctly I suppose amavis calls spamassassin and doesn't use spamc. I would change that for performance reasons.
On another note, I went with dovecot instead of courier, since dovecot supports Maildir, which I've been wanting to use. my mbox files are *huge*.
Courier also can handle Maildir, even more, I don't think it can do mbox.
I'm not familiar with Maildrop, is it a procmail alternative?
Yes: http://www.courier-mta.org/maildrop/
( I'm currently finding out how the get it bounce messages to over filesystemquota users. It just tempfails and I don't want my queues filled this way :-( )
Cheers,
-
Ajay Sharma -
Benjamin J. Weiss -
Eric Sorenson -
Henk van Lingen -
Ryan Sweet