On Sat, Aug 22, 2009, Dave wrote: >On Sat, Aug 22, 2009 at 6:49 AM, Bill Campbell<centos at celestial.com> wrote: >> I review daily reports from over 50 systems every morning, checking changes >> found, usually taking no more than 10 minutes a day. The key is to keep >> the reports simple, and to make updating easy (and to have procedures that >> monitor systems to be sure they's still alive and reporting in). > >So how do you track the inevitable changes? Not saying you can't, just >curious. For me, when I look at a batch of changes, some of them are >obviously stuff I've done, other stuff not so obvious. I also filter >reports through a script that sort of does a diff and makes an attempt >to limit the boilerplate. Sometimes it is a bit too terse. First off, we don't allow automatic updates on most systems, much preferring to do them manually making it pretty easy to refresh the comparison database immediately after the update is complete. The odds that a cracker will get in and do their dirty deeds while this are going on are pretty low, and can probably be ignored. We handle pretty much all server stuff under the OpenPKG portable package management system so things like spamassassin, amavisd, clamav, and postfix are not the distribution versions, but those from OpenPKG (which are generally updated more quickly then the distribution's). A typical occurrence will be that we get an e-mail saying that clamav is out of date from the nightly freshclam update, I will pick up the new sources, update the OpenPKG SRPM for it, and deploy it 40 or so systems running it, and expect to see a corresponding set of notices the next morning that files under clamav have changed. The clusterssh program makes this sort of thing much more efficient as one can execute shell commands on multiple systems simultaneously. >> We create a file system initially, the same size as ``/'', and make a copy >> of ``/'' in it identical except for the /etc/fstab entry. This is not >> mounted in normal operations, but the system can be booted from it to get >> to a clean system. > >Wow, elaborate. How do you protect this file system from intruders? >Exterrnal and powerred off? That's one way to do it. We also run a fair number of Linux servers under VMware so periodic snapshots and backups simplify the task. I have not seen many successful cracks of Linux boxes that we have configured from scratch. Some basic things can be done to minimize the chances of cracks. + Create the baseline for intrusion detection tools before putting the syste on line, and monitor it daily. + Configure openssh to refuse password authentication requiring authorized_keys access. + Configure openssh with tcp_wrappers support, restricting access by IP address and/or domain names. I consider this absolutely mandatory if one needs to all username and password authentication. + Use fail2ban or similar techniques to quickly block IP addresses that are found probing the system (don't forget to look at POP and IMAP logs for failed login attempts). + Use /bin/false as the standard shell for accounts that don't have good reason for shell access. This does not affect e-mail or most services that a typical ISP customer needs. + Use OpenVPN for access. This works well even when in hotels with NAT firewalls, and is not easily hacked anonymously. + Restrict access of webmin and usermin to local networks so they are not vulnerable to outside attack. These services are available to people outside connecting with OpenVPN. + Restrict webmail, pop, and imap access to secure connections using https, tls, ssl. We have never been able to get the average ISP customer to use good passwords, but every little bit helps. Bill -- INTERNET: bill at celestial.com Bill Campbell; Celestial Software LLC URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way Voice: (206) 236-1676 Mercer Island, WA 98040-0820 Fax: (206) 232-9186 Skype: jwccsllc (206) 855-5792 bad economics will sink any economy no matter how much they believe this time things are different. They aren't. -- Arthur Laffer