On Sat, Aug 22, 2009 at 6:07 PM, Bill Campbell<centos at celestial.com> wrote: > On Sat, Aug 22, 2009, Dave wrote: >>On Sat, Aug 22, 2009 at 6:49 AM, Bill Campbell<centos at celestial.com> wrote: >>> I review daily reports from over 50 systems every morning, checking changes >>> found, usually taking no more than 10 minutes a day. The key is to keep >>> the reports simple, and to make updating easy (and to have procedures that >>> monitor systems to be sure they's still alive and reporting in). >> >>So how do you track the inevitable changes? Not saying you can't, just >>curious. For me, when I look at a batch of changes, some of them are >>obviously stuff I've done, other stuff not so obvious. I also filter >>reports through a script that sort of does a diff and makes an attempt >>to limit the boilerplate. Sometimes it is a bit too terse. > > First off, we don't allow automatic updates on most systems, much > preferring to do them manually making it pretty easy to refresh > the comparison database immediately after the update is complete. > The odds that a cracker will get in and do their dirty deeds > while this are going on are pretty low, and can probably be > ignored. > > We handle pretty much all server stuff under the OpenPKG portable > package management system so things like spamassassin, amavisd, > clamav, and postfix are not the distribution versions, but those > from OpenPKG (which are generally updated more quickly then the > distribution's). A typical occurrence will be that we get an > e-mail saying that clamav is out of date from the nightly > freshclam update, I will pick up the new sources, update the > OpenPKG SRPM for it, and deploy it 40 or so systems running it, > and expect to see a corresponding set of notices the next morning > that files under clamav have changed. > > The clusterssh program makes this sort of thing much more efficient > as one can execute shell commands on multiple systems simultaneously. > >>> We create a file system initially, the same size as ``/'', and make a copy >>> of ``/'' in it identical except for the /etc/fstab entry. This is not >>> mounted in normal operations, but the system can be booted from it to get >>> to a clean system. >> >>Wow, elaborate. How do you protect this file system from intruders? >>Exterrnal and powerred off? > > That's one way to do it. We also run a fair number of Linux > servers under VMware so periodic snapshots and backups simplify > the task. > > I have not seen many successful cracks of Linux boxes that we > have configured from scratch. Some basic things can be done to > minimize the chances of cracks. > > + Create the baseline for intrusion detection tools before putting the > syste on line, and monitor it daily. > > + Configure openssh to refuse password authentication requiring > authorized_keys access. > > + Configure openssh with tcp_wrappers support, restricting access by IP > address and/or domain names. I consider this absolutely mandatory if > one needs to all username and password authentication. > > + Use fail2ban or similar techniques to quickly block IP addresses that > are found probing the system (don't forget to look at POP and IMAP > logs for failed login attempts). > > + Use /bin/false as the standard shell for accounts that don't have good > reason for shell access. This does not affect e-mail or most services > that a typical ISP customer needs. > > + Use OpenVPN for access. This works well even when in hotels with NAT > firewalls, and is not easily hacked anonymously. > > + Restrict access of webmin and usermin to local networks so they are > not vulnerable to outside attack. These services are available to > people outside connecting with OpenVPN. Cross Site Attacks (CSRF, XSS) make webmin very vulnerable in this scenario. It is a bad idea to use a single browser. If in Firefox, you already logged in to webmin and browse to a malicious site (many reputable sites unknowingly have malicious javascript -- see HoneyNet), the malicious site could do nasty things via webmin or any other internal webserver. Yes, NoScript may help, but NoScript has to be updated daily and Firefox restarted. The best practice is to Install three separate browser application such as Epiphany or Dillo and only use this for internal websites. Use Firefox for email. Use Chrome for everything else. The idea is to have completely separate processes using completely separate memory and harddrive locations. I don't think there are many malicious variants of InvisibleThings's BluePill or BlueChicken, but if a malicious variant can elevate itself to become the Hypervisor, then all of your virtual machines could be monitored by a HyperKit -- rootkit in the hypervisor. Again, i don't know if there are many malicious in-the-wild versions of bluepill, but if just one malicious vmware image is uploaded to the Amazon EC2, then every other VM on that same hardware at Amazon can be controlled by a hyperkit. InvisibleThings are professional security researchers in Poland, so the original bluepill isn't malicious. BlueChicken is to evade detection by moving bluepill back and forth between the hypervisor and a vm at will. Others are going to know more about what actually is in-the-wild versus in the lab. > > + Restrict webmail, pop, and imap access to secure connections using > https, tls, ssl. We have never been able to get the average ISP > customer to use good passwords, but every little bit helps. > > Bill > -- > INTERNET: bill at celestial.com Bill Campbell; Celestial Software LLC > URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way > Voice: (206) 236-1676 Mercer Island, WA 98040-0820 > Fax: (206) 232-9186 Skype: jwccsllc (206) 855-5792 > > bad economics will sink any economy no matter how much they believe this > time things are different. They aren't. -- Arthur Laffer > _______________________________________________ > CentOS mailing list > CentOS at centos.org > http://lists.centos.org/mailman/listinfo/centos >