Message-ID: 4991E3B7.6090503@andrei.myip.org
On: Tue, 10 Feb 2009 12:29:43 -0800, Florin Andrei florin@andrei.myip.org wrote:
Jake wrote:
We're about to start moving our public DNS to in-house managed servers. My first thought was "Linux + BIND" and we're done. Someone in another business unit's IT dept. has suggested tinydns be used.
But things have changed. Nowadays Bind is solid enough. If you're still worried about security issues (you shouldn't, but I'm assuming the paranoid scenario) then CentOS has a good SELinux policy around it, so just install the latest CentOS, keep SELinux enabled, do a "yum update" every once in a while, and be at peace. By the way, this is also the most sweat-free solution from a sysadmining perspective.
With one very large caveat.
Be aware that updating bind via yum can result in your existing bind configuration files being renamed to something.rmpsave and your name server left in a dysfunctional state. I suggest that you consider excluding bind from normal updates and only update it when you are ready and able to check for possible configuration issues.
James B. Byrne wrote:
Be aware that updating bind via yum can result in your existing bind configuration files being renamed to something.rmpsave and your name server left in a dysfunctional state. I suggest that you consider excluding bind from normal updates and only update it when you are ready and able to check for possible configuration issues.
Or put those files under Cfengine or Puppet and have auto-yum work with the config management system.
But yeah, I agree, if the number of systems is small (so config management is not justified), avoid doing auto-updates to the critical components.
On Wed, 2009-02-11 at 17:34 -0500, James B. Byrne wrote:
With one very large caveat.
Be aware that updating bind via yum can result in your existing bind configuration files being renamed to something.rmpsave and your name server left in a dysfunctional state. I suggest that you consider excluding bind from normal updates and only update it when you are ready and able to check for possible configuration issues.
That's true of pretty much every rpm that has config files... it's part of my standard updating routine:
yum update updatedb locate rpmsave locate rpmnew fix any config files... restart services and/or reboot if necessary...
-I
Ian Forde wrote on Wed, 11 Feb 2009 20:01:21 -0800:
locate rpmsave locate rpmnew
rpmsave is left from *un*installations, rpmnew is the *new* file, there is no file overwritten. rpm usually doesn't overwrite files if they got changed. And I haven't seen any overwrites with all the bind updates in the past months. So, I cannot back James' claim.
Kai
Kai Schaetzl wrote:
Ian Forde wrote on Wed, 11 Feb 2009 20:01:21 -0800:
locate rpmsave locate rpmnew
rpmsave is left from *un*installations, rpmnew is the *new* file, there is no file overwritten. rpm usually doesn't overwrite files if they got changed.
AFAIK this is not correct, a package upgrade can create either of these (or both, or neither of them despite your having edited a file). And that's the way it should be, either choice can be justified. It depends on the package's SPEC file. rpm just does what it's told, everything is in the hands of the package maintainer.
On Thu, Feb 12, 2009, Nicolas Thierry-Mieg wrote:
Kai Schaetzl wrote:
Ian Forde wrote on Wed, 11 Feb 2009 20:01:21 -0800:
locate rpmsave locate rpmnew
rpmsave is left from *un*installations, rpmnew is the *new* file, there is no file overwritten. rpm usually doesn't overwrite files if they got changed.
AFAIK this is not correct, a package upgrade can create either of these (or both, or neither of them despite your having edited a file). And that's the way it should be, either choice can be justified. It depends on the package's SPEC file. rpm just does what it's told, everything is in the hands of the package maintainer.
I think that the only time a .rpmnew file is created is if the SPEC file specifies ``%config(noreplace)'' for a file. If the ``noreplace'' option is not used, the .rpmsave files are created either when a package is removed, or when a file specified as a configuration file in the RPM SPEC file is updated and the file is sufficiently different from the default (for some definition of suffieiently).
In the OpenPKG portable packaging system, which is RPM based, the presence of any .rpmnew or .rpmsave configuration files will prevent a package from starting, and warning messages will be generated until the situation is resolved.
Bill
Bill Campbell wrote:
locate rpmsave locate rpmnew
rpmsave is left from *un*installations, rpmnew is the *new* file, there is no file overwritten. rpm usually doesn't overwrite files if they got changed.
AFAIK this is not correct, a package upgrade can create either of these (or both, or neither of them despite your having edited a file). And that's the way it should be, either choice can be justified. It depends on the package's SPEC file. rpm just does what it's told, everything is in the hands of the package maintainer.
I think that the only time a .rpmnew file is created is if the SPEC file specifies ``%config(noreplace)'' for a file. If the ``noreplace'' option is not used, the .rpmsave files are created either when a package is removed, or when a file specified as a configuration file in the RPM SPEC file is updated and the file is sufficiently different from the default (for some definition of suffieiently).
In the OpenPKG portable packaging system, which is RPM based, the presence of any .rpmnew or .rpmsave configuration files will prevent a package from starting, and warning messages will be generated until the situation is resolved.
That sounds like the kiss of death for any critical service. Can't it figure out ahead of time that this is going to happen and let the service keep running unchanged with a warning message about needing the update instead?
On Thu, Feb 12, 2009, Les Mikesell wrote:
Bill Campbell wrote:
locate rpmsave locate rpmnew
rpmsave is left from *un*installations, rpmnew is the *new* file, there is no file overwritten. rpm usually doesn't overwrite files if they got changed.
AFAIK this is not correct, a package upgrade can create either of these (or both, or neither of them despite your having edited a file). And that's the way it should be, either choice can be justified. It depends on the package's SPEC file. rpm just does what it's told, everything is in the hands of the package maintainer.
I think that the only time a .rpmnew file is created is if the SPEC file specifies ``%config(noreplace)'' for a file. If the ``noreplace'' option is not used, the .rpmsave files are created either when a package is removed, or when a file specified as a configuration file in the RPM SPEC file is updated and the file is sufficiently different from the default (for some definition of suffieiently).
In the OpenPKG portable packaging system, which is RPM based, the presence of any .rpmnew or .rpmsave configuration files will prevent a package from starting, and warning messages will be generated until the situation is resolved.
That sounds like the kiss of death for any critical service. Can't it figure out ahead of time that this is going to happen and let the service keep running unchanged with a warning message about needing the update instead?
This has not proven a problem as we monitor updates, and generally know which packages may generate the rpmsave or rpmnew files as a result of testing on development machines before deploying to production systems.
We have an administrative script that monitors the status of all servers running under our OpenPKG system, and quickly indicates anything that is not running while doing updates. In addition, our systems all check critical services under cron control, attempt to restart services that are not running, and notifying our support system via direct SMTP (bypassing postfix, amavisd, and clamv in case one of them is down) and with xmlrpc calls to our support servers as well. Our systems keep track of systems that are supposed to check in, and generate alerts when one or more miss checkins.
Nothing is perfect of course, but this has worked well for us for almost 10 years now.
Bill
On Thu, 2009-02-12 at 11:08 -0600, Les Mikesell wrote:
That sounds like the kiss of death for any critical service. Can't it figure out ahead of time that this is going to happen and let the service keep running unchanged with a warning message about needing the update instead?
You're missing the point. If the service is already running, the changes won't take effect until you restart the service with the new binaries. And the whole patching exercise is what maintenance windows are for, anyway. Note that it's critical SERVICE, not critical SERVER. The former is more important than the latter, so ideally you should be able to take down the latter in order to upgrade one implementation of the former.
-I
On Thu, Feb 12, 2009, Ian Forde wrote:
On Thu, 2009-02-12 at 11:08 -0600, Les Mikesell wrote:
That sounds like the kiss of death for any critical service. Can't it figure out ahead of time that this is going to happen and let the service keep running unchanged with a warning message about needing the update instead?
You're missing the point. If the service is already running, the changes won't take effect until you restart the service with the new binaries. And the whole patching exercise is what maintenance windows are for, anyway. Note that it's critical SERVICE, not critical SERVER. The former is more important than the latter, so ideally you should be able to take down the latter in order to upgrade one implementation of the former.
I understand the distinction very well. In the time we have been using this method, we have never taken down a service for any significant period of time (the services are restarted on installation by the RPM SPEC files' %pre, %post processing).
Of course we don't do things that are likely to take a critical service down without proper prior planning (often found out the hard way on our own systems :-). If an update is likely to have an impact on operations, it is scheduled during a maintenance window.
The services that are most frequently updated are clamav, spamassassin, and amavisd-new, and we have often done this on heavily loaded MX servers (millions of e-mails a day) without having a service down for more than a minute or two while dealing with configuration file changes.
Bill
Bill Campbell wrote:
That sounds like the kiss of death for any critical service. Can't it figure out ahead of time that this is going to happen and let the service keep running unchanged with a warning message about needing the update instead?
You're missing the point. If the service is already running, the changes won't take effect until you restart the service with the new binaries. And the whole patching exercise is what maintenance windows are for, anyway. Note that it's critical SERVICE, not critical SERVER. The former is more important than the latter, so ideally you should be able to take down the latter in order to upgrade one implementation of the former.
I understand the distinction very well. In the time we have been using this method, we have never taken down a service for any significant period of time (the services are restarted on installation by the RPM SPEC files' %pre, %post processing).
Of course we don't do things that are likely to take a critical service down without proper prior planning (often found out the hard way on our own systems :-). If an update is likely to have an impact on operations, it is scheduled during a maintenance window.
In other words you'd dedicated sufficient human resources to undo whatever damage the package management system causes...
On Thu, Feb 12, 2009, Les Mikesell wrote:
Bill Campbell wrote:
That sounds like the kiss of death for any critical service. Can't it figure out ahead of time that this is going to happen and let the service keep running unchanged with a warning message about needing the update instead?
You're missing the point. If the service is already running, the changes won't take effect until you restart the service with the new binaries. And the whole patching exercise is what maintenance windows are for, anyway. Note that it's critical SERVICE, not critical SERVER. The former is more important than the latter, so ideally you should be able to take down the latter in order to upgrade one implementation of the former.
I understand the distinction very well. In the time we have been using this method, we have never taken down a service for any significant period of time (the services are restarted on installation by the RPM SPEC files' %pre, %post processing).
Of course we don't do things that are likely to take a critical service down without proper prior planning (often found out the hard way on our own systems :-). If an update is likely to have an impact on operations, it is scheduled during a maintenance window.
In other words you'd dedicated sufficient human resources to undo whatever damage the package management system causes...
Isn't that what our customers are paying us to do?
That has to be true now matter how one is doing updates.
I have personally updated clamav on more than 50 machines in an afternoon without having any of them down for more than a minute, and that time mostly because clamav takes a while to restart.
FWIW, we normally have clamav updates installed at all our client sites with 24 hours of the first notice that there's a new version out from swatch looking at the freshclamav.log file. This includes downloading the new tarball, updating the OpenPKG SRPM, building, testing in-house, and deployment. Often this is complete before people on this CentOS list start asking questions about the update or saying it won't build.
Oh, and these updates are on a variety of Linux systems ranging from SuSE 9.0 Pro, SLES9, SLES10, CentOS 4.5 through CentOS 5.x, and at least one FreeBSD box -- all from the same SRPM file.
Bill
Bill Campbell wrote:
Of course we don't do things that are likely to take a critical service down without proper prior planning (often found out the hard way on our own systems :-). If an update is likely to have an impact on operations, it is scheduled during a maintenance window.
In other words you'd dedicated sufficient human resources to undo whatever damage the package management system causes...
Isn't that what our customers are paying us to do?
That has to be true now matter how one is doing updates.
Yes, but the extent to which it is actually required depends on how badly the intended automation fails. I think at least in theory, the parts of config files that are likely to need user modifications are supposed to be extracted to /etc/sysconfig/... so the files included in RPM updates generally won't have local changes and can be replaced without regard to the old contents. And programs suitable for inclusion in an 'enterprise' distribution should be designed so as not to require non-backwards-compatible changes in updates.
On Thu, Feb 12, 2009, Les Mikesell wrote:
Bill Campbell wrote:
Of course we don't do things that are likely to take a critical service down without proper prior planning (often found out the hard way on our own systems :-). If an update is likely to have an impact on operations, it is scheduled during a maintenance window.
In other words you'd dedicated sufficient human resources to undo whatever damage the package management system causes...
Isn't that what our customers are paying us to do?
That has to be true now matter how one is doing updates.
Yes, but the extent to which it is actually required depends on how badly the intended automation fails. I think at least in theory, the parts of config files that are likely to need user modifications are supposed to be extracted to /etc/sysconfig/... so the files included in RPM updates generally won't have local changes and can be replaced without regard to the old contents. And programs suitable for inclusion in an 'enterprise' distribution should be designed so as not to require non-backwards-compatible changes in updates.
With OpenPKG all configuration files are under $prefix/etc/packagename where $prefix is the base directory of an OpenPKG instance (there may be more than one on a single system), and packagename is the name of the package, postfix, amavisd, clamav, mysql, etc. One of the basic principles of OpenPKG is to have absolutely minimal footprint on the installed system, only 7 lines in /etc/crontab, and the appropriate /etc/init.d entries (these actually vary depending on the type of host system).
Some packages have multiple configuration files with only those for site parameters being declared at %config files in the RPM SPEC file. The issues occur when one has large, ugly configuration files (can we spell amavisd.conf :-), and there's a major version update with lots of new variables or variable name changes.
FWIW, to bring this back to the djbdns topic, the *ONLY* configuration file in our OpenPKG packaging of djbdns, daemontools, and ucspi-tcp is the dnsroots.global file used by dnscache. Each server installed is in its own directory which is not affected by updates.
Bill
FWIW, to bring this back to the djbdns topic, the *ONLY* configuration file in our OpenPKG packaging of djbdns, daemontools, and ucspi-tcp is the dnsroots.global file used by dnscache. Each server installed is in its own directory which is not affected by updates.
uscpi-tcp? Where does that come in? rsyncing of the cdb file?
But yeah, thanks for pointing out that djbdns is zero maintenance save for its zone data which can be automated and therefore be to a large degree zero maintenance too.
On Fri, Feb 13, 2009, Christopher Chan wrote:
FWIW, to bring this back to the djbdns topic, the *ONLY* configuration file in our OpenPKG packaging of djbdns, daemontools, and ucspi-tcp is the dnsroots.global file used by dnscache. Each server installed is in its own directory which is not affected by updates.
uscpi-tcp? Where does that come in? rsyncing of the cdb file?
It's mostly to allow BIND sites to do zone transfers from djbdns sites.
But yeah, thanks for pointing out that djbdns is zero maintenance save for its zone data which can be automated and therefore be to a large degree zero maintenance too.
Bill
uscpi-tcp? Where does that come in? rsyncing of the cdb file?
It's mostly to allow BIND sites to do zone transfers from djbdns sites.
Oh the axfr-dns daemon. This is the answer for the 'djbdns does not support tcp or zone transfer' nonsense routinely vomited by DJB haters.
Nicolas Thierry-Mieg wrote on Thu, 12 Feb 2009 16:16:14 +0100:
AFAIK this is not correct, a package upgrade can create either of these (or both, or neither of them despite your having edited a file). And that's the way it should be, either choice can be justified.
Sure, a apckage can do anything, but that's how it usually is done.
Kai
-
Bill Campbell -
Christopher Chan -
Florin Andrei -
Ian Forde -
James B. Byrne -
Kai Schaetzl -
Les Mikesell -
Nicolas Thierry-Mieg