on 17:50 Thu 24 Mar, Lamar Owen (lowen at pari.edu) wrote: > On Thursday, March 24, 2011 05:37:41 pm Dr. Ed Morbius wrote: > > on 17:14 Thu 24 Mar, Lamar Owen (lowen at pari.edu) wrote: > > > Prior to PostgreSQL supporting syslog I used [logger] to > > > pipe PostgreSQL output to syslog. Worked fine. > > > I haven't, looking at it. > > It is one option that is definitely in vanilla CentOS. Quite. > > OK. Any pointers on configuration are greatly appreciated. Docs, etc. > > Whew. Large scale remote syslog operation is a large subject; I've > never had anything large-enough scale to need more than logwatch or > site-grown scripts to do processing. The biggest thing to do is set > up NTP and have three reference time sources (three so that if one is > wrong you know which one). Otherwise, log correlation is impossible. It is. There've been a few advances in sysadmin practice since the Nemeth books were first produced, and while there are some titles dealing with portions of this, codifying practices in docs would be a wonderful thing. I've considered (and been approached regarding) tackling at least parts of this myself. Useful logging is definitely part of this. > > Yeah, we're aware of that (I mentioned this in my first post to the > > thread). > > Yep, that you did. > > > We've got a locally-compiled version of nginx, so patching isn't out of > > the question. Just looking at all our options. > > While CentOS doesn't provide nginx itself, it does provide tools for > dealing with logs; I saw several things doing a 'yum list | grep log' > (I know there's easier ways of doing that; that's just the way I > prefer to go about it). Also try grepping a yum list for 'watch' as I > remember some logwatching stuff..... Right, and the general solution also generalizes to other tools. Postgresql (which we aren't using currently) also has its own log handler (a small frustration of mine with the database). And I turned up the rsyslogd feature: http://www.rsyslog.com/doc/imfile.html Text File Input Module Module Name: imfile Author: Rainer Gerhards <rgerhards at adiscon.com> Description: Provides the ability to convert any standard text file into a syslog message. A standard text file is a file consisting of printable characters with lines being delimited by LF. The file is read line-by-line and any line read is passed to rsyslog's rule engine. The rule engine applies filter conditons and selects which actions needs to be carried out. As new lines are written they are taken from the file and processed. Please note that this happens based on a polling interval and not immediately. The file monitor support file rotation. To fully work, rsyslogd must run while the file is rotated. Then, any remaining lines from the old file are read and processed and when done with that, the new file is being processed from the beginning. If rsyslogd is stopped during rotation, the new file is read, but any not-yet-reported lines from the previous file can no longer be obtained. When rsyslogd is stopped while monitoring a text file, it records the last processed location and continues to work from there upon restart. So no data is lost during a restart (except, as noted above, if the file is rotated just in this very moment). Currently, the file must have a fixed name and location (directory). It is planned to add support for dynamically generating file names in the future. Multiple files may be monitored by specifying $InputRunFileMonitor multiple times. -- Dr. Ed Morbius, Chief Scientist / | Robot Wrangler / Staff Psychologist | When you seek unlimited power Krell Power Systems Unlimited | Go to Krell!