On 3/25/2011 3:42 PM, Dr. Ed Morbius wrote:
My concern with buffering / blocking output has more to do with some critical service saying "wups, no more serving until I can flush my log buffers" than it does losing a few lines of logging periodically (though that should also be minimized).
Does this have to be centralized in realtime?
It'd be nice / helpful / useful.
We're at the point now where syncing daily will exceed local storage allocations soon with projected growth rates.
We could do more frequent log rotation / distribution, but given the role and volume, real-time (or very close to it) updates would be preferred, and workfactor is largely orthogonal.
If we need to queue, we could always have rsyslog (we're using it, not syslog-ng) write locally and rotate those frequently. There's still the risk of a hiccup between nginx and rsyslog, but we can keep an eye on that via monit.
Aren't you building in a single point of failure if you use a central syslog receiver - or do you have some sort of failover there? My servers are distributed over different data centers and I wouldn't want to depend on constant connectivity.