First: thanks very much for spelling this out, Ilyas. This was along the lines of what I'd been considering. You addressed a number of concerns I had (e.g.: non-blocking output) which is really helpful.
on 08:39 Fri 25 Mar, Ilyas -- (umask00@gmail.com) wrote:
Hi!
I'm using follow method for remote logging and catch logs from many servers. Nginx writes logs into fifo, which created via nginx init script:
cat /etc/sysconfig/nginx ... # syslog-ng support for nginx if [ ! -p /var/log/nginx/access.log ]; then /bin/rm -f /var/log/nginx/access.log /usr/bin/mkfifo --mode=0640 /var/log/nginx/access.log fi if [ ! -p /var/log/nginx/error.log ] ; then /bin/rm -f /var/log/nginx/error.log /usr/bin/mkfifo --mode=0640 /var/log/nginx/error.log fi /bin/chown nginx:root /var/log/nginx/access.log /var/log/nginx/error.log
Nginx just writes to fifo as to file. Nginx has nonblocking output to logs and if nobody read fifo nginx dont stop on logs write.
Bingo.
From other side pipe reads syslog-ng. cat /etc/syslog-ng/syslog-ng.conf ... source s_nginx_20 { fifo ("/var/log/nginx/access.log" log_prefix("nginx-access-log: ")); };
source s_nginx_21 { fifo ("/var/log/nginx/error.log" log_prefix("nginx-error-log: ")); }; ... destination d_remote { tcp("remote.example.com", port(514)); }; ... # nginx filter f_nginx_20 { match("nginx-access-log: "); }; filter f_nginx_21 { match("nginx-error-log: "); }; ... # nginx log { source(s_nginx_20); filter(f_nginx_20); destination(d_remote); }; log { source(s_nginx_21); filter(f_nginx_21); destination(d_remote); };
Nice.
To avoid syslog-ng problems on startup (ex. if fifo does not exists) used follow solution: cat /etc/sysconfig/syslog-ng ... # syslog-ng support for nginx if [ ! -p /var/log/nginx/access.log ]; then /bin/rm -f /var/log/nginx/access.log /usr/bin/mkfifo --mode=0640 /var/log/nginx/access.log fi if [ ! -p /var/log/nginx/error.log ] ; then /bin/rm -f /var/log/nginx/error.log /usr/bin/mkfifo --mode=0640 /var/log/nginx/error.log fi /bin/chown nginx:root /var/log/nginx/access.log /var/log/nginx/error.log
On remote side (remote.example.com): cat /etc/syslog-ng/syslog-ng.conf ... source s_net { udp(ip(0.0.0.0) port(514)); tcp(ip(0.0.0.0) port(514) keep-alive(yes) max-connections(128)); }; ... filter f_nginx_20 { match("nginx-access-log: "); }; filter f_nginx_21 { match("nginx-error-log: "); }; ... destination d_nginx_20 { file("/var/log/nginx/access.log"); }; destination d_nginx_21 { file("/var/log/nginx/error.log"); }; ... log { source(s_sys); filter(f_nginx_20); destination(d_nginx_20); }; log { source(s_sys); filter(f_nginx_21); destination(d_nginx_21); };
In the same way I catch logs from 20-30 servers to 1 server, approx. 300GB gzipped logs per day.
Great. That also answers the scaling question. We're comfortably under that scale for now.
Very, very helpful post, thanks again.
On Thu, Mar 24, 2011 at 11:23 PM, Dr. Ed Morbius dredmorbius@gmail.com wrote:
I'm looking for suggestions as to a good general method of remote-logging services such as nginx or anything else which doesn't support syslog natively.
I'm aware that there's an nginx patch, and we're evaluating this. It may be the way we fly.
However there are other tools which may not have a patch for which remote logging would be useful. If there's a general soution (something as naive as tailing local logs and firing these off on a regular basis).
I've heard rumors of a Perl script used for apache logs.
Also that rsyslog supports logging from local files to a remote syslog server, possibly. I'm RTFMing on that.
Thanks in advance.
-- Dr. Ed Morbius, Chief Scientist / | Robot Wrangler / Staff Psychologist | When you seek unlimited power Krell Power Systems Unlimited | Go to Krell! _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
-- Ilyas R. Khasyanov Unix/Linux System Administrator GPG Key ID: 6EC5EB27 (Changed since 2009-05-12)