Hi!
I'm using follow method for remote logging and catch logs from many servers.
Nginx writes logs into fifo, which created via nginx init script:
cat /etc/sysconfig/nginx
...
# syslog-ng support for nginx
if [ ! -p /var/log/nginx/access.log ]; then
/bin/rm -f /var/log/nginx/access.log
/usr/bin/mkfifo --mode=0640 /var/log/nginx/access.log
fi
if [ ! -p /var/log/nginx/error.log ] ; then
/bin/rm -f /var/log/nginx/error.log
/usr/bin/mkfifo --mode=0640 /var/log/nginx/error.log
fi
/bin/chown nginx:root /var/log/nginx/access.log /var/log/nginx/error.log
Nginx just writes to fifo as to file. Nginx has nonblocking output to
logs and if nobody read fifo nginx dont stop on logs write.
>From other side pipe reads syslog-ng.
cat /etc/syslog-ng/syslog-ng.conf
...
source s_nginx_20 {
fifo ("/var/log/nginx/access.log" log_prefix("nginx-access-log: "));
};
source s_nginx_21 {
fifo ("/var/log/nginx/error.log" log_prefix("nginx-error-log: "));
};
...
destination d_remote { tcp("remote.example.com", port(514)); };
...
# nginx
filter f_nginx_20 { match("nginx-access-log: "); };
filter f_nginx_21 { match("nginx-error-log: "); };
...
# nginx
log { source(s_nginx_20); filter(f_nginx_20); destination(d_remote); };
log { source(s_nginx_21); filter(f_nginx_21); destination(d_remote); };
To avoid syslog-ng problems on startup (ex. if fifo does not exists)
used follow solution:
cat /etc/sysconfig/syslog-ng
...
# syslog-ng support for nginx
if [ ! -p /var/log/nginx/access.log ]; then
/bin/rm -f /var/log/nginx/access.log
/usr/bin/mkfifo --mode=0640 /var/log/nginx/access.log
fi
if [ ! -p /var/log/nginx/error.log ] ; then
/bin/rm -f /var/log/nginx/error.log
/usr/bin/mkfifo --mode=0640 /var/log/nginx/error.log
fi
/bin/chown nginx:root /var/log/nginx/access.log /var/log/nginx/error.log
On remote side (remote.example.com):
cat /etc/syslog-ng/syslog-ng.conf
...
source s_net {
udp(ip(0.0.0.0) port(514));
tcp(ip(0.0.0.0) port(514) keep-alive(yes) max-connections(128));
};
...
filter f_nginx_20 { match("nginx-access-log: "); };
filter f_nginx_21 { match("nginx-error-log: "); };
...
destination d_nginx_20 { file("/var/log/nginx/access.log"); };
destination d_nginx_21 { file("/var/log/nginx/error.log"); };
...
log { source(s_sys); filter(f_nginx_20); destination(d_nginx_20); };
log { source(s_sys); filter(f_nginx_21); destination(d_nginx_21); };
In the same way I catch logs from 20-30 servers to 1 server, approx.
300GB gzipped logs per day.
On Thu, Mar 24, 2011 at 11:23 PM, Dr. Ed Morbius <dredmorbius at gmail.com> wrote:
> I'm looking for suggestions as to a good general method of
> remote-logging services such as nginx or anything else which doesn't
> support syslog natively.
>
> I'm aware that there's an nginx patch, and we're evaluating this. It
> may be the way we fly.
>
> However there are other tools which may not have a patch for which
> remote logging would be useful. If there's a general soution (something
> as naive as tailing local logs and firing these off on a regular basis).
>
> I've heard rumors of a Perl script used for apache logs.
>
> Also that rsyslog supports logging from local files to a remote syslog
> server, possibly. I'm RTFMing on that.
>
> Thanks in advance.
>
> --
> Dr. Ed Morbius, Chief Scientist / |
> Robot Wrangler / Staff Psychologist | When you seek unlimited power
> Krell Power Systems Unlimited | Go to Krell!
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos
>
--
Ilyas R. Khasyanov
Unix/Linux System Administrator
GPG Key ID: 6EC5EB27 (Changed since 2009-05-12)