Hi all,
I've recently switched to the latest rt kernel available from http://people.centos.org/hughesjr/kernel-rt And one of the two problems I'm having with this new kernel is that the logs have been flooded with nagios alerts which indicate high percentage (50-60) of packet loss when pinging localhost. The problem shows up several hours after the system reboot.
Any ideas on what might be wrong and how to fix this on a running system?
Are there any current plans to integrate MRG Realtime into CentOS project?
Thank you, Sasha
Greetings,
On Thu, Sep 3, 2009 at 2:32 PM, A. Kirillovnevis2us@infoline.su wrote:
Hi all,
Are there any current plans to integrate MRG Realtime into CentOS project?
Is the source of MRG open at all?
I was under the impression that it is one of those RH proprietary available thru RHN.
Thanks and Regards
Rajagopal
Are there any current plans to integrate MRG Realtime into CentOS project?
Is the source of MRG open at all?
I was under the impression that it is one of those RH proprietary available thru RHN.
The source for rt kernels at least is available at ftp://ftp.redhat.com/pub/redhat/linux/enterprise/5Server/en/RHEMRG/SRPMS as been mentioned by Johnny Hughes in a number of threads.
Regards, Sasha
I've recently switched to the latest rt kernel available from http://people.centos.org/hughesjr/kernel-rt And one of the two problems I'm having with this new kernel is that the logs have been flooded with nagios alerts which indicate high percentage (50-60) of packet loss when pinging localhost. The problem shows up several hours after the system reboot.
The problem also shows up with the latest kernel-rt-2.6.24.7-132 and seems to be triggered by scheduled cron jobs. That's probably the only time when this system might be under any significant load. Below are the relevant excerpts from /var/log/messages indicating when the system booted into which kernel and the first 10 ping alerts after system reboot.
Aug 28 20:03:40 angara kernel: Linux version 2.6.24.7-65.el5rt.centos
Aug 29 02:12:54 PING CRITICAL - Packet loss = 60%, RTA = 0.13 ms Aug 29 02:13:44 PING WARNING - Packet loss = 44%, RTA = 0.09 ms Aug 29 02:14:54 PING WARNING - Packet loss = 54%, RTA = 0.08 ms Aug 29 02:15:45 PING WARNING - Packet loss = 44%, RTA = 0.09 ms Aug 29 03:10:54 PING CRITICAL - Packet loss = 70%, RTA = 0.11 ms Aug 29 03:20:46 PING WARNING - Packet loss = 44%, RTA = 0.09 ms Aug 29 03:30:56 PING CRITICAL - Packet loss = 72%, RTA = 0.09 ms Aug 29 03:35:45 PING WARNING - Packet loss = 44%, RTA = 0.12 ms Aug 29 03:40:44 PING OK - Packet loss = 16%, RTA = 0.10 ms Aug 29 03:45:44 PING WARNING - Packet loss = 50%, RTA = 0.09 ms ...
Aug 29 11:30:56 angara kernel: Linux version 2.6.24.7-65.el5rt.centos
Aug 31 03:43:13 PING WARNING - Packet loss = 28%, RTA = 0.10 ms Aug 31 03:44:13 PING WARNING - Packet loss = 37%, RTA = 0.10 ms Aug 31 03:45:23 PING WARNING - Packet loss = 54%, RTA = 0.08 ms Aug 31 03:46:23 PING CRITICAL - Packet loss = 60%, RTA = 0.12 ms Aug 31 03:51:13 PING WARNING - Packet loss = 44%, RTA = 0.10 ms Aug 31 04:31:23 PING CRITICAL - Packet loss = 60%, RTA = 0.09 ms Aug 31 04:41:14 PING WARNING - Packet loss = 50%, RTA = 0.11 ms Aug 31 06:16:13 PING OK - Packet loss = 16%, RTA = 0.11 ms Aug 31 06:21:13 PING WARNING - Packet loss = 44%, RTA = 0.10 ms Aug 31 06:22:13 PING WARNING - Packet loss = 50%, RTA = 0.10 ms ...
Sep 3 23:44:20 angara kernel: Linux version 2.6.24.7-132.el5.local
Sep 5 04:19:53 PING WARNING - Packet loss = 50%, RTA = 0.14 ms Sep 5 04:21:01 PING CRITICAL - Packet loss = 60%, RTA = 0.10 ms Sep 5 04:21:51 PING WARNING - Packet loss = 28%, RTA = 0.12 ms Sep 5 04:22:56 PING WARNING - Packet loss = 50%, RTA = 0.11 ms Sep 5 04:28:01 PING CRITICAL - Packet loss = 80%, RTA = 0.11 ms Sep 5 04:32:53 PING WARNING - Packet loss = 44%, RTA = 0.10 ms Sep 5 04:37:51 PING OK - Packet loss = 16%, RTA = 0.12 ms Sep 5 04:42:51 PING WARNING - Packet loss = 50%, RTA = 0.11 ms Sep 5 04:44:02 PING CRITICAL - Packet loss = 60%, RTA = 0.09 ms Sep 5 04:45:01 PING CRITICAL - Packet loss = 70%, RTA = 0.10 ms ...
When it happens the pattern of ping failures remains basically the same until the system reboots.
# ping -n -c10 localhost PING localhost.localdomain (127.0.0.1) 56(84) bytes of data. 64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.074 ms 64 bytes from 127.0.0.1: icmp_seq=4 ttl=64 time=0.075 ms 64 bytes from 127.0.0.1: icmp_seq=6 ttl=64 time=0.074 ms 64 bytes from 127.0.0.1: icmp_seq=8 ttl=64 time=0.097 ms 64 bytes from 127.0.0.1: icmp_seq=10 ttl=64 time=0.067 ms
--- localhost.localdomain ping statistics --- 10 packets transmitted, 5 received, 50% packet loss, time 9004ms rtt min/avg/max/mdev = 0.067/0.077/0.097/0.012 ms
Interestingly enough flood pings yeild much better results.
# ping -nf -c10000 localhost PING localhost.localdomain (127.0.0.1) 56(84) bytes of data. ....... --- localhost.localdomain ping statistics --- 10000 packets transmitted, 9993 received, 0% packet loss, time 646ms rtt min/avg/max/mdev = 0.019/0.025/0.119/0.010 ms, ipg/ewma 0.064/0.020 ms
Any ideas or suggestions welcome.
Thanks, Sasha