Hello all:
I have a couple CentOS 4 servers (all up-to-date) that are having strange command failures. I first noticed this with a perl script that uses lots of system calls.
Basically, sometimes a command just won't run:
thoth(52) /tmp> ls
thoth(53) /tmp> ls
thoth(54) /tmp> ls
thoth(55) /tmp> ls learner lost+found/
thoth(56) /tmp> ls learner lost+found/
thoth(57) /tmp> ls learner lost+found/
thoth(58) /tmp> ls learner lost+found/
thoth(59) /tmp> ls learner lost+found/
thoth(60) /tmp> ls learner lost+found/
thoth(61) /tmp> ls learner lost+found/
thoth(62) /tmp> ls
thoth(63) /tmp> ls
thoth(64) /tmp> ls
thoth(65) /tmp> ls
thoth(66) /tmp> uname -a Linux thoth.ssc.wisc.edu 2.6.9-67.0.7.ELsmp #1 SMP Sat Mar 15 06:54:55 EDT 2008 i686 i686 i386 GNU/Linux
Nothing in either dmesg or /var/log/messages seems to indicate any problems. It also doesn't seem to matter what the command is -- ls is the quickest test, but sshd will sometimes to fail to spawn children, etc. There aren't a large amount of processes on the machine either -- only 122 at the moment.
Has anyone seen this behavior before? Have I been hit with some sort of cunning rootkit? This machine shouldn't be publicly accessible; it's behind our firewall.
Thanks.