[CentOS] Commands failing silently?

Wed Mar 26 02:11:42 UTC 2008
mouss <mouss at netoyen.net>

Dan Bongert wrote:
> mouss wrote:
>> Dan Bongert wrote:
>>> Hello all:
>>>
>>> I have a couple CentOS 4 servers (all up-to-date) that are having 
>>> strange command failures. I first noticed this with a perl script 
>>> that uses lots of system calls.
>>>
>>> thoth(66) /tmp> uname -a
>>> Linux thoth.ssc.wisc.edu 2.6.9-67.0.7.ELsmp #1 SMP Sat Mar 15 
>>> 06:54:55 EDT 2008 i686 i686 i386 GNU/Linux
>>>
>>> Nothing in either dmesg or /var/log/messages seems to indicate any 
>>> problems. It also doesn't seem to matter what the command is -- ls 
>>> is the quickest test, but sshd will sometimes to fail to spawn 
>>> children, etc. There aren't a large amount of processes on the 
>>> machine either -- only 122 at the moment.
>>>
>>> Has anyone seen this behavior before? Have I been hit with some sort 
>>> of cunning rootkit? This machine shouldn't be publicly accessible; 
>>> it's behind our firewall.
>>
>> where is /tmp mounted? is this an external disk (usb, ...)? is it an 
>> nfs mount?
>
> It's a local disk:
>
> thoth(97) /tmp> df -h .
> Filesystem            Size  Used Avail Use% Mounted on
> /dev/md4               16G   77M   15G   1% /tmp
>
> Though 'ls' was just an example -- just about any program will fail. 
> The 'w'
> command will fail too:


maybe check your PATH. try
$ /bin/ls

>
> thoth(118) /tmp> w
>   16:06:51 up  5:34,  1 user,  load average: 0.94, 1.46, 2.04
> USER     TTY      FROM              LOGIN@   IDLE   JCPU   PCPU WHAT
> dbongert pts/0    copland.ssc.wisc 14:16    0.00s  0.22s  0.05s w
>
> thoth(119) /tmp> w
>   16:06:52 up  5:34,  1 user,  load average: 0.94, 1.46, 2.04
> USER     TTY      FROM              LOGIN@   IDLE   JCPU   PCPU WHAT
> dbongert pts/0    copland.ssc.wisc 14:16    0.00s  0.22s  0.05s w
>
> thoth(120) /tmp> w
>
> thoth(121) /tmp> w
>