[CentOS] Unusual System State

Wed Dec 7 21:51:45 UTC 2016
m.roth at 5-cent.us <m.roth at 5-cent.us>

Chris Olson wrote:
> Our smallest network of systems has only four computers connected
> via Gigabit Ethernet.  The oldest and most stable platform is an eight
> year old Dell E520 running CentOS 6.8.  We often try out applications
> on this Dell/CentOS machine before moving them to other systems on our
> other networks.
> Last night, one of our users decided to create a single, 228GB home
> directory tar archive on an empty, 500GB, external, USB, Ext4 disk
> drive. This was obviously a poor decision. The extent of the results
> were not obvious until this morning.
> All disk activity had stopped and the system appeared to be hibernation.
> A push on the power button usually brings the system back to life, but
> in this case, the unlock screen was presented for only three seconds
> and then the hibernation mode was resumed.  Repeated attempts to log
> on were all thwarted due to this behavior.  ssh from other systems wasalso
> not possible.
> Holding the power button in order to initiate power down did not work
> either.  The result was the same as a one second press of front panel
> power button bringing up the unlock screen for only a short time.  We
> eventually removed the power cord for five minutes and then restarted
> the machine.

Not that this will be of any help, but we, once in a while, will suddenly
find a machine unresponsive, and in an undefined state. IIRC, pingable,
but can't ssh in, nor is there any response whatsoever to plugging in a
keyboard and monitor. Power cycle is the only answer, and there's never
anything in the logs.

Mostly, those systems are used for very serious scientific computing (as
in, no VM, and I've seen loads of > 80).