[CentOS] How to troubleshoot partial shutdown problem?

Tue Jan 1 11:47:42 UTC 2019
Marko Vojinovic <vvmarko at gmail.com>

Hi folks,

I've never encountered the following problem before (but I guess there
is a first time for everything) --- after issuing a regular shutdown,
the system starts the shutdown procedure, but stalls at some point, and
never finishes. It gets to the console, writes the "powering down"
message and stops there --- the hardware never actually powers off. At
that time, the machine is completely unresponsive to anything, I have to
hold the power button for 5 seconds to actually force it to turn off.

This is a clean install of C7, fully updated.

I doubt that hardware is at fault, since it *does* properly shut down
when I boot it from the C7 live usb (which was used to install the
system in the first place). The only difference is that the live usb is
not updated at all, while the os on the hard drive has received cca
1000 updates after initial installation. So I'm guessing that something
in the updates broke something in the shutdown procedure.

Btw, rebooting the machine works properly, no issues.

How am I to troubleshoot this? Most importantly, what is the best way
to check (after the power cycle) if the hard drive had been unmounted
properly during the previous shutdown, i.e. if the unmounting finished
before the stall? I don't want the hard drive to "suffer" from unclean
shutdowns, if possible. Also, what piece of code prints the "powering
off" message in the console (since that appears to be the last thing
working)? What else to look for, and where? Is this maybe a known issue,
is there a fix?

Other than shutting down, the machine works completely ok, so arguably
this is not such a big problem (as I plan to have it running 24/7),
but still, failing to shutdown when I want it to feels somewhat
disturbing, I'd rather have that fixed.

Any suggestions?

Best, :-)
Marko

P.S. Happy new year to everyone! ;-)