[CentOS] Hung kernel task on CentOS 7.2, fprintd and libvirtd

Tue Mar 1 00:50:08 UTC 2016
Warren Young <wyml at etr-usa.com>

On Feb 26, 2016, at 8:17 PM, Gordon Messmer <gordon.messmer at gmail.com> wrote:
> 
> On 02/26/2016 05:17 PM, Warren Young wrote:
>> I tracked this down with strace.  The 25 seconds above is the timeout value passed to poll(2); that timeout is hit while sudo (via PAM) is trying to talk to dbus, since apparently fprintd communicates via dbus.
> 
> If you can still replicate the problem, it might be instructive to use "dbus-monitor" to find out what is (and isn't) happening during authentication.

That didn’t turn up anything enlightening, so I went back to strace, this time on libvirtd, and NAILED IT.  (Don'tcha love that feeling?)

It turns out that libvirtd was blocking on a read(2) call for 

  /sys/devices/pci0000:00/0000:00:1c.0/0000:02:00.0/usb2/manufacturer

That call should complete near-instantly.  Hmm!!

So, I said lsusb, and *it blocked, too!*

I had the lid off this box recently, and remembered seeing another card beside the RAID card, a cheap 2-port USB-3 card we dropped in there a few years ago for a one-off project and then never used it again.  I removed that card, rebooted the server, and libvirtd is now behaving correctly.

Presumably this is also what was going wrong with fprintd: there must be USB-attached fingerprint readers, so if it went and tried to enumerate the USB bus looking for one, it would get stuck just like libvirtd did.

I’ve had more problems with cheap USB gear…grrrr.