Hello,
I've noticed a strange delay while booting a CentOS 7 guest on a CentOS 7 host with slow disks (7200RPM) with write cache off.
The guest and host are freshly installed Centos 7 (host was fully patched before guest install). Guest is installed on an lvm pool residing on an md raid1 with two SATA 7200 RPM drives with their write caches off.
The delay is on mounting /boot, the dmesg on the guest says:
[ 23.491857] XFS (vda1): Mounting V4 Filesystem [ 46.668396] XFS (vda1): Ending clean mount [ 46.668404] SELinux: initialized (dev vda1, type xfs), uses xattr
Subsequent mounts by hand on a running system last even longer - dmesg records:
[ 96.442544] XFS (vda1): Unmounting Filesystem [ 100.086741] XFS (vda1): Mounting V4 Filesystem [ 146.098703] XFS (vda1): Ending clean mount [ 146.098721] SELinux: initialized (dev vda1, type xfs), uses xattr
mounting "-o ro" doesn't help - dmesg records:
[ 205.552778] XFS (vda1): Unmounting Filesystem [ 212.452724] XFS (vda1): Mounting V4 Filesystem [ 258.537459] XFS (vda1): Ending clean mount [ 258.537477] SELinux: initialized (dev vda1, type xfs), uses xattr
but mounting with "-o ro,norecovery" does help - dmesg records:
[ 263.795813] XFS (vda1): Unmounting Filesystem [ 271.973701] XFS (vda1): Mounting V4 filesystem in no-recovery mode. Filesystem will be inconsistent. [ 272.003618] SELinux: initialized (dev vda1, type xfs), uses xattr
xfs_logprint -t /dev/vda1 says that the device is clean:
xfs_logprint: data device: 0xfc01 log device: 0xfc01 daddr: 512032 length: 6824
log tail: 3447 head: 3447 state: <CLEAN>
however the full xfs log contains a lot of entries xfs_logprint /dev/vda1 results in ~23000 lines of text.
Permanent help is from xfs_repair which zeroes the log - a subsequent mount is fast, and subsequent reboots are fast.
[ 411.290031] XFS (vda1): Mounting V4 Filesystem [ 411.456661] XFS (vda1): Ending clean mount [ 411.456676] SELinux: initialized (dev vda1, type xfs), uses xattr
During the problematic mounts the host reports a lot of small disk writes with queue depth ~1. Also, when I move the problematic filesystem using dd to an lvm volume, mount -o ro, unmount, make snapshot, mount -o ro again, then the contents of the original volume are the same as the snapshot but the snapshot usage increases from 0 to ~2MB - so something writes to the original volume in a ro mode - possibly a log replay.
Is is normal for the xfs mount to replay a log after a clean unmount?
With best regards, Andrzej