[CentOS] Strange du/df behaviour.

Thu Jun 28 08:25:19 UTC 2012
Rafał Radecki <radecki.rafal at gmail.com>

Hi all.

I have currently a server:
cat /etc/redhat-release
CentOS release 5.7 (Final)
uname -a
Linux host.domain.com 2.6.18-274.18.1.el5 #1 SMP Thu Feb 9 12:45:44
EST 2012 x86_64 x86_64 x86_64 GNU/Linux
I have there a filesystem mounted:

/dev/vg0/paczki              /home/paczki-workdir  ext4
defaults,noatime           0 0



on which df  gives strange output:

LANG=C df -h /home/paczki-workdir;sleep 3;df -h /home/paczki-workdir
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg0-paczki
                     1008G  268G  690G  28% /home/paczki-workdir
System plików         rozm. użyte dost. %uż. zamont. na
/dev/mapper/vg0-paczki
                     1008G  155G  803G  17% /home/paczki-workdir


du -sk /home/paczki-workdir;sleep 3;du -sk /home/paczki-workdir
48426364	/home/paczki-workdir
48428980	/home/paczki-workdir

du -sh /home/paczki-workdir;sleep 3;du -sh /home/paczki-workdir
47G	/home/paczki-workdir
47G	/home/paczki-workdir

Currently there is only one process writing to the filesystem:

root     27945 27918  3 09:16 ?        00:01:44 tar
--exclude=*/Maildir/.Spam/cur/* --exclude=*/Maildir/.Spam/new/*
--use-compress-program /usr/bin/pigz -cf
/home/paczki-workdir/abaksa-mail-20120628-0413.tgz

and it writes so much data:

du -sh /home/paczki-workdir/abaksa-mail-20120628-0413.tgz;sleep 3;du
-sh /home/paczki-workdir/abaksa-mail-20120628-0413.tgz
13G     /home/paczki-workdir/abaksa-mail-20120628-0413.tgz
13G     /home/paczki-workdir/abaksa-mail-20120628-0413.tgz

du -sk /home/paczki-workdir/abaksa-mail-20120628-0413.tgz;sleep 3;du
-sk /home/paczki-workdir/abaksa-mail-20120628-0413.tgz
13410988        /home/paczki-workdir/abaksa-mail-20120628-0413.tgz
13425040        /home/paczki-workdir/abaksa-mail-20120628-0413.tgz

I've also noticed that when nothing writes to this filesystem than df
output also varies over short time periods.
When I've updated the kernel and restarted the host for few hours df
was showing ok info. There is no info about problems in
/var/log/messages and dmesg.

Additional info:

vgdisplay -v vg0
    Using volume group(s) on command line
    Finding volume group "vg0"
  --- Volume group ---
  VG Name               vg0
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  7
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               2
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               3,64 TB
  PE Size               4,00 MB
  Total PE              953863
  Alloc PE / Size       786432 / 3,00 TB
  Free  PE / Size       167431 / 654,03 GB
  VG UUID               wRjDqu-TUlW-7YH2-j6xl-nuc1-EczW-D2CAOK

  --- Logical volume ---
  LV Name                /dev/vg0/paczki
  VG Name                vg0
  LV UUID                0EyRjA-p90A-bfX6-1zEA-eMba-5Hn5-jc9slo
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                1,00 TB
  Current LE             262144
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     512
  Block device           253:0

  --- Logical volume ---
  LV Name                /dev/vg0/msklon5
  VG Name                vg0
  LV UUID                8nlU4T-Qozl-bPnr-JOmh-g9Pi-cabs-J7yI7V
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                2,00 TB
  Current LE             524288
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     512
  Block device           253:1

  --- Physical volumes ---
  PV Name               /dev/md0
  PV UUID               hly53e-wbbR-5xLo-3P4k-U4xt-8XeR-W0YCul
  PV Status             allocatable
  Total PE / Free PE    953863 / 167431

mdadm --detail /dev/md0
/dev/md0:
        Version : 0.90
  Creation Time : Wed Apr 11 10:37:04 2012
     Raid Level : raid10
     Array Size : 3907023872 (3726.03 GiB 4000.79 GB)
  Used Dev Size : 1953511936 (1863.01 GiB 2000.40 GB)
   Raid Devices : 4
  Total Devices : 4
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Thu Jun 28 10:17:04 2012
          State : active
 Active Devices : 4
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 0

         Layout : near=2
     Chunk Size : 64K

           UUID : 423fd5cf:beedc018:915808f0:8ec673de
         Events : 0.845339

    Number   Major   Minor   RaidDevice State
       0       8       17        0      active sync   /dev/sdb1
       1       8       33        1      active sync   /dev/sdc1
       2       8       49        2      active sync   /dev/sdd1
       3       8       65        3      active sync   /dev/sde1

Any clues why du shows wrong and floating info?

Best regards,
Rafal Radecki.