[CentOS] mount bind problem

Fri Apr 15 06:05:14 UTC 2016
望月忠雄 <tadao at creative-japan.org>

Dear Robert,

Before sending 'grep -r /home /etc' data, I want tell you what happned this
morning.

In order to solve the /home/home problem, 'umount /home' had been done,
system had been running in a normal file system.
But suddenly /home has been lost.


Key information of that time is as in the **** lines.
And I found in the //// lines messages log.
How do you think the reason of trouble.

********************
# ls -l /home
total 0
# ls -l /
total 3614
-rw-------    1 root root   12288 Apr 15 05:00 aquota.group
-rw-------    1 root root   10240 Apr 15 05:00 aquota.user
dr-xr-xr-x.   2 root root    4096 Nov 15 18:35 bin
dr-xr-xr-x.   5 root root    5120 Nov 15 18:36 boot
drwxr-xr-x    2 root root    4096 Jul 24  2015 cgroup
drwxr-xr-x   17 root root    3820 Apr 14 10:24 dev
drwxr-xr-x.  75 root root    4096 Apr 14 10:31 etc
drwxr-xr-x    2 root root    4096 Nov 15 19:04 home
dr-xr-xr-x.   8 root root    4096 Aug  8  2015 lib
dr-xr-xr-x.   8 root root   12288 Nov 15 18:35 lib64


# ls -l /mnt/extradiskA
total 48
-rw------- 1 root  root   7168 Apr 15 02:14 aquota.group
-rw------- 1 root  root   7168 Apr 14 10:22 aquota.user
drwxr-x--- 2 root  root   4096 Jan 18 21:55 backups
drwxr-xr-x 4 root  root   4096 Jan  1 11:59 home
drwxr-xr-x 8 root  root   4096 Apr 15 01:10 log
drwx------ 2 root  root  16384 Nov 15 18:58 lost+found
drwxr-xr-x 6 mysql mysql  4096 Apr 14 10:24 mysql
# ls -l /mnt/extradiskA/home
total 8
drwxr-xr-x. 7 admin   admin   4096 Sep 21  2015 admin
drwxr-xr-x  4 kavfile kavfile 4096 Jan  1 12:00 kavfile
# mount
/dev/mapper/VolGroup-lv_root on / type ext4 (rw,usrquota,grpquota)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
/dev/vda1 on /boot type ext4 (rw)
/dev/vdb on /mnt/extradiskA type ext4 (rw,usrquota,grpquota)
/mnt/extradiskA/home on /home type none (rw,bind)
/mnt/extradiskA/log on /var/log type none (rw,bind)
/mnt/extradiskA/mysql on /var/lib/mysql type none (rw,bind)
/mnt/extradiskA/.backup on /.backup type none (rw,bind)
/mnt/extradiskA/.daily_backup on /.daily_backup type none (rw,bind)
/mnt/extradiskA/backups on /var/backups type none (rw,bind)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
/ on /tmp/tmproot type none (rw,bind)
# df
Filesystem           1K-blocks    Used Available Use% Mounted on
/dev/mapper/VolGroup-lv_root
                      12778200 3149592   8972840  26% /
tmpfs                   961176       0    961176   0% /dev/shm
/dev/vda1               487652  205564    256488  45% /boot
/dev/vdb              41153856  542076  38514628   2% /mnt/extradiskA

# cat /etc/fstab

#
# /etc/fstab
# Created by anaconda on Mon Jul 28 08:21:01 2014
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/VolGroup-lv_root /                       ext4
 defaults,usrquota,grpquota        1 1
UUID=9a855356-ee00-4ae7-a185-226566ce1dba /boot                   ext4
 defaults        1 2
UUID=093c9b95-8336-4682-9927-132dbdf2f082 /mnt/extradiskA     ext4
 defaults,usrquota,grpquota      1 2
/dev/mapper/VolGroup-lv_swap swap                    swap    defaults
 0 0
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
/mnt/extradiskA/home    /home                   none    bind            0 0
/mnt/extradiskA/log     /var/log                none    bind            0 0
/mnt/extradiskA/mysql    /var/lib/mysql         none    bind            0 0
/mnt/extradiskA/.backup /.backup                none    bind            0 0
/mnt/extradiskA/.daily_backup  /.daily_backup   none    bind            0 0
/mnt/extradiskA/backups  /var/backups           none    bind            0 0


********************

////////////////////////
Apr 15 00:00:03 teisui2 kernel: ------------[ cut here ]------------
Apr 15 00:00:03 teisui2 kernel: WARNING: at fs/ext4/inode.c:3945
ext4_flush_unwritten_io+0x74/0x80 [ext4]() (Not tainted)
Apr 15 00:00:03 teisui2 kernel: Hardware name: KVM
Apr 15 00:00:03 teisui2 kernel: Modules linked in: dazukofs(U) ipv6
xt_multiport iptable_filter ip_tables joydev sg virtio_balloon serio_raw
virtio_net i2c_piix4 i2
c_core ext4 jbd2 mbcache virtio_blk sr_mod cdrom virtio_pci virtio_ring
virtio pata_acpi ata_generic ata_piix dm_mirror dm_region_hash dm_log
dm_mod [last unloaded:
 scsi_wait_scan]
Apr 15 00:00:03 teisui2 kernel: Pid: 1707, comm: spamd Not tainted
2.6.32-573.1.1.el6.x86_64 #1
Apr 15 00:00:03 teisui2 kernel: Call Trace:
Apr 15 00:00:03 teisui2 kernel: [<ffffffff81077491>] ?
warn_slowpath_common+0x91/0xe0
Apr 15 00:00:03 teisui2 kernel: [<ffffffff810774fa>] ?
warn_slowpath_null+0x1a/0x20
Apr 15 00:00:03 teisui2 kernel: [<ffffffffa00aebb4>] ?
ext4_flush_unwritten_io+0x74/0x80 [ext4]
Apr 15 00:00:03 teisui2 kernel: [<ffffffffa00aafc8>] ?
ext4_sync_file+0x88/0x1d0 [ext4]
Apr 15 00:00:03 teisui2 kernel: [<ffffffffa01fd55f>] ?
dazukofs_unlocked_ioctl+0x10f/0x950 [dazukofs]
Apr 15 00:00:03 teisui2 kernel: [<ffffffff811c4461>] ?
vfs_fsync_range+0xa1/0x100
Apr 15 00:00:03 teisui2 kernel: [<ffffffff811c452d>] ? vfs_fsync+0x1d/0x20
Apr 15 00:00:03 teisui2 kernel: [<ffffffff811c456e>] ? do_fsync+0x3e/0x60
Apr 15 00:00:03 teisui2 kernel: [<ffffffff811c45a3>] ?
sys_fdatasync+0x13/0x20
Apr 15 00:00:03 teisui2 kernel: [<ffffffff8100b0d2>] ?
system_call_fastpath+0x16/0x1b
Apr 15 00:00:03 teisui2 kernel: ---[ end trace 6a7331f9bcd6af40 ]---
////////////////////////


2016-04-14 23:18 GMT+09:00 Robert Nichols <rnicholsNOSPAM at comcast.net>:

> On 04/13/2016 08:44 PM, 望月忠雄 wrote:
>
>> # mount
>> /dev/mapper/VolGroup-lv_root on / type ext4 (rw,usrquota,grpquota)
>> proc on /proc type proc (rw)
>> sysfs on /sys type sysfs (rw)
>> devpts on /dev/pts type devpts (rw,gid=5,mode=620)
>> tmpfs on /dev/shm type tmpfs (rw)
>> /dev/vda1 on /boot type ext4 (rw)
>> /dev/vdb on /mnt/extradiskA type ext4 (rw,usrquota,grpquota)
>> /mnt/extradiskA/home on /home type none (rw,bind)
>> /mnt/extradiskA/log on /var/log type none (rw,bind)
>> /mnt/extradiskA/mysql on /var/lib/mysql type none (rw,bind)
>> /mnt/extradiskA/.backup on /.backup type none (rw,bind)
>> /mnt/extradiskA/.daily_backup on /.daily_backup type none (rw,bind)
>> /mnt/extradiskA/backups on /var/backups type none (rw,bind)
>> none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
>> /dev/vdb on /home type ext4
>>
>   ^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
>> (rw,relatime,barrier=1,data=ordered,usrquota,grpquota)
>>
>
> Well, there it is, the extra mount of /dev/vdb on /home.  It's
> not apparent how it got that way.  Since it appears to be at
> the bottom of /etc/mtab, it apparently happened _after_ all
> the other mounts.  Some script must have done that.  If it's
> not in /etc/rc.d/rc.local, then I'd do a progressively wider
> search for references to /home, starting with
>
>     grep -r /home /etc
>
>
> --
> Bob Nichols     "NOSPAM" is really part of my email address.
>                 Do NOT delete it.
>
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> https://lists.centos.org/mailman/listinfo/centos
>