Hi!
I am experiencing strange behavior with my new CentOS 6.3
installation (system up to date). I have a big 11 TB EXT4 array mounted
on /home/data. The server is a Tyan 2U (TA26-B3992-E) with Dual Opteron
2216, one cpu has 4 Gigs RAM and the other has 8 Gigs (so 12 Gigs
total). The server uses 8 x Seagate 2TB SAS hard disks (7200 RPM) and
all the disks are grouped together to form 1 x 500 Gigs RAID 6 array
(for CentOS) and 1 x 11 TB RAID 6 for data storage (mounted on
/home/data). The controller is a LSI SAS 9265-8i:
[root at radium ~]# lspci
00:01.0 PCI bridge: Broadcom BCM5785 [HT1000] PCI/PCI-X Bridge
00:02.0 Host bridge: Broadcom BCM5785 [HT1000] Legacy South Bridge
00:02.1 IDE interface: Broadcom BCM5785 [HT1000] IDE
00:02.2 ISA bridge: Broadcom BCM5785 [HT1000] LPC
00:03.0 USB controller: Broadcom BCM5785 [HT1000] USB (rev 01)
00:03.1 USB controller: Broadcom BCM5785 [HT1000] USB (rev 01)
00:03.2 USB controller: Broadcom BCM5785 [HT1000] USB (rev 01)
00:04.0 Ethernet controller: Intel Corporation 82557/8/9/0/1 Ethernet
Pro 100 (rev 10)
00:06.0 PCI bridge: Broadcom BCM5780 [HT2000] PCI-X bridge (rev b0)
00:07.0 PCI bridge: Broadcom BCM5780 [HT2000] PCI-X bridge (rev b0)
00:08.0 PCI bridge: Broadcom BCM5780 [HT2000] PCI-Express Bridge (rev b0)
00:09.0 PCI bridge: Broadcom BCM5780 [HT2000] PCI-Express Bridge (rev b0)
00:0a.0 PCI bridge: Broadcom BCM5780 [HT2000] PCI-Express Bridge (rev b0)
00:0b.0 PCI bridge: Broadcom BCM5780 [HT2000] PCI-Express Bridge (rev b0)
00:0c.0 VGA compatible controller: Advanced Micro Devices [AMD] nee ATI
ES1000 (rev 02)
00:18.0 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron]
HyperTransport Technology Configuration
00:18.1 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron]
Address Map
00:18.2 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron]
DRAM Controller
00:18.3 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron]
Miscellaneous Control
00:19.0 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron]
HyperTransport Technology Configuration
00:19.1 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron]
Address Map
00:19.2 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron]
DRAM Controller
00:19.3 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron]
Miscellaneous Control
01:0d.0 PCI bridge: Broadcom BCM5785 [HT1000] PCI/PCI-X Bridge (rev c0)
04:04.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5780
Gigabit Ethernet (rev 10)
04:04.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5780
Gigabit Ethernet (rev 10)
05:00.0 RAID bus controller: LSI Logic / Symbios Logic MegaRAID SAS 2208
[Thunderbolt] (rev 01)
I have KVM/QEMU running 3 VMs: 2 stored in the 500 gigs array and
one on the 11 TB array.
I have a couple of samba shares from the 11 TB array (all of them
are on the 11TB array). The strange behavior is that if i "ls -al
/home/data", i have to wait 7-8 seconds to see the output. If i do a
simple ls, output is instantaneous. I see about the same delay when i
access the samba shares from the server (related ?). I don't know where
to start with this!
Here are other informations:
[root at radium data]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_radium-lv_root 50G 7.4G 40G 16% /
tmpfs 5.9G 88K 5.9G 1% /dev/shm
/dev/sda1 485M 92M 368M 21% /boot
/dev/mapper/vg_radium-lv_home 437G 4.9G 410G 2% /home
/dev/mapper/vg_radium2-lv_data 11T 6.1T 3.7T 63% /home/data
[root at radium data]# uname -a
Linux radium 2.6.32-279.19.1.el6.x86_64 #1 SMP Wed Dec 19 07:05:20 UTC
2012 x86_64 x86_64 x86_64 GNU/Linux
[root at radium data]# rpm -qa |grep samba
samba-winbind-clients-3.5.10-125.el6.x86_64
samba-3.5.10-125.el6.x86_64
samba-client-3.5.10-125.el6.x86_64
samba-winbind-3.5.10-125.el6.x86_64
samba-common-3.5.10-125.el6.x86_64
[root at radium data]# pvdisplay
--- Physical volume ---
PV Name /dev/sdb
VG Name vg_radium2
PV Size 10.42 TiB / not usable 4.12 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 2732799
Free PE 0
Allocated PE 2732799
PV UUID vTDqS3-wVLz-ZZfu-yfKi-dE6k-rYwr-NVMPcA
--- Physical volume ---
PV Name /dev/sda2
VG Name vg_radium
PV Size 499.51 GiB / not usable 2.00 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 127874
Free PE 0
Allocated PE 127874
PV UUID alKttt-X0HC-WT4X-ieju-VHqa-YcvU-gHR6Px
[root at radium ~]# lvdisplay
--- Logical volume ---
LV Path /dev/vg_radium2/lv_data
LV Name lv_data
VG Name vg_radium2
LV UUID pDGfTc-qD3t-p5c3-FDHH-z0HS-pUKq-dClZwM
LV Write Access read/write
LV Creation host, time radium.canal.qc.ca, 2012-11-28 15:52:17 -0500
LV Status available
# open 1
LV Size 10.42 TiB
Current LE 2732799
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:2
--- Logical volume ---
LV Path /dev/vg_radium/lv_root
LV Name lv_root
VG Name vg_radium
LV UUID Lp5Mkd-Ht2J-XBfF-Yafx-w1HY-nIlN-1771T3
LV Write Access read/write
LV Creation host, time radium.canal.qc.ca, 2012-11-28 13:57:36 -0500
LV Status available
# open 1
LV Size 50.00 GiB
Current LE 12800
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:0
--- Logical volume ---
LV Path /dev/vg_radium/lv_home
LV Name lv_home
VG Name vg_radium
LV UUID VCvRSg-x1On-9GgR-09we-yIx6-uMvi-dYO2UZ
LV Write Access read/write
LV Creation host, time radium.canal.qc.ca, 2012-11-28 13:57:40 -0500
LV Status available
# open 1
LV Size 443.59 GiB
Current LE 113560
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:3
--- Logical volume ---
LV Path /dev/vg_radium/lv_swap
LV Name lv_swap
VG Name vg_radium
LV UUID GZorIv-iovS-n2S2-JkSe-bDUl-kwhk-E6potz
LV Write Access read/write
LV Creation host, time radium.canal.qc.ca, 2012-11-28 13:58:01 -0500
LV Status available
# open 1
LV Size 5.91 GiB
Current LE 1514
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:1
I did directly over lvm:
mkfs.ext4 /dev/vg_radium2/lv_data
[root at radium ~]# parted
GNU Parted 2.1
Using /dev/sda
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) select /dev/sdb
Using /dev/sdb
(parted) print
Error: /dev/sdb: unrecognised disk label
/etc/fstab
=========
/dev/mapper/vg_radium-lv_root / ext4
defaults,acl 1 1
UUID=2169688f-e830-458d-b31e-25aa0dc9c753 /boot ext4
defaults 1 2
/dev/mapper/vg_radium-lv_home /home ext4
defaults 1 2
/dev/mapper/vg_radium-lv_swap swap swap
defaults 0 0
tmpfs /dev/shm tmpfs
defaults 0 0
devpts /dev/pts devpts
gid=5,mode=620 0 0
sysfs /sys sysfs
defaults 0 0
proc /proc proc
defaults 0 0
/dev/mapper/vg_radium2-lv_data /home/data ext4
defaults,acl,noatime 0 3
Sorry for the long post!
Guy Boisvert
IngTegration inc.
http://www.ingtegration.com