[CentOS] LVM and hotswap (USB/iSCSI) devices?

Sun Aug 31 09:19:46 UTC 2008
Simen Timian Thoresen <simentt at dolphinics.no>

Hi list,

I'm having one of those 'I'm stupid' -problems with LVM on CentOS 5.2. 
I've been working with traditional partitions until now, but I've 
finally been sold on the theoretical benefits of using LVM, but for now 
I only have a huge pile of broken filesystems to show for my efforts.

My scenario;
I attach a disk, either over USB or iSCSI.
I create a PV on this device, create a VG using the device, and slice 
off a tiny LV for tests.

[root at kasse ~]# dmesg
usb 1-3: new high speed USB device using ehci_hcd and address 7
usb 1-3: configuration #1 chosen from 1 choice
scsi27 : SCSI emulation for USB Mass Storage devices
usb-storage: device found at 7
usb-storage: waiting for device to settle before scanning
   Vendor: ST375064  Model: 0A                Rev: 3.AA
   Type:   Direct-Access                      ANSI SCSI revision: 00
SCSI device sdg: 1465149168 512-byte hdwr sectors (750156 MB)
sdg: Write Protect is off
sdg: Mode Sense: 10 00 00 00
sdg: assuming drive cache: write through
SCSI device sdg: 1465149168 512-byte hdwr sectors (750156 MB)
sdg: Write Protect is off
sdg: Mode Sense: 10 00 00 00
sdg: assuming drive cache: write through
  sdg: sdg1
sd 27:0:0:0: Attached scsi disk sdg
sd 27:0:0:0: Attached scsi generic sg6 type 0
usb-storage: device scan complete
[root at kasse ~]# pvcreate /dev/sdg1
   Physical volume "/dev/sdg1" successfully created
[root at kasse ~]# vgcreate testgroup /dev/sdg1
   Volume group "testgroup" successfully created
[root at kasse ~]# lvcreate -L 200M testgroup -n testLV
   Logical volume "testLV" created


At this point, I have /dev/testgroup/testLV, which I can stick a fs on, 
mount, store files on, unmount, remount and recover the files from. Just 
as I'd expect.

[root at kasse ~]# lvdisplay
   --- Logical volume ---
   LV Name                /dev/testgroup/testLV
   VG Name                testgroup
   LV UUID                3KjDiZ-gTAD-sC9E-sOSA-772o-41Yk-ZKK7cx
   LV Write Access        read/write
   LV Status              available
   # open                 0
   LV Size                200.00 MB
   Current LE             50
   Segments               1
   Allocation             inherit
   Read ahead sectors     auto
   - currently set to     256
   Block device           253:1


I have not explicitly set the vg active, as I understand that it 
implicitly will be set active in non-clustered environments like mine. 
LVM works as I'd expect it to up until here in any case.

So - with the LV unmounted, I power my USB-device down, and then back up;

[root at kasse ~]# dmesg
usb 1-3: USB disconnect, address 7
usb 1-3: new high speed USB device using ehci_hcd and address 8
usb 1-3: configuration #1 chosen from 1 choice
scsi28 : SCSI emulation for USB Mass Storage devices
usb-storage: device found at 8
usb-storage: waiting for device to settle before scanning
   Vendor: ST375064  Model: 0A                Rev: 3.AA
   Type:   Direct-Access                      ANSI SCSI revision: 00
SCSI device sdi: 1465149168 512-byte hdwr sectors (750156 MB)
sdi: Write Protect is off
sdi: Mode Sense: 10 00 00 00
sdi: assuming drive cache: write through
SCSI device sdi: 1465149168 512-byte hdwr sectors (750156 MB)
sdi: Write Protect is off
sdi: Mode Sense: 10 00 00 00
sdi: assuming drive cache: write through
  sdi: sdi1
sd 28:0:0:0: Attached scsi disk sdi
sd 28:0:0:0: Attached scsi generic sg6 type 0
usb-storage: device scan complete

(note that my disk is now sdi, not sdg, and connected on scsi28 and not 
scsi27)

If I now try to access the device, all falls appart;
[root at kasse ~]# lvdisplay
   /dev/testgroup/testLV: read failed after 0 of 4096 at 209649664: 
Input/output error
   /dev/testgroup/testLV: read failed after 0 of 4096 at 209707008: 
Input/output error
   /dev/testgroup/testLV: read failed after 0 of 4096 at 0: Input/output 
error
   /dev/testgroup/testLV: read failed after 0 of 4096 at 4096: 
Input/output error
   /dev/testgroup/testLV: read failed after 0 of 4096 at 0: Input/output 
error
   --- Logical volume ---
   LV Name                /dev/testgroup/testLV
   VG Name                testgroup
   LV UUID                3KjDiZ-gTAD-sC9E-sOSA-772o-41Yk-ZKK7cx
   LV Write Access        read/write
   LV Status              available
   # open                 0
   LV Size                200.00 MB
   Current LE             50
   Segments               1
   Allocation             inherit
   Read ahead sectors     auto
   - currently set to     256
   Block device           253:1


Trying to mount or otherwise access the LV also shows the problems;


scsi 27:0:0:0: rejecting I/O to dead device
scsi 27:0:0:0: rejecting I/O to dead device
scsi 27:0:0:0: rejecting I/O to dead device
scsi 27:0:0:0: rejecting I/O to dead device
scsi 27:0:0:0: rejecting I/O to dead device
scsi 27:0:0:0: rejecting I/O to dead device
printk: 25 messages suppressed.
Buffer I/O error on device dm-1, logical block 30
Buffer I/O error on device dm-1, logical block 31
scsi 27:0:0:0: rejecting I/O to dead device
Buffer I/O error on device dm-1, logical block 0
Buffer I/O error on device dm-1, logical block 1
Buffer I/O error on device dm-1, logical block 2
Buffer I/O error on device dm-1, logical block 3
Buffer I/O error on device dm-1, logical block 4
Buffer I/O error on device dm-1, logical block 5
Buffer I/O error on device dm-1, logical block 6
Buffer I/O error on device dm-1, logical block 7
scsi 27:0:0:0: rejecting I/O to dead device
scsi 27:0:0:0: rejecting I/O to dead device
scsi 27:0:0:0: rejecting I/O to dead device
scsi 27:0:0:0: rejecting I/O to dead device
hfs: unable to find HFS+ superblock

So - LVM seems to believe my LV is still attached to scsi27, while it 
now actually sits on scsi28.

What am I missing here?

How do I get LVM to use a persistent device instead of a transient path?

Yours,
-S
-- 
Simen Thoresen, Dolphin ICS
Systems Administration and Wulfkit Support