[CentOS-virt] Problems with scsi-target-utils when hosted on dom0 centos 7 xen box

Wed Apr 13 00:18:32 UTC 2016
Nathan Coulson <nathan at bravenet.com>

On 2016-04-12 09:43 AM, Nathan Coulson wrote:
> By natively, I take it using
> kernel /vmlinuz (vs kernel /xen)
>
> Not yet, but working on setting up such an environment.
>
> (At this time, I was using virt-install to reproduce the problem, and 
> the original server we are testing on did not support kvm but the 2nd 
> server does).
>
> On 2016-04-12 03:26 AM, George Dunlap wrote:
>> On Mon, Apr 11, 2016 at 9:14 PM, Nathan Coulson <nathan at bravenet.com> 
>> wrote:
>>> Hello
>>>
>>> We were attempting to use scsi-target-utils, hosted on a xen dom0 vm 
>>> using
>>> localhost, and running into some problems.  I was not able to 
>>> reproduce this
>>> on a centos 7.2 server using the default kernel.
>> Have you tried booting the Virt SIG kernel natively and seeing if you
>> can reproduce the problem at all?
>>
>> Thanks,
>>   -George


(Apologies for the earlier top post)

Running the kernel natively, on 3.10 or 3.18 (kernel from virt sig)
* CentOS Linux (3.18.25-19.el7.x86_64) 7 (Core))
* CentOS Linux (3.10.0-327.13.1.el7.x86_64) 7 (Core)

It works as expected with no issues.



But when booting as dom0 using the xen hypervisor
* CentOS Linux, with Xen hypervisor

tested with dom0_mem=2048M,max:2048M (as well as 1024M)

the problems occur as I have describe.



>>>
>>> (From dmesg)
>>> Apr  4 11:18:42 funk kernel: [  596.511204]  connection2:0: detected 
>>> conn
>>> error (1022)
>>> Apr  4 11:18:42 funk kernel: connection2:0: ping timeout of 5 secs 
>>> expired,
>>> recv timeout 5, last rx 4295253788, last ping 4295258790, now 
>>> 4295263808
>>> Apr  4 11:18:42 funk kernel: connection2:0: detected conn error (1022)
>>> Apr  4 11:18:42 funk iscsid: Kernel reported iSCSI connection 2:0 error
>>> (1022 - Invalid or unknown error code) state (3)
>>> Apr  4 11:18:44 funk iscsid: connection2:0 is operational after 
>>> recovery (1
>>> attempts)
>>>
>>> Repeated a few times, until eventually
>>>
>>>
>>> Apr  4 11:19:44 funk kernel: Result: hostbyte=DID_TRANSPORT_DISRUPTED
>>> driverbyte=DRIVER_OK
>>> Apr  4 11:19:44 funk kernel: sd 7:0:0:1: [sdd] CDB:
>>> Apr  4 11:19:44 funk kernel: Write(10): 2a 00 01 df c7 e8 00 00 18 00
>>> Apr  4 11:19:44 funk kernel: blk_update_request: I/O error, dev sdd, 
>>> sector
>>> 31442920
>>> Apr  4 11:19:44 funk kernel: [  658.127596] sd 7:0:0:1: [sdd]
>>> Apr  4 11:19:44 funk kernel: [  658.127688] Result:
>>> hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK
>>> Apr  4 11:19:44 funk kernel: [  658.127761] sd 7:0:0:1: [sdd] CDB:
>>> Apr  4 11:19:44 funk kernel: [  658.127826] Write(10): 2a 00 01 df 
>>> c7 e8 00
>>> 00 18 00
>>> Apr  4 11:19:44 funk kernel: [  658.127927] blk_update_request: I/O 
>>> error,
>>> dev sdd, sector 31442920
>>> Apr  4 11:19:44 funk kernel: [  658.128040] sd 7:0:0:1: [sdd]
>>> Apr  4 11:19:44 funk kernel: sd 7:0:0:1: [sdd]
>>> Apr  4 11:19:44 funk kernel: [  658.128105] Result:
>>> hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK
>>> Apr  4 11:19:44 funk kernel: [  658.128177] sd 7:0:0:1: [sdd] CDB:
>>> Apr  4 11:19:44 funk kernel: [  658.128241] Write(10): 2a 00 00 00 
>>> 08 00 00
>>> 00 18 00
>>> Apr  4 11:19:44 funk kernel: [  658.128339] blk_update_request: I/O 
>>> error,
>>> dev sdd, sector 2048
>>> Apr  4 11:19:44 funk kernel: Result: hostbyte=DID_TRANSPORT_DISRUPTED
>>> driverbyte=DRIVER_OK
>>> Apr  4 11:19:44 funk kernel: sd 7:0:0:1: [sdd] CDB:
>>> Apr  4 11:19:44 funk kernel: Write(10): 2a 00 00 00 08 00 00 00 18 00
>>> Apr  4 11:19:44 funk kernel: blk_update_request: I/O error, dev sdd, 
>>> sector
>>> 2048
>>>
>>>
>>> (Test Setup)
>>> scsi-target-utils installed via yum, default config
>>> /etc/tgt/conf.d/xenguests.conf
>>> <target iqn.2016-02.com.bravenet:test>
>>>      backing-store //mnt/vmdisk/test # vm image
>>> </target>
>>>
>>> systemctl tgtd restart
>>>
>>> iscsiadm -m discovery -t sendtargets -p localhost
>>>
>>> iscsiadm -m node -T iqn.2016-02.com.bravenet:test -l
>>>
>>>
>>> add it to lvm (pvcreate, vgcreate), let's call it 
>>> /dev/vmdisk.vg/test.lv
>>>
>>> and then use libvirt to attempt to install an os on 
>>> /dev/vmdisk.vg/test.lv
>>> (using anaconda)
>>>
>>>
>>>
>>>
>>> Around the time it tries to create the disk label, is when the conn 
>>> errors
>>> start, until eventually it gives up trying to create the disk label.
>>>
>>>
>>>
>>> We tested a similar setup on a centos 7.2 host we use kvm based
>>> virtualmachine hosting on (default 3.10 kernel), and it worked 
>>> fine.  It may
>>> be similar to what was reported on
>>> https://bugzilla.redhat.com/show_bug.cgi?id=1245990, but I never saw a
>>> resolution on what they discovered (other then a reference to comment18
>>> which does not appear to exist).
>>>
>>> Testing over the network appears to also work as well (where another 
>>> machine
>>> connects to scsi-target-utils on the funk server above.
>>>
>>>
>>>
>>>
>>>
>>> Longterm Purpose of the above setup, was to get direct access to a
>>> filesystem image hosted on a gluster setup, using bs-type glfs on
>>> scsi-target-utils.
>>>
>>> -- 
>>> Nathan Coulson
>>> www.bravenet.com
>>> nathan at bravenet.com
>>> _______________________________________________
>>> CentOS-virt mailing list
>>> CentOS-virt at centos.org
>>> https://lists.centos.org/mailman/listinfo/centos-virt
>> _______________________________________________
>> CentOS-virt mailing list
>> CentOS-virt at centos.org
>> https://lists.centos.org/mailman/listinfo/centos-virt
>>
>
>


-- 
Nathan Coulson
www.bravenet.com
nathan at bravenet.com