/ On Mon, 2007-12-10 at 22:34 +0000, Karanbir Singh wrote:

/>>/ Just wondering if people had started using i386 Xen DomU's on a x86_64 />>/ dom0 machine with 5.1 as yet ? And just wondering what their experiences />>/ have been.

/I thought I would chime in here with my experience. I am running a Cent5.1 x86_64 Dom0 with a Cent 5.1 x386 domU (both fully updated).

Everything works well including installation and subsequent "xm create" boots. A gotcha that I have not been able to work around has to do with the automatic save/restore with the xendomains script.

When I mimic Dom0 shutdown behavior (i.e., xendomains stop) and run "xm save sdatest /var/lib/xen/save/sdatest" the following gets generated to xend.log

[2008-03-24 18:33:29 xend.XendDomainInfo 19801] INFO (XendDomainInfo:1719) Dev 51728 still active, looping... [2008-03-24 18:33:29 xend.XendDomainInfo 19801] DEBUG (XendDomainInfo:1611) XendDomainInfo.resumeDomain: devices released [2008-03-24 18:33:29 xend.XendDomainInfo 19801] DEBUG (XendDomainInfo:988) XendDomainInfo.handleShutdownWatch [2008-03-24 18:33:29 xend.XendDomainInfo 19801] DEBUG (XendDomainInfo:791) Storing domain details: {'console/ring-ref': '6706833', 'console/port': '2', 'name': 'migrating-sdatest', 'console/limit': '1048576', 'vm': '/vm/a9b4b850-95ee-1364-6b2b-5e79375ce47e', 'domid': '1', 'cpu/0/availability': 'online', 'memory/target': '131072', 'store/ring-ref': '6706834', 'store/port': '1'} [2008-03-24 18:33:29 xend 19801] DEBUG (blkif:24) exception looking up device number for xvda: [Errno 2] No such file or directory: '/dev/xvda' [2008-03-24 18:33:29 xend 19801] DEBUG (DevController:110) DevController: writing {'virtual-device': '51712', 'device-type': 'disk', 'protocol': 'x86_32-abi', 'backend-id': '0', 'state': '1', 'backend': '/local/domain/0/backend/vbd/1/51712'} to /local/domain/1/device/vbd/51712. [2008-03-24 18:33:29 xend 19801] DEBUG (DevController:112) DevController: writing {'domain': 'migrating-sdatest', 'frontend': '/local/domain/1/device/vbd/51712', 'dev': 'xvda', 'state': '1', 'params': '/dev/VolGroup00/sdatest.boot', 'mode': 'w', 'online': '1', 'frontend-id': '1', 'type': 'phy'} to /local/domain/0/backend/vbd/1/51712. [2008-03-24 18:33:29 xend 19801] DEBUG (blkif:24) exception looking up device number for xvdb: [Errno 2] No such file or directory: '/dev/xvdb' [2008-03-24 18:33:29 xend 19801] DEBUG (DevController:110) DevController: writing {'virtual-device': '51728', 'device-type': 'disk', 'protocol': 'x86_32-abi', 'backend-id': '0', 'state': '1', 'backend': '/local/domain/0/backend/vbd/1/51728'} to /local/domain/1/device/vbd/51728. [2008-03-24 18:33:29 xend 19801] DEBUG (DevController:112) DevController: writing {'domain': 'migrating-sdatest', 'frontend': '/local/domain/1/device/vbd/51728', 'dev': 'xvdb', 'state': '1', 'params': '/dev/VolGroup00/sdatest', 'mode': 'w', 'online': '1', 'frontend-id': '1', 'type': 'phy'} to /local/domain/0/backend/vbd/1/51728. [2008-03-24 18:33:29 xend 19801] DEBUG (DevController:110) DevController: writing {'mac': '00:16:3e:00:00:11', 'handle': '0', 'protocol': 'x86_32-abi', 'backend-id': '0', 'state': '1', 'backend': '/local/domain/0/backend/vif/1/0'} to /local/domain/1/device/vif/0. [2008-03-24 18:33:29 xend 19801] DEBUG (DevController:112) DevController: writing {'bridge': 'xenbr0', 'domain': 'migrating-sdatest', 'handle': '0', 'script': '/etc/xen/scripts/vif-bridge', 'state': '1', 'frontend': '/local/domain/1/device/vif/0', 'mac': '00:16:3e:00:00:11', 'online': '1', 'frontend-id': '1'} to /local/domain/0/backend/vif/1/0. [2008-03-24 18:33:29 xend.XendDomainInfo 19801] DEBUG (XendDomainInfo:1623) XendDomainInfo.resumeDomain: devices created [2008-03-24 18:33:29 xend.XendDomainInfo 19801] ERROR (XendDomainInfo:1628) XendDomainInfo.resume: xc.domain_resume failed on domain 1. Traceback (most recent call last): File "/usr/lib64/python2.4/site-packages/xen/xend/XendDomainInfo.py", line 1625, in resumeDomain xc.domain_resume(self.domid, fast) Error: (1, 'Internal error', "Couldn't map p2m_frame_list_list") [2008-03-24 18:33:29 xend 19801] DEBUG (XendCheckpoint:136) XendCheckpoint.save: resumeDomain ERROR Internal error: Couldn't map p2m_frame_list_list

There are two errors of note -- [2008-03-24 18:33:29 xend 19801] DEBUG (blkif:24) exception looking up device number for xvda: [Errno 2] No such file or directory: '/dev/xvda'

and

ERROR Internal error: Couldn't map p2m_frame_list_list

Someone with greater knowledge of Xen's internal workings might comment as to whether they're related errors.

It looks like this is a known issue with Xen based on the following bug report (which, alas, is not showing any activity). http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1064

I suppose a temporary workaround (for a development environment) is to alter the xendomains script so that a save is not conducted but a shutdown followed by a destroy.

Cheers, Frans