[CentOS] xen guest won't boot after yum update

Sat Jan 5 16:48:53 UTC 2008
Peter Peltonen <peter.peltonen at gmail.com>

After a yum update my two guests that had been shutdown do not start
any more. I have two other guests running that seem to work okay but I
am now afraid that they won't come up again if I restart them.

I am running CentOS5 with kernel 2.6.18-8.1.8.el5xen. Previously I had
following xen versions installed:

Oct 14 23:45:10 Updated: xen-libs.i386 3.0.3-25.0.4.el5
Oct 14 23:46:05 Updated: xen.i386 3.0.3-25.0.4.el5

After a yum update I now have:

Jan 03 22:22:59 Installed: kernel-xen.i686 2.6.18-53.1.4.el5
Jan 03 22:23:55 Updated: xen.i386 3.0.3-41.el5

I have now two guests that won't boot. This is what I get after
issuing `xm create web04`:

Using config file "/etc/xen/web04".
Error: destroyDevice() takes exactly 3 arguments (2 given)

My config file looks likes this:

name = "web04"
memory = "512"
disk = [ 'phy:/dev/VolGroup00/LogVol03-web04,xvda,w', ]
vif = [ 'mac=e9:00:36:fe:dc:e0, bridge=xenbr0', ]
vfb = ["type=vnc,vncunused=1"]
#uuid = "cc64dbee-64f6-4f6b-9d3f-fc3e185206bc"
bootloader="/usr/bin/pygrub"
cpus = "^0,1" # run only on cpu 1
vcpus=1
on_reboot   = 'restart'
on_crash    = 'restart'

This is what comes to /var/log/xen/xend.log:

[2008-01-05 17:59:56 xend.XendDomainInfo 2632] DEBUG (XendDomainInfo:190) XendDo
mainInfo.create(['vm', ['name', 'web04'], ['memory', '512'], ['on_reboot', 'rest
art'], ['on_crash', 'restart'], ['vcpus', 1], ['cpus', '^0,1'], ['bootloader', '
/usr/bin/pygrub'], ['image', ['linux', ['ramdisk', '/var/lib/xen/boot_ramdisk.2C
2kAy'], ['kernel', '/var/lib/xen/boot_kernel.Tg4J_f'], ['args', 'ro root=/dev/Vo
lGroup00/LogVol00']]], ['device', ['vbd', ['uname', 'phy:/dev/VolGroup00/LogVol0
3-web04'], ['dev', 'xvda'], ['mode', 'w']]], ['device', ['vif', ['bridge', 'xenb
r0'], ['mac', 'e9:00:36:fe:dc:e0']]], ['device', ['vkbd']], ['device', ['vfb', [
'vncunused', '1'], ['type', 'vnc'], ['xauthority', '/root/.Xauthority']]]])
[2008-01-05 17:59:56 xend.XendDomainInfo 2632] DEBUG (XendDomainInfo:296) parseC
onfig: config is ['vm', ['name', 'web04'], ['memory', '512'], ['on_reboot', 'res
tart'], ['on_crash', 'restart'], ['vcpus', 1], ['cpus', '^0,1'], ['bootloader',
'/usr/bin/pygrub'], ['image', ['linux', ['ramdisk', '/var/lib/xen/boot_ramdisk.2
C2kAy'], ['kernel', '/var/lib/xen/boot_kernel.Tg4J_f'], ['args', 'ro root=/dev/V
olGroup00/LogVol00']]], ['device', ['vbd', ['uname', 'phy:/dev/VolGroup00/LogVol
03-web04'], ['dev', 'xvda'], ['mode', 'w']]], ['device', ['vif', ['bridge', 'xen
br0'], ['mac', 'e9:00:36:fe:dc:e0']]], ['device', ['vkbd']], ['device', ['vfb',
['vncunused', '1'], ['type', 'vnc'], ['xauthority', '/root/.Xauthority']]]]
[2008-01-05 17:59:56 xend.XendDomainInfo 2632] DEBUG (XendDomainInfo:397) parseC
onfig: result is {'shadow_memory': None, 'start_time': None, 'uuid': None, 'on_c
rash': 'restart', 'on_reboot': 'restart', 'localtime': None, 'image': ['linux',
['ramdisk', '/var/lib/xen/boot_ramdisk.2C2kAy'], ['kernel', '/var/lib/xen/boot_k
ernel.Tg4J_f'], ['args', 'ro root=/dev/VolGroup00/LogVol00']], 'on_poweroff': No
ne, 'bootloader_args': None, 'cpus': [1], 'name': 'web04', 'backend': [], 'vcpus
': 1, 'cpu_weight': None, 'features': None, 'vcpu_avail': None, 'memory': 512, '
device': [('vbd', ['vbd', ['uname', 'phy:/dev/VolGroup00/LogVol03-web04'], ['dev
', 'xvda'], ['mode', 'w']]), ('vif', ['vif', ['bridge', 'xenbr0'], ['mac', 'e9:0
0:36:fe:dc:e0']]), ('vkbd', ['vkbd']), ('vfb', ['vfb', ['vncunused', '1'], ['typ
e', 'vnc'], ['xauthority', '/root/.Xauthority']])], 'bootloader': '/usr/bin/pygr
ub', 'cpu': None, 'maxmem': None}
[2008-01-05 17:59:56 xend.XendDomainInfo 2632] DEBUG (XendDomainInfo:1264) XendD
omainInfo.construct: None
[2008-01-05 17:59:56 xend.XendDomainInfo 2632] DEBUG (XendDomainInfo:1296) XendD
omainInfo.initDomain: 8 1.0
[2008-01-05 17:59:56 xend 2632] DEBUG (balloon:127) Balloon: 4412080 KiB free; n
eed 524288; done.
[2008-01-05 17:59:56 xend 2632] INFO (image:136) buildDomain os=linux dom=8 vcpu
s=1
[2008-01-05 17:59:56 xend 2632] DEBUG (image:199) dom            = 8
[2008-01-05 17:59:56 xend 2632] DEBUG (image:200) image          = /var/lib/xen/
boot_kernel.Tg4J_f
[2008-01-05 17:59:56 xend 2632] DEBUG (image:201) store_evtchn   = 1
[2008-01-05 17:59:56 xend 2632] DEBUG (image:202) console_evtchn = 2
[2008-01-05 17:59:56 xend 2632] DEBUG (image:203) cmdline        =  ro root=/dev
/VolGroup00/LogVol00
[2008-01-05 17:59:56 xend 2632] DEBUG (image:204) ramdisk        = /var/lib/xen/
boot_ramdisk.2C2kAy
[2008-01-05 17:59:56 xend 2632] DEBUG (image:205) vcpus          = 1
[2008-01-05 17:59:56 xend 2632] DEBUG (image:206) features       =
[2008-01-05 17:59:56 xend 2632] DEBUG (blkif:24) exception looking up device num
ber for xvda: [Errno 2] No such file or directory: '/dev/xvda'
[2008-01-05 17:59:56 xend 2632] DEBUG (DevController:110) DevController: writing
 {'backend-id': '0', 'virtual-device': '51712', 'device-type': 'disk', 'state':
'1', 'backend': '/local/domain/0/backend/vbd/8/51712'} to /local/domain/8/device
/vbd/51712.
[2008-01-05 17:59:56 xend 2632] DEBUG (DevController:112) DevController: writing
 {'domain': 'web04', 'frontend': '/local/domain/8/device/vbd/51712', 'dev': 'xvd
a', 'state': '1', 'params': '/dev/VolGroup00/LogVol03-web04', 'mode': 'w', 'onli
ne': '1', 'frontend-id': '8', 'type': 'phy'} to /local/domain/0/backend/vbd/8/51
712.
[2008-01-05 17:59:56 xend 2632] DEBUG (DevController:110) DevController: writing
 {'backend-id': '0', 'mac': 'e9:00:36:fe:dc:e0', 'handle': '0', 'state': '1', 'b
ackend': '/local/domain/0/backend/vif/8/0'} to /local/domain/8/device/vif/0.
[2008-01-05 17:59:56 xend 2632] DEBUG (DevController:112) DevController: writing
 {'bridge': 'xenbr0', 'domain': 'web04', 'handle': '0', 'script': '/etc/xen/scri
pts/vif-bridge', 'state': '1', 'frontend': '/local/domain/8/device/vif/0', 'mac'
: 'e9:00:36:fe:dc:e0', 'online': '1', 'frontend-id': '8'} to /local/domain/0/bac
kend/vif/8/0.
[2008-01-05 17:59:56 xend 2632] DEBUG (DevController:110) DevController: writing
 {'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/vkbd/8/0'
} to /local/domain/8/device/vkbd/0.
[2008-01-05 17:59:56 xend 2632] DEBUG (DevController:112) DevController: writing
 {'frontend-id': '8', 'domain': 'web04', 'frontend': '/local/domain/8/device/vkb
d/0', 'state': '1', 'online': '1'} to /local/domain/0/backend/vkbd/8/0.
[2008-01-05 17:59:56 xend 2632] DEBUG (DevController:110) DevController: writing
 {'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/vfb/8/0'}
 to /local/domain/8/device/vfb/0.
[2008-01-05 17:59:56 xend 2632] DEBUG (DevController:112) DevController: writing
 {'vncunused': '1', 'domain': 'web04', 'frontend': '/local/domain/8/device/vfb/0
', 'xauthority': '/root/.Xauthority', 'state': '1', 'online': '1', 'frontend-id'
: '8', 'type': 'vnc'} to /local/domain/0/backend/vfb/8/0.
[2008-01-05 17:59:56 xend 2632] DEBUG (vfbif:92) No VNC passwd configured for vf
b access
[2008-01-05 17:59:56 xend.XendDomainInfo 2632] DEBUG (XendDomainInfo:715) Storin
g VM details: {'shadow_memory': '0', 'uuid': '546a262f-1696-181f-6785-8355e6a78e
37', 'on_reboot': 'restart', 'start_time': '1199548796.41', 'on_poweroff': 'dest
roy', 'name': 'web04', 'xend/restart_count': '0', 'vcpus': '1', 'vcpu_avail': '1
', 'memory': '512', 'on_crash': 'restart', 'image': "(linux (ramdisk /var/lib/xe
n/boot_ramdisk.2C2kAy) (kernel /var/lib/xen/boot_kernel.Tg4J_f) (args 'ro root=/
dev/VolGroup00/LogVol00'))", 'maxmem': '512'}
[2008-01-05 17:59:56 xend.XendDomainInfo 2632] DEBUG (XendDomainInfo:750) Storin
g domain details: {'console/ring-ref': '630052', 'console/port': '2', 'name': 'w
eb04', 'console/limit': '1048576', 'vm': '/vm/546a262f-1696-181f-6785-8355e6a78e
37', 'domid': '8', 'cpu/0/availability': 'online', 'memory/target': '524288', 's
tore/ring-ref': '630053', 'store/port': '1'}
[2008-01-05 17:59:56 xend.XendDomainInfo 2632] DEBUG (XendDomainInfo:940) XendDo
mainInfo.handleShutdownWatch
[2008-01-05 17:59:56 xend 2632] DEBUG (DevController:143) Waiting for devices vi
f.
[2008-01-05 17:59:56 xend 2632] DEBUG (DevController:149) Waiting for 0.
[2008-01-05 17:59:56 xend 2632] DEBUG (DevController:471) hotplugStatusCallback
/local/domain/0/backend/vif/8/0/hotplug-status.
[2008-01-05 18:01:36 xend 2632] ERROR (xmlrpclib2:184) destroyDevice() takes exa
ctly 3 arguments (2 given)
Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/xen/util/xmlrpclib2.py", line 162, in _
marshaled_dispatch
    response = self._dispatch(method, params)
  File "/usr/lib/python2.4/SimpleXMLRPCServer.py", line 406, in _dispatch
    return func(*params)
  File "<string>", line 1, in <lambda>
  File "/usr/lib/python2.4/site-packages/xen/xend/server/XMLRPCServer.py", line
38, in dispatch
    return getattr(info, fn)(*args)
  File "/usr/lib/python2.4/site-packages/xen/xend/XendDomainInfo.py", line 1610,
 in waitForDevices
    self.testvifsComplete()
  File "/usr/lib/python2.4/site-packages/xen/xend/XendDomainInfo.py", line 1101,
 in waitForDevices_
    self.info.update({'security': [['ssidref', str(info['ssidref'])]]})
  File "/usr/lib/python2.4/site-packages/xen/xend/server/DevController.py", line
 145, in waitForDevices
    return map(self.waitForDevice, self.deviceIDs())
  File "/usr/lib/python2.4/site-packages/xen/xend/server/DevController.py", line
 154, in waitForDevice
    self.destroyDevice(devid)
TypeError: destroyDevice() takes exactly 3 arguments (2 given)
[2008-01-05 18:01:36 xend.XendDomainInfo 2632] DEBUG (XendDomainInfo:1463) XendD
omainInfo.destroy: domid=8
[2008-01-05 18:01:36 xend.XendDomainInfo 2632] DEBUG (XendDomainInfo:1471) XendD
omainInfo.destroyDomain(8)

Is anytone else with CentOS 5 experiencing this kind of problems?
Should I try downgrading?

Regards,
Peter