Hi list,
I installed two new servers to be our virtualization HA. This servers have CentOS 5.2, Xen 3.0.3, Drbd 8.2.6. The setup is ok, lvm drbd resources sincronizing, Guest VM running without any problem.
My only doubt is that I can only use drbd resources inside guest vms when I specify the disk resource in vms config like this:
disk = ['phy:/dev/VolGroup00/vappprtem01,hda,w' ]
if I specify de disk like:
disk = [drbd:vappprtem01,hda,w' ]
The Guest VM found the disk but this appear with zero size.
As far what I read the second way is the correct to live migration work without any race condiction...
This link say that centos has a problem with block-drbd script provided by drbd, is anybody aware of that ?
http://lists.xensource.com/archives/html/xen-users/2008-08/msg00176.html
Any hints ?
Best Regards,
On 8/25/08, Alexandre Biancalana biancalana@gmail.com wrote:
Hi list,
I installed two new servers to be our virtualization HA. This servers have CentOS 5.2, Xen 3.0.3, Drbd 8.2.6. The setup is ok, lvm drbd resources sincronizing, Guest VM running without any problem.
My only doubt is that I can only use drbd resources inside guest vms when I specify the disk resource in vms config like this:
disk = ['phy:/dev/VolGroup00/vappprtem01,hda,w' ]
if I specify de disk like:
disk = [drbd:vappprtem01,hda,w' ]
The Guest VM found the disk but this appear with zero size.
As far what I read the second way is the correct to live migration work without any race condiction...
This link say that centos has a problem with block-drbd script provided by drbd, is anybody aware of that ?
http://lists.xensource.com/archives/html/xen-users/2008-08/msg00176.html
This is a know problem ??? Any ideas ???
Regards,
Hi,
after i've read this post
http://article.gmane.org/gmane.comp.emulators.xen.user/33157/match=re+xen+dr...
I've given up on xen and drbd disks for now, and am using phy:, but you'll have to give it drbd device not the volume
disk = [ "phy:/dev/drbd14,xvda,w" ]
Unfortunately, that means that you'll have to handle setting drbd devices primary/secondary yourself. I've written a wrapper script around drbdadm+xm and it has been working more or less ok.
That also means no live migration since drbd device needs to change state sometime mid-migration, but save/move/restore has been working ok for me with really minimal downtimes.
There is also a primary/primary setup with newer drbd but it was awfully unstable when i was testing it a few moths ago, most of the time it was crashing and taking down the complete cluster with it.
regards,
Alexandre Biancalana wrote:
Hi list,
I installed two new servers to be our virtualization HA. This servers have CentOS 5.2, Xen 3.0.3, Drbd 8.2.6. The setup is ok, lvm drbd resources sincronizing, Guest VM running without any problem.
My only doubt is that I can only use drbd resources inside guest vms when I specify the disk resource in vms config like this:
disk = ['phy:/dev/VolGroup00/vappprtem01,hda,w' ]
if I specify de disk like:
disk = [drbd:vappprtem01,hda,w' ]
The Guest VM found the disk but this appear with zero size.
As far what I read the second way is the correct to live migration work without any race condiction...
This link say that centos has a problem with block-drbd script provided by drbd, is anybody aware of that ?
http://lists.xensource.com/archives/html/xen-users/2008-08/msg00176.html
Any hints ?
Best Regards, _______________________________________________ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Aleksandar Ivanisevic ha scritto:
Hi,
after i've read this post
http://article.gmane.org/gmane.comp.emulators.xen.user/33157/match=re+xen+dr...
I've given up on xen and drbd disks for now, and am using phy:, but you'll have to give it drbd device not the volume
disk = [ "phy:/dev/drbd14,xvda,w" ]
Unfortunately, that means that you'll have to handle setting drbd devices primary/secondary yourself. I've written a wrapper script around drbdadm+xm and it has been working more or less ok.
That also means no live migration since drbd device needs to change state sometime mid-migration, but save/move/restore has been working ok for me with really minimal downtimes.
There is also a primary/primary setup with newer drbd but it was awfully unstable when i was testing it a few moths ago, most of the time it was crashing and taking down the complete cluster with it.
regards,
Hi Aleksandar,
I was just starting doing some test following this article:
http://fghaas.wordpress.com/2007/09/03/drbd-806-brings-full-live-migration-f...
So if you say that doesn't work I'll give up on this and I'll go directly to Openvz (actually proxmox)
Thanks
Lorenzo
Lorenzo Quatrini wrote:
I was just starting doing some test following this article:
http://fghaas.wordpress.com/2007/09/03/drbd-806-brings-full-live-migration-f...
So if you say that doesn't work I'll give up on this and I'll go directly to Openvz (actually proxmox)
Eh, never trust someone you met over the Internet ;) I've actually used that same page as a starting point for my tests and it's definitely not usable for me.
Don't know much about proxmox but it seems to be the right approach, use openvz for stuff that is distro/kernel/OS agnostic and KVM/XEN/whatever for full virtualization.
I'm now trying to achieve the same thing using openvz-xen kernel, but only not to waste all the work i've invested in getting the xen cluster going. Maybe I'll move to kvm after all, but I don't know yet if it would read xen disk images.
Alexandre Biancalana wrote:
Hi list,
I installed two new servers to be our virtualization HA. This servers have CentOS 5.2, Xen 3.0.3, Drbd 8.2.6. The setup is ok, lvm drbd resources sincronizing, Guest VM running without any problem.
My only doubt is that I can only use drbd resources inside guest vms when I specify the disk resource in vms config like this:
disk = ['phy:/dev/VolGroup00/vappprtem01,hda,w' ]
if I specify de disk like:
disk = [drbd:vappprtem01,hda,w' ]
The Guest VM found the disk but this appear with zero size.
As far what I read the second way is the correct to live migration work without any race condiction...
This link say that centos has a problem with block-drbd script provided by drbd, is anybody aware of that ?
http://lists.xensource.com/archives/html/xen-users/2008-08/msg00176.html
Any hints ?
Just specify the drbd devices directly as:
disk = ['phy:/dev/drbd0,hda,w' ]
And you should be ok. There is no need to use the scripts.
Be sure you don't specify the underlying device, but the drbd device!
Using the underlying device will cause the replicas to become out-of-sync as what is written to the underlying device will not be replicated and a consistency check will bring your drbd resources offline!
You are also going to want to use the multi-master as master-slave is too difficult to get to work properly between xend and heartbeat!
-Ross
______________________________________________________________________ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof.
On 8/26/08, Aleksandar Ivanisevic aleksandar.ivanisevic@2e-systems.com wrote:
Ross S. W. Walker wrote:
Just specify the drbd devices directly as:
disk = ['phy:/dev/drbd0,hda,w' ]
And you should be ok. There is no need to use the scripts.
You actually got it working stably, live migration and all? which kernel, xen, drbd if I may ask?
What about that ?
http://fghaas.wordpress.com/2007/09/03/drbd-806-brings-full-live-migration-f...
Alexandre Biancalana wrote:
On 8/26/08, Aleksandar Ivanisevic aleksandar.ivanisevic@2e-systems.com wrote:
Ross S. W. Walker wrote:
Just specify the drbd devices directly as:
disk = ['phy:/dev/drbd0,hda,w' ]
And you should be ok. There is no need to use the scripts.
You actually got it working stably, live migration and
all? which kernel,
xen, drbd if I may ask?
What about that ?
http://fghaas.wordpress.com/2007/09/03/drbd-806-brings-full-li ve-migration-for-xen-on-drbd/
Are you running Xen 3.2 from xen.org?
If not then specify your devices as the top portion of the blog talks about, then config drbd with the allow-two-primaries as the bottom talks about, and ignore the middle.
If you upgrade to xen.org release of Xen then you can use the block-drbd script to specify by drbd resource name instead of device, but that's just fluff in stuff, and doesn't have anything to do with the ability to live migrate which is all handled by the allow-two-primaries option.
-Ross
______________________________________________________________________ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof.
On 8/26/08, Ross S. W. Walker RWalker@medallion.com wrote:
Alexandre Biancalana wrote:
On 8/26/08, Aleksandar Ivanisevic aleksandar.ivanisevic@2e-systems.com wrote:
Ross S. W. Walker wrote:
Just specify the drbd devices directly as:
disk = ['phy:/dev/drbd0,hda,w' ]
And you should be ok. There is no need to use the scripts.
You actually got it working stably, live migration and
all? which kernel,
xen, drbd if I may ask?
What about that ?
http://fghaas.wordpress.com/2007/09/03/drbd-806-brings-full-li ve-migration-for-xen-on-drbd/
Are you running Xen 3.2 from xen.org?
If not then specify your devices as the top portion of the blog talks about, then config drbd with the allow-two-primaries as the bottom talks about, and ignore the middle.
If you upgrade to xen.org release of Xen then you can use the block-drbd script to specify by drbd resource name instead of device, but that's just fluff in stuff, and doesn't have anything to do with the ability to live migrate which is all handled by the allow-two-primaries option.
Are you saying that we can only use block-drbd script with Xen 3.2 ?
Alexandre Biancalana wrote:
On 8/26/08, Ross S. W. Walker RWalker@medallion.com wrote:
Alexandre Biancalana wrote:
On 8/26/08, Aleksandar Ivanisevic aleksandar.ivanisevic@2e-systems.com wrote:
Ross S. W. Walker wrote:
Just specify the drbd devices directly as:
disk = ['phy:/dev/drbd0,hda,w' ]
And you should be ok. There is no need to use the scripts.
You actually got it working stably, live migration and all? which kernel, xen, drbd if I may ask?
What about that ?
http://fghaas.wordpress.com/2007/09/03/drbd-806-brings-full-li ve-migration-for-xen-on-drbd/
Are you running Xen 3.2 from xen.org?
If not then specify your devices as the top portion of the blog talks about, then config drbd with the allow-two-primaries as the bottom talks about, and ignore the middle.
If you upgrade to xen.org release of Xen then you can use the block-drbd script to specify by drbd resource name instead of device, but that's just fluff in stuff, and doesn't have anything to do with the ability to live migrate which is all handled by the allow-two-primaries option.
Are you saying that we can only use block-drbd script with Xen 3.2 ?
It might work with the Xen 3.1 libraries too, but that's also irrelevant as CentOS/RHEL has decided to use the Xen 3.0.3 libraries (though they are using the Xen 3.1 hypervisor, go figure!).
Linbit wrote the block-drbd script for Xen 3.1 (maybe 3.2) and up, but definitely not pre-3.1.
For the definitive answer though I recommend posting on drbd-user. Someone there might even have a work-around to get it to work on earlier Xen libraries...
But like I said it's just fluff and stuff and specifying the drbd device directly will work just the same and there is no fear of it breaking in the future if the Xen libs change or some feature in bash changes!
-Ross
______________________________________________________________________ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof.
On Tue, Aug 26, 2008 at 11:25 AM, Ross S. W. Walker RWalker@medallion.com wrote:
Alexandre Biancalana wrote:
On 8/26/08, Ross S. W. Walker RWalker@medallion.com wrote:
If you upgrade to xen.org release of Xen then you can use the block-drbd script to specify by drbd resource name instead of device, but that's just fluff in stuff, and doesn't have anything to do with the ability to live migrate which is all handled by the allow-two-primaries option.
True. In my case : CentOS 5.3, incl. kernel-xen and heartbeat, Xen.org 3.x, DRBD 8.2. It works, but has no cookie-cutter howto.
Are you saying that we can only use block-drbd script with Xen 3.2 ?
It's really not so much the Xen version, but the distro.
It might work with the Xen 3.1 libraries too, but that's also irrelevant as CentOS/RHEL has decided to use the Xen 3.0.3 libraries (though they are using the Xen 3.1 hypervisor, go figure!).
How did you determine they are different versions?
Linbit wrote the block-drbd script for Xen 3.1 (maybe 3.2) and up, but definitely not pre-3.1.
For the definitive answer though I recommend posting on drbd-user. Someone there might even have a work-around to get it to work on earlier Xen libraries...
http://lists.linbit.com/pipermail/drbd-user/2008-August/010077.html
jerry
yes. i replied to myself.
On Tue, Aug 26, 2008 at 10:56 PM, Jerry Amundson jamundso@gmail.com wrote:
On Tue, Aug 26, 2008 at 11:25 AM, Ross S. W. Walker RWalker@medallion.com wrote:
Alexandre Biancalana wrote:
On 8/26/08, Ross S. W. Walker RWalker@medallion.com wrote:
If you upgrade to xen.org release of Xen then you can use the block-drbd script to specify by drbd resource name instead of device, but that's just fluff in stuff, and doesn't have anything to do with the ability to live migrate which is all handled by the allow-two-primaries option.
True. In my case : CentOS 5.3, incl. kernel-xen and heartbeat, Xen.org 3.x, DRBD 8.2. It works, but has no cookie-cutter howto.
Are you saying that we can only use block-drbd script with Xen 3.2 ?
It's really not so much the Xen version, but the distro.
more so, within the distro, the bootstrap method used for the domU
It might work with the Xen 3.1 libraries too, but that's also irrelevant as CentOS/RHEL has decided to use the Xen 3.0.3 libraries (though they are using the Xen 3.1 hypervisor, go figure!).
How did you determine they are different versions?
oh, maybe the thread I just noticed in xen-users? http://lists.xensource.com/archives/html/xen-users/2008-08/msg00791.html where Pasi says, -> To be more clear about it, RHEL 5.2 version of Xen _hypervisor_ is 3.1.2 + -> patches. -> RHEL 5.2 kernel-xen is based on older Xen (3.0.3 iirc) kernel code.
Linbit wrote the block-drbd script for Xen 3.1 (maybe 3.2) and up, but definitely not pre-3.1.
For the definitive answer though I recommend posting on drbd-user. Someone there might even have a work-around to get it to work on earlier Xen libraries...
http://lists.linbit.com/pipermail/drbd-user/2008-August/010077.html
jerry
-- There's plenty of youth in America - it's time we find the "fountain of smart".
Jerry Amundson wrote:
yes. i replied to myself.
Talking to yourself is the first sign of insanity!
The second being, disagreeing with the first.
On Tue, Aug 26, 2008 at 10:56 PM, Jerry Amundson jamundso@gmail.com wrote:
On Tue, Aug 26, 2008 at 11:25 AM, Ross S. W. Walker RWalker@medallion.com wrote:
Alexandre Biancalana wrote:
On 8/26/08, Ross S. W. Walker RWalker@medallion.com wrote:
If you upgrade to xen.org release of Xen then you can use the block-drbd script to specify by drbd resource name instead of device, but that's just fluff in stuff, and doesn't have anything to do with the ability to live migrate which is all handled by the allow-two-primaries option.
True. In my case : CentOS 5.3, incl. kernel-xen and heartbeat, Xen.org 3.x, DRBD 8.2. It works, but has no cookie-cutter howto.
Are you saying that we can only use block-drbd script with Xen 3.2 ?
It's really not so much the Xen version, but the distro.
more so, within the distro, the bootstrap method used for the domU
More specifically 'pygrub' itself as mentioned below.
It might work with the Xen 3.1 libraries too, but that's also irrelevant as CentOS/RHEL has decided to use the Xen 3.0.3 libraries (though they are using the Xen 3.1 hypervisor, go figure!).
How did you determine they are different versions?
oh, maybe the thread I just noticed in xen-users? http://lists.xensource.com/archives/html/xen-users/2008-08/msg00791.html where Pasi says, -> To be more clear about it, RHEL 5.2 version of Xen _hypervisor_ is 3.1.2 + -> patches. -> RHEL 5.2 kernel-xen is based on older Xen (3.0.3 iirc) kernel code.
Actually I found the version information through a 'xm info' which shows it's running 3.1, but the libraries and supporting apps are all 3.0.3.
xen_major : 3 xen_minor : 1 xen_extra : .2-92.1.10.el5 xen_caps : xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p
Linbit wrote the block-drbd script for Xen 3.1 (maybe 3.2) and up, but definitely not pre-3.1.
For the definitive answer though I recommend posting on drbd-user. Someone there might even have a work-around to get it to work on earlier Xen libraries...
http://lists.linbit.com/pipermail/drbd-user/2008-August/010077.html
Ah so the culprit is pygrub, add that tidbit to my knowledge base.
So if you define the kernel and ramdisk from external sources then the block script should work, but using pygrub, it only recognizes standard block devices.
I suppose this will also affect the block-iscsi script out there too...
It would be nice if someone could patch pygrub to look for handler scripts for any device NOT file, tap, phy and to use them to bootload as it's better IMHO to manage the domU's kernel and ramdisk from within the domU.
-Ross
______________________________________________________________________ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof.
-
Aleksandar Ivanisevic -
Alexandre Biancalana -
Jerry Amundson -
Lorenzo Quatrini -
Ross S. W. Walker