Hi all,
I want to start experimenting with clusters, and I would like to use normal desktop grade hardware for this. I have some extra PC components lying around, enough to build 3 - 4 moderate desktops with a PIV / C2D CPU & 512MB - 1GB RAM each. All the machines should have at least a 100MB NIC, but I can add a gigabit NIC to the machines that doesn't have it if need be.
I have used Linux Heartbeat to failover a MySQL cluster before, and it's actually been running very well for about 3 years now. But, I want to start looking @ total clustering, with DRBD - where everything (not just MySQL) is being replicated across the 4 machines. I'm also intereated in load balancing.
Has anyone done anything like this?
The cluster will be running / hosting a few XEN Virtual Machines, and idealy if one PC / server dies (hardware failure / power failure / etc), then on of the other machines should still be running the virtual machines, i.e. no downtime. Is this possible? Can somone steer me into a direction where I can get more info on this?
My aim: to use common, cheap PC equipment & CentOS for the project, I don't want to spend any more money, and want to use what I have.
Rudi Ahlers wrote:
Hi all,
I want to start experimenting with clusters, and I would like to use normal desktop grade hardware for this. I have some extra PC components lying around, enough to build 3 - 4 moderate desktops with a PIV / C2D CPU & 512MB - 1GB RAM each. All the machines should have at least a 100MB NIC, but I can add a gigabit NIC to the machines that doesn't have it if need be.
I have used Linux Heartbeat to failover a MySQL cluster before, and it's actually been running very well for about 3 years now. But, I want to start looking @ total clustering, with DRBD - where everything (not just MySQL) is being replicated across the 4 machines. I'm also intereated in load balancing.
Has anyone done anything like this?
The cluster will be running / hosting a few XEN Virtual Machines, and idealy if one PC / server dies (hardware failure / power failure / etc), then on of the other machines should still be running the virtual machines, i.e. no downtime. Is this possible? Can somone steer me into a direction where I can get more info on this?
My aim: to use common, cheap PC equipment & CentOS for the project, I don't want to spend any more money, and want to use what I have.
So if i understand you correctly, the goal is to have a Xen farm running on multiple hosts ? You can use the included clustering solution (conga) to do that but normally it needs a shared storage to store the domU's .. What i've already tested is to use drbd with GFS2 so that all the local storage act like only one for all the nodes but that specific drbd part is not covered by conga. But i don't get the point about load-balancing ? you were talking about xen domU's and previously about mysql ... i suppose mysql will be running inside of a domU ? BTW the RHCS (conga) doesn't provide by itself a load-balancing solution You can read the following article that was really interesting when i wanted to test that (except that it doesn't deal with drbd, but you can find informations on the wiki for that) : http://www.redhatmagazine.com/2007/08/23/automated-failover-and-recovery-of-...
On Wed, Dec 3, 2008 at 11:52 AM, Fabian Arrotin fabian.arrotin@arrfab.net wrote:
Rudi Ahlers wrote:
Hi all,
I want to start experimenting with clusters, and I would like to use normal desktop grade hardware for this. I have some extra PC components lying around, enough to build 3 - 4 moderate desktops with a PIV / C2D CPU & 512MB - 1GB RAM each. All the machines should have at least a 100MB NIC, but I can add a gigabit NIC to the machines that doesn't have it if need be.
I have used Linux Heartbeat to failover a MySQL cluster before, and it's actually been running very well for about 3 years now. But, I want to start looking @ total clustering, with DRBD - where everything (not just MySQL) is being replicated across the 4 machines. I'm also intereated in load balancing.
Has anyone done anything like this?
The cluster will be running / hosting a few XEN Virtual Machines, and idealy if one PC / server dies (hardware failure / power failure / etc), then on of the other machines should still be running the virtual machines, i.e. no downtime. Is this possible? Can somone steer me into a direction where I can get more info on this?
My aim: to use common, cheap PC equipment & CentOS for the project, I don't want to spend any more money, and want to use what I have.
So if i understand you correctly, the goal is to have a Xen farm running on multiple hosts ? You can use the included clustering solution (conga) to do that but normally it needs a shared storage to store the domU's .. What i've already tested is to use drbd with GFS2 so that all the local storage act like only one for all the nodes but that specific drbd part is not covered by conga. But i don't get the point about load-balancing ? you were talking about xen domU's and previously about mysql ... i suppose mysql will be running inside of a domU ? BTW the RHCS (conga) doesn't provide by itself a load-balancing solution You can read the following article that was really interesting when i wanted to test that (except that it doesn't deal with drbd, but you can find informations on the wiki for that) : http://www.redhatmagazine.com/2007/08/23/automated-failover-and-recovery-of-...
--
Fabian Arrotin fabian.arrotin@arrfab.net "Internet network currently down, TCP/IP packets delivered now by UPS/Fedex ..."
Yes, the idea is to run a XEN farm. Each VPS will have it own set of sofware running, from Windows to Linux, with SQL / Web / Email / etc.
With regards to load balancing, I need it so that the total load of all the XEN VPS's can be spread out among the XEN servers, almost like a super computer. How will I do this?
Yes, the idea is to run a XEN farm. Each VPS will have it own set of sofware running, from Windows to Linux, with SQL / Web / Email / etc.
With regards to load balancing, I need it so that the total load of all the XEN VPS's can be spread out among the XEN servers, almost like a super computer. How will I do this?
depends on how much of your own effort you want to put in getting this running - it might be that Xen Enterprise is something that would benefit you more. It can be done with other tools but if you want VM's to magically start working over >> there when the host here << dies then i think you need to look at a commercial product. I dont like Xen Enterprise and never will but if you want to use Xen then you may have to go that route. You could make it so that disk images are on shred storage and so recovery to a different dom0 is short but if you want seamless then i think you'll have to pay $$
On Wed, Dec 3, 2008 at 2:22 PM, Tom Brown tom@ng23.net wrote:
Yes, the idea is to run a XEN farm. Each VPS will have it own set of sofware running, from Windows to Linux, with SQL / Web / Email / etc.
With regards to load balancing, I need it so that the total load of all the XEN VPS's can be spread out among the XEN servers, almost like a super computer. How will I do this?
depends on how much of your own effort you want to put in getting this running - it might be that Xen Enterprise is something that would benefit you more. It can be done with other tools but if you want VM's to magically start working over >> there when the host here << dies then i think you need to look at a commercial product. I dont like Xen Enterprise and never will but if you want to use Xen then you may have to go that route. You could make it so that disk images are on shred storage and so recovery to a different dom0 is short but if you want seamless then i think you'll have to pay $$
Hi Tom,
Thank you for the input. Let's forget about XEN for a moment, I was actually looking at setting up a cluster which has fail-over & load balancing capabilities, regardless of what runs on it. If XEN enterprise is the only option,then I'm not going to bother. I don't see why I need to pay for a tool which has a helpdesk and "professional technicians standing by" to help me when I get stuck, if XEN can do the same.
On Wed, Dec 3, 2008 at 3:35 PM, Rudi Ahlers rudiahlers@gmail.com wrote:
On Wed, Dec 3, 2008 at 2:22 PM, Tom Brown tom@ng23.net wrote:
Yes, the idea is to run a XEN farm. Each VPS will have it own set of sofware running, from Windows to Linux, with SQL / Web / Email / etc.
With regards to load balancing, I need it so that the total load of all the XEN VPS's can be spread out among the XEN servers, almost like a super computer. How will I do this?
depends on how much of your own effort you want to put in getting this running - it might be that Xen Enterprise is something that would benefit you more. It can be done with other tools but if you want VM's to magically start working over >> there when the host here << dies then i think you need to look at a commercial product. I dont like Xen Enterprise and never will but if you want to use Xen then you may have to go that route. You could make it so that disk images are on shred storage and so recovery to a different dom0 is short but if you want seamless then i think you'll have to pay $$
Hi Tom,
Thank you for the input. Let's forget about XEN for a moment, I was actually looking at setting up a cluster which has fail-over & load balancing capabilities, regardless of what runs on it. If XEN enterprise is the only option,then I'm not going to bother. I don't see why I need to pay for a tool which has a helpdesk and "professional technicians standing by" to help me when I get stuck, if XEN can do the same.
--
Kind Regards Rudi Ahlers
I've actually been looking at VMWare's grid which costs about $5000 - which is much easier to go for, than with XEN enterprise, but I was hoping to accomplish this task without spending extra money
Thank you for the input. Let's forget about XEN for a moment, I was actually looking at setting up a cluster which has fail-over & load balancing capabilities, regardless of what runs on it. If XEN enterprise is the only option,then I'm not going to bother. I don't see why I need to pay for a tool which has a helpdesk and "professional technicians standing by" to help me when I get stuck, if XEN can do the same.
i cant speak for others but when i talk of clusters and load balancing i talk of different things. For load balancing i'd lean towards LVS and for clusters then it very much depends on what you are clustering. Application servers, databases, mail servers etc etc. For a MySQL 'cluster' i'd probably go for master<>master depending on how many nodes i need and the application type etc. If its application clusters then things like tomcat can know about each other and take over if one of them dies. I think that the point i'm trying to make is that the solution very much depends on what you are trying to achieve, so to me 'regardless what runs on it' is not really something to aim a good answer at.
As mentioned i am pretty sure that if you want to make your own 'cloud' in todays speak then you may well be looking commercial.
Thats just my thoughts and its most probable i am wrong.
On Wed, Dec 3, 2008 at 4:29 PM, Tom Brown tom@ng23.net wrote:
Thank you for the input. Let's forget about XEN for a moment, I was actually looking at setting up a cluster which has fail-over & load balancing capabilities, regardless of what runs on it. If XEN enterprise is the only option,then I'm not going to bother. I don't see why I need to pay for a tool which has a helpdesk and "professional technicians standing by" to help me when I get stuck, if XEN can do the same.
i cant speak for others but when i talk of clusters and load balancing i talk of different things. For load balancing i'd lean towards LVS and for clusters then it very much depends on what you are clustering. Application servers, databases, mail servers etc etc. For a MySQL 'cluster' i'd probably go for master<>master depending on how many nodes i need and the application type etc. If its application clusters then things like tomcat can know about each other and take over if one of them dies. I think that the point i'm trying to make is that the solution very much depends on what you are trying to achieve, so to me 'regardless what runs on it' is not really something to aim a good answer at.
As mentioned i am pretty sure that if you want to make your own 'cloud' in todays speak then you may well be looking commercial.
Thats just my thoughts and its most probable i am wrong. _______________________________________________
Hi Tom,
I do use MySQL clusters, but this is an application level cluster, and is limited. I would like to go further and do an OS level cluster. With DRBD, one could mirror 2 servers identical, i.e. everything on 1 server to the other, which is even better than MySQL clustering. But, DRBD only offers high-availability, i.e. if one server goes down, the other can take over.
What I'm looking for, is how to build what is called a super computer. Google used to, or still even does this, where they put hundreds of computers into the same "cluster" / super computer, and end up with a 1 huge hard drive, and plenty of RAM to use :) So, my question is, how does one do this? I know that I can pay someone a LOT of money for it, but I don't have a lot of money for this. If it's not possible, I'll probably just go and purchase VMWare's grid application and use that, but I would prefer to try this myself if possible.
Rudi Ahlers schrieb:
On Wed, Dec 3, 2008 at 4:29 PM, Tom Brown tom@ng23.net wrote:
Thank you for the input. Let's forget about XEN for a moment, I was actually looking at setting up a cluster which has fail-over & load balancing capabilities, regardless of what runs on it. If XEN enterprise is the only option,then I'm not going to bother. I don't see why I need to pay for a tool which has a helpdesk and "professional technicians standing by" to help me when I get stuck, if XEN can do the same.
i cant speak for others but when i talk of clusters and load balancing i talk of different things. For load balancing i'd lean towards LVS and for clusters then it very much depends on what you are clustering. Application servers, databases, mail servers etc etc. For a MySQL 'cluster' i'd probably go for master<>master depending on how many nodes i need and the application type etc. If its application clusters then things like tomcat can know about each other and take over if one of them dies. I think that the point i'm trying to make is that the solution very much depends on what you are trying to achieve, so to me 'regardless what runs on it' is not really something to aim a good answer at.
As mentioned i am pretty sure that if you want to make your own 'cloud' in todays speak then you may well be looking commercial.
Thats just my thoughts and its most probable i am wrong. _______________________________________________
Hi Tom,
I do use MySQL clusters, but this is an application level cluster, and is limited. I would like to go further and do an OS level cluster. With DRBD, one could mirror 2 servers identical, i.e. everything on 1 server to the other, which is even better than MySQL clustering. But, DRBD only offers high-availability, i.e. if one server goes down, the other can take over.
What I'm looking for, is how to build what is called a super computer. Google used to, or still even does this, where they put hundreds of computers into the same "cluster" / super computer, and end up with a 1 huge hard drive, and plenty of RAM to use :) So, my question is, how does one do this? I know that I can pay someone a LOT of money for it, but I don't have a lot of money for this. If it's not possible, I'll probably just go and purchase VMWare's grid application and use that, but I would prefer to try this myself if possible.
There's nothing free and COTS, AFAIK. Only building-blocks: hadoop and it's related tools.
You must do work on your own and parallelize the way your problem is solved.
Rainer
Rudi Ahlers wrote:
What I'm looking for, is how to build what is called a super computer. Google used to, or still even does this, where they put hundreds of computers into the same "cluster" / super computer, and end up with a 1 huge hard drive, and plenty of RAM to use :)
Doing this is really only useful for very specialized applications, do not expect this to work normally for most off the shelf/off the repository applications.
If you want a global file system look into AFS, it can do what you want I believe. Though I would not use AFS for anything like MySQL or Xen. AFS is extremely complicated but does provide a real generic distributed file system.
Don't even bother going down the route where you cluster the memory together, latency will kill performance, and again you need very specialized apps to use it.
And in any sort of disk/ram clustering performance will generally be shit until you have many nodes.
Basically what I'm saying here is, based on your past posts you don't appear to be an extremely experienced admin/developer so what you are wanting to do will not work the way you hope it might. So don't waste any more time and drop the idea.
nate
Hi,
I have a tutorial on HPC 101.. check it out at
http://www.hpccommunity.org/f55/kusu-101-what-beowulf-cluster-computing-tabl...
It should give you an understanding of HPC.
As for 1 huge disk and plenty of RAM
- huge disk - you will need a cluster filesystem.. you may wish to check out PVFS2
- for RAM - shared memory across multiple nodes using commodity hardware and software (distributed shared memory DSM type software) will work.. but latency will be the biggest problem you will face... good for academic exercise, but in the real world, such shared memory systems are built with proprietary high speed interconnects such as Quadrics or Infiniband...
Hope this helps.
-- Laurence Kusu - HPC Management Simplified http://www.hpccommunity.org/kusu
On 3 Dec 2008, at 11:08 PM, Rudi Ahlers wrote:
On Wed, Dec 3, 2008 at 4:29 PM, Tom Brown tom@ng23.net wrote:
Thank you for the input. Let's forget about XEN for a moment, I was actually looking at setting up a cluster which has fail-over & load balancing capabilities, regardless of what runs on it. If XEN enterprise is the only option,then I'm not going to bother. I don't see why I need to pay for a tool which has a helpdesk and "professional technicians standing by" to help me when I get stuck, if XEN can do the same.
i cant speak for others but when i talk of clusters and load balancing i talk of different things. For load balancing i'd lean towards LVS and for clusters then it very much depends on what you are clustering. Application servers, databases, mail servers etc etc. For a MySQL 'cluster' i'd probably go for master<>master depending on how many nodes i need and the application type etc. If its application clusters then things like tomcat can know about each other and take over if one of them dies. I think that the point i'm trying to make is that the solution very much depends on what you are trying to achieve, so to me 'regardless what runs on it' is not really something to aim a good answer at.
As mentioned i am pretty sure that if you want to make your own 'cloud' in todays speak then you may well be looking commercial.
Thats just my thoughts and its most probable i am wrong. _______________________________________________
Hi Tom,
I do use MySQL clusters, but this is an application level cluster, and is limited. I would like to go further and do an OS level cluster. With DRBD, one could mirror 2 servers identical, i.e. everything on 1 server to the other, which is even better than MySQL clustering. But, DRBD only offers high-availability, i.e. if one server goes down, the other can take over.
What I'm looking for, is how to build what is called a super computer. Google used to, or still even does this, where they put hundreds of computers into the same "cluster" / super computer, and end up with a 1 huge hard drive, and plenty of RAM to use :) So, my question is, how does one do this? I know that I can pay someone a LOT of money for it, but I don't have a lot of money for this. If it's not possible, I'll probably just go and purchase VMWare's grid application and use that, but I would prefer to try this myself if possible.
--
Kind Regards Rudi Ahlers _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
-
Fabian Arrotin -
Laurence Liew -
nate -
Rainer Duffner -
Rudi Ahlers -
Tom Brown