Hi,
We are starting a new project, and are trying to decide the best way to proceed. We want to setup a LAMP configuration using Centos, something we have been doing in the past with great success.
The question is load balancing. We antisipate the potential for the system to receive 500,000 requests/ day with in the next year. We want to plan for that extra load now as we start the project. What would you suggest for setups for multiple servers for redundancy and load balancing?
I have setup MySQL replication and that works fine but what about the rest of the system. I know it is quite simple to setup with windows 2003 server.
Would a cluster be the way to go? Ideally we would like 2-? severs setup that are all identical and sharing the load as need be, and if one fails users would notice nothing.
I have also thought of just looking for a hosting company that offers load balancing servers and not worry about it but we like to have control.
Thanks for any suggestions
Mace
We are starting a new project, and are trying to decide the best way to proceed. We want to setup a LAMP configuration using Centos, something we have been doing in the past with great success.
The question is load balancing. We antisipate the potential for the system to receive 500,000 requests/ day with in the next year. We want to plan for that extra load now as we start the project. What would you suggest for setups for multiple servers for redundancy and load balancing?
I have setup MySQL replication and that works fine but what about the rest of the system. I know it is quite simple to setup with windows 2003 server.
Would a cluster be the way to go? Ideally we would like 2-? severs setup that are all identical and sharing the load as need be, and if one fails users would notice nothing.
I have also thought of just looking for a hosting company that offers load balancing servers and not worry about it but we like to have control.
Thanks for any suggestions
personally is you want a commercial system i would go for Zeus as it installs onto commodity x86 hardware or if you want open source then the Linux LVS project is a good bet
thanks
Tom Brown wrote:
Would a cluster be the way to go? Ideally we would like 2-? severs setup that are all identical and sharing the load as need be, and if one fails users would notice nothing.
personally is you want a commercial system i would go for Zeus as it installs onto commodity x86 hardware or if you want open source then the Linux LVS project is a good bet
take a look at the csgfs repo's as well - that has pretty much everything you need to make this sort of a cluster.
I have been searching the internet looking for info on setting up a cluster using csgfs repo, but I am not sure which files I need. All the instruction on the RH site talk about using up2date but that doesn't work.
Is there a yum groupinstall "Cluster" ? I don't see anything on the grouplist.
Which files should be install to run a cluster. I realize it depends on my situation. Basically I want to cluster LAMP.
Karanbir Singh wrote:
Tom Brown wrote:
Would a cluster be the way to go? Ideally we would like 2-? severs setup that are all identical and sharing the load as need be, and if one fails users would notice nothing.
personally is you want a commercial system i would go for Zeus as it installs onto commodity x86 hardware or if you want open source then the Linux LVS project is a good bet
take a look at the csgfs repo's as well - that has pretty much everything you need to make this sort of a cluster.
Mace Eliason wrote:
I have been searching the internet looking for info on setting up a cluster using csgfs repo, but I am not sure which files I need. All the instruction on the RH site talk about using up2date but that doesn't work. Is there a yum groupinstall "Cluster" ? I don't see anything on the grouplist. Which files should be install to run a cluster. I realize it depends on my situation. Basically I want to cluster LAMP.
Karanbir Singh wrote:
Tom Brown wrote:
Would a cluster be the way to go? Ideally we would like 2-? severs setup that are all identical and sharing the load as need be, and if one fails users would notice nothing.
personally is you want a commercial system i would go for Zeus as it installs onto commodity x86 hardware or if you want open source then the Linux LVS project is a good bet
take a look at the csgfs repo's as well - that has pretty much everything you need to make this sort of a cluster.
the docs at http://www.centos.org/docs/4/ should all work fine, and with the right repo setup in /etc/sysconfig/rhn/sources up2date should also work, you need a line at the end that goes like this :
yum centos4-csgfs http://mirror.centos.org/centos/4/csgfs/$ARCH/
also, please dont top post - and specially not if you expect a response from someone. you'll find a large number of people here in this list ( and on most such lists ) tend to ignore top posted emails.
On Wed, 2006-05-24 at 00:26 +0100, Karanbir Singh wrote:
Mace Eliason wrote:
I have been searching the internet looking for info on setting up a cluster using csgfs repo, but I am not sure which files I need. All the instruction on the RH site talk about using up2date but that doesn't work. Is there a yum groupinstall "Cluster" ? I don't see anything on the grouplist. Which files should be install to run a cluster. I realize it depends on my situation. Basically I want to cluster LAMP.
Karanbir Singh wrote:
Tom Brown wrote:
Would a cluster be the way to go? Ideally we would like 2-? severs setup that are all identical and sharing the load as need be, and if one fails users would notice nothing.
personally is you want a commercial system i would go for Zeus as it installs onto commodity x86 hardware or if you want open source then the Linux LVS project is a good bet
take a look at the csgfs repo's as well - that has pretty much everything you need to make this sort of a cluster.
the docs at http://www.centos.org/docs/4/ should all work fine, and with the right repo setup in /etc/sysconfig/rhn/sources up2date should also work, you need a line at the end that goes like this :
yum centos4-csgfs http://mirror.centos.org/centos/4/csgfs/$ARCH/
OR ... use yum to install the required packages after adding the csgfs repo file per:
Hi,
Mace Eliason schrieb:
The question is load balancing. We antisipate the potential for the system to receive 500,000 requests/ day with in the next year. We want to plan for that extra load now as we start the project. What would you suggest for setups for multiple servers for redundancy and load balancing?
For a customer of ours we're managing a dedicated cluster with about 10.000.000 Hits/Day hitting the webservers (currently 5 running CentOS/LAMP). Loadbalancing is done by a set of reverse squid's acting as caches for static content also (round about 70% of all hits are handled by squid without bothering the webservers).
We're quite happy with this setup at the moment. By now Database-Load is handled by only one DB-Server (with manual failover).
[...] I have also thought of just looking for a hosting company that offers load balancing servers and not worry about it but we like to have control.
In my opinion this is only a matter of negotiation with your hosting company - but sure this is dependent on how much control you want to have ;-)
Mace Eliason wrote:
Hi,
We are starting a new project, and are trying to decide the best way to proceed. We want to setup a LAMP configuration using Centos, something we have been doing in the past with great success.
The question is load balancing. We antisipate the potential for the system to receive 500,000 requests/ day with in the next year. We want to plan for that extra load now as we start the project. What would you suggest for setups for multiple servers for redundancy and load balancing?
I have setup MySQL replication and that works fine but what about the rest of the system. I know it is quite simple to setup with windows 2003 server.
Would a cluster be the way to go? Ideally we would like 2-? severs setup that are all identical and sharing the load as need be, and if one fails users would notice nothing.
I have also thought of just looking for a hosting company that offers load balancing servers and not worry about it but we like to have control.
Thanks for any suggestions
Perhaps this will help get you started:
http://www.howtoforge.com/high_availability_loadbalanced_apache_cluster
Cheers,
Chris Mauritz wrote:
Mace Eliason wrote:
Hi,
We are starting a new project, and are trying to decide the best way to proceed. We want to setup a LAMP configuration using Centos, something we have been doing in the past with great success.
The question is load balancing. We antisipate the potential for the system to receive 500,000 requests/ day with in the next year. We want to plan for that extra load now as we start the project. What would you suggest for setups for multiple servers for redundancy and load balancing?
I have setup MySQL replication and that works fine but what about the rest of the system. I know it is quite simple to setup with windows 2003 server.
Would a cluster be the way to go? Ideally we would like 2-? severs setup that are all identical and sharing the load as need be, and if one fails users would notice nothing.
I have also thought of just looking for a hosting company that offers load balancing servers and not worry about it but we like to have control.
Thanks for any suggestions
Perhaps this will help get you started:
http://www.howtoforge.com/high_availability_loadbalanced_apache_cluster
Cheers,
Hi -
FWIW, I've been toying around with this as well. Right now I'm trying to decide which shared storage mechanism we'll be using for the nodes themselves. We need to keep the data consistant across 10+ machines, which will be serving this content.
If this hasn't been mentioned before, I've been using LVS for a while, with a whole lot of success. It's smart, scalable, and works quite well. If you're looking for an open-source load balancing and distribution system, I highly suggest you investigate this.
If anyone wouldn't mind chiming in with some ideas, I'd greatly appreciate it. I'm sure others would, too,
Thanks!- -dant
On Tuesday 23 May 2006 13:38, Dan Trainor wrote:
Chris Mauritz wrote:
Mace Eliason wrote:
Hi,
We are starting a new project, and are trying to decide the best way to proceed. We want to setup a LAMP configuration using Centos, something we have been doing in the past with great success.
The question is load balancing. We antisipate the potential for the system to receive 500,000 requests/ day with in the next year. We want to plan for that extra load now as we start the project. What would you suggest for setups for multiple servers for redundancy and load balancing?
I have setup MySQL replication and that works fine but what about the rest of the system. I know it is quite simple to setup with windows 2003 server.
Would a cluster be the way to go? Ideally we would like 2-? severs setup that are all identical and sharing the load as need be, and if one fails users would notice nothing.
I have also thought of just looking for a hosting company that offers load balancing servers and not worry about it but we like to have control.
Thanks for any suggestions
Perhaps this will help get you started:
http://www.howtoforge.com/high_availability_loadbalanced_apache_cluster
Cheers,
Hi -
FWIW, I've been toying around with this as well. Right now I'm trying to decide which shared storage mechanism we'll be using for the nodes themselves. We need to keep the data consistant across 10+ machines, which will be serving this content.
If this hasn't been mentioned before, I've been using LVS for a while, with a whole lot of success. It's smart, scalable, and works quite well. If you're looking for an open-source load balancing and distribution system, I highly suggest you investigate this.
If anyone wouldn't mind chiming in with some ideas, I'd greatly appreciate it. I'm sure others would, too,
Thanks!- -dant
Yes ideas is good, i'm currently evaluation cluster solutions for dynamic php/mysql website.
let the ideas flow :)
On Tue, 2006-05-23 at 10:38 -0700, Dan Trainor wrote:
Chris Mauritz wrote:
Mace Eliason wrote:
Hi,
We are starting a new project, and are trying to decide the best way to proceed. We want to setup a LAMP configuration using Centos, something we have been doing in the past with great success.
The question is load balancing. We antisipate the potential for the system to receive 500,000 requests/ day with in the next year. We want to plan for that extra load now as we start the project. What would you suggest for setups for multiple servers for redundancy and load balancing?
I have setup MySQL replication and that works fine but what about the rest of the system. I know it is quite simple to setup with windows 2003 server.
Would a cluster be the way to go? Ideally we would like 2-? severs setup that are all identical and sharing the load as need be, and if one fails users would notice nothing.
I have also thought of just looking for a hosting company that offers load balancing servers and not worry about it but we like to have control.
Thanks for any suggestions
Perhaps this will help get you started:
http://www.howtoforge.com/high_availability_loadbalanced_apache_cluster
Cheers,
Hi -
FWIW, I've been toying around with this as well. Right now I'm trying to decide which shared storage mechanism we'll be using for the nodes themselves. We need to keep the data consistant across 10+ machines, which will be serving this content.
If this hasn't been mentioned before, I've been using LVS for a while, with a whole lot of success. It's smart, scalable, and works quite well. If you're looking for an open-source load balancing and distribution system, I highly suggest you investigate this.
If anyone wouldn't mind chiming in with some ideas, I'd greatly appreciate it. I'm sure others would, too,
Thanks!- -dant _
For the backend storage, it depends what's your budget ... :o) A minimal setup is to use nfs on a central server to host/share the same data across all your machines ... the problem in this config is that the nfs server becomes the single point of failure ... so why not using a simple heartbeat solution for 2 nfs servers acting as one and uses drdb between these 2 nodes for the replication ... Other method is to have a dedicate san with hba in each webservers but that's another budget ... :o)
Just my two cents ...
Fabian Arrotin wrote:
On Tue, 2006-05-23 at 10:38 -0700, Dan Trainor wrote:
Chris Mauritz wrote:
Mace Eliason wrote:
Hi,
We are starting a new project, and are trying to decide the best way to proceed. We want to setup a LAMP configuration using Centos, something we have been doing in the past with great success.
The question is load balancing. We antisipate the potential for the system to receive 500,000 requests/ day with in the next year. We want to plan for that extra load now as we start the project. What would you suggest for setups for multiple servers for redundancy and load balancing?
I have setup MySQL replication and that works fine but what about the rest of the system. I know it is quite simple to setup with windows 2003 server.
Would a cluster be the way to go? Ideally we would like 2-? severs setup that are all identical and sharing the load as need be, and if one fails users would notice nothing.
I have also thought of just looking for a hosting company that offers load balancing servers and not worry about it but we like to have control.
Thanks for any suggestions
Perhaps this will help get you started:
http://www.howtoforge.com/high_availability_loadbalanced_apache_cluster
Cheers,
Hi -
FWIW, I've been toying around with this as well. Right now I'm trying to decide which shared storage mechanism we'll be using for the nodes themselves. We need to keep the data consistant across 10+ machines, which will be serving this content.
If this hasn't been mentioned before, I've been using LVS for a while, with a whole lot of success. It's smart, scalable, and works quite well. If you're looking for an open-source load balancing and distribution system, I highly suggest you investigate this.
If anyone wouldn't mind chiming in with some ideas, I'd greatly appreciate it. I'm sure others would, too,
Thanks!- -dant _
For the backend storage, it depends what's your budget ... :o) A minimal setup is to use nfs on a central server to host/share the same data across all your machines ... the problem in this config is that the nfs server becomes the single point of failure ... so why not using a simple heartbeat solution for 2 nfs servers acting as one and uses drdb between these 2 nodes for the replication ... Other method is to have a dedicate san with hba in each webservers but that's another budget ... :o)
Just my two cents ...
HI, Fabian -
I've been toying aroudn with both NFS and GFS, but NFS does leave me with a single point of failure. I'd rather not use something like drdb, however. I'm still researching GFS to see if it's a viable alternative for what I'm looking for.
Thanks! -dant
On Tue, 2006-05-23 at 12:49 -0700, Dan Trainor wrote:
For the backend storage, it depends what's your budget ... :o) A minimal setup is to use nfs on a central server to host/share the same data across all your machines ... the problem in this config is that the nfs server becomes the single point of failure ... so why not using a simple heartbeat solution for 2 nfs servers acting as one and uses drdb between these 2 nodes for the replication ... Other method is to have a dedicate san with hba in each webservers but that's another budget ... :o)
Just my two cents ...
HI, Fabian -
I've been toying aroudn with both NFS and GFS, but NFS does leave me with a single point of failure. I'd rather not use something like drdb, however. I'm still researching GFS to see if it's a viable alternative for what I'm looking for.
Thanks! -dant
GFS can do the job, but in this case you should have a real shared storage to permit all the servers to access the shared data in the same time ... If you don't want to invest a lot, you can still use iscsi but the single point of failure still exists ...
On Tue, 23 May 2006 21:46:48 +0200 Fabian Arrotin fabian.arrotin@arrfab.net wrote:
For the backend storage, it depends what's your budget ... :o) A minimal setup is to use nfs on a central server to host/share the same data across all your machines ...
Even more minimal setup is to rsync www data to all the web servers. In most kind of LAMP database driven setups all the active data is in the db anyway and www data (php or whatever) should only change on updates & bugfixes. So there's absolutely no need for some expensive and/or complicated setup.
Mace Eliason wrote:
Hi,
We are starting a new project, and are trying to decide the best way to proceed. We want to setup a LAMP configuration using Centos, something we have been doing in the past with great success.
The question is load balancing. We antisipate the potential for the system to receive 500,000 requests/ day with in the next year. We want to plan for that extra load now as we start the project. What would you suggest for setups for multiple servers for redundancy and load balancing?
I have setup MySQL replication and that works fine but what about the rest of the system. I know it is quite simple to setup with windows 2003 server.
Would a cluster be the way to go? Ideally we would like 2-? severs setup that are all identical and sharing the load as need be, and if one fails users would notice nothing.
I have also thought of just looking for a hosting company that offers load balancing servers and not worry about it but we like to have control.
Thanks for any suggestions
And what seems to be an inexpensive commercially supported solution:
http://www.interworx.com/products/iworx-cp/
I also remember there being a few other cluster solutions that used CentOS which is how I initially stumbled on its existence.
Cheers,
On Tue, 23 May 2006 08:52:06 -0700 Mace Eliason meliason@shaw.ca wrote:
Hi,
We are starting a new project, and are trying to decide the best way to proceed. We want to setup a LAMP configuration using Centos, something we have been doing in the past with great success.
For LAMP setups I suggest mod_backhand - it evenly redistributes your requests among as many web servers you provide it. It works really great and I use it for a large webmail (~million hits daily).
http://www.backhand.org/mod_backhand/
Jure Pečar wrote:
For LAMP setups I suggest mod_backhand - it evenly redistributes your requests among as many web servers you provide it. It works really great and I use it for a large webmail (~million hits daily).
Wow, that's a pretty cool hack.
Cheers,
On 5/23/06, Chris Mauritz chrism@imntv.com wrote:
Jure Pečar wrote:
For LAMP setups I suggest mod_backhand - it evenly redistributes your requests among as many web servers you provide it. It works really great and I use it for a large webmail (~million hits daily).
Wow, that's a pretty cool hack.
Indeed, but it's worth noting that it doesn't support apache 2.x, so that limits it to centos 2.x I believe. http://www.backhand.org/mod_backhand/FAQ.shtml
Jim Perrin wrote:
On 5/23/06, Chris Mauritz chrism@imntv.com wrote:
Jure Pečar wrote:
For LAMP setups I suggest mod_backhand - it evenly redistributes
your requests among as many web servers you provide it. It works really great and I use it for a large webmail (~million hits daily).
Wow, that's a pretty cool hack.
Indeed, but it's worth noting that it doesn't support apache 2.x, so that limits it to centos 2.x I believe. http://www.backhand.org/mod_backhand/FAQ.shtml
Eeek, I didn't notice that. There also doesn't appear to be any active development on it anymore. Pity....
Cheers,
On Tue, 23 May 2006 16:18:29 -0400 "Jim Perrin" jperrin@gmail.com wrote:
On 5/23/06, Chris Mauritz chrism@imntv.com wrote:
Jure Pečar wrote:
Wow, that's a pretty cool hack.
Indeed, but it's worth noting that it doesn't support apache 2.x, so that limits it to centos 2.x I believe.
What, is it so difficult to compile apache these days? :) I run it on centos 3 & 4. I rolled my own rpms with some other patches (lingerd comes to mind) and site-specific tweaks and installed them instead of httpd-2.0. No problems at all.
Jure Pečar wrote:
On Tue, 23 May 2006 16:18:29 -0400 "Jim Perrin" jperrin@gmail.com wrote:
On 5/23/06, Chris Mauritz chrism@imntv.com wrote:
Jure Pečar wrote:
Wow, that's a pretty cool hack.
Indeed, but it's worth noting that it doesn't support apache 2.x, so that limits it to centos 2.x I believe.
What, is it so difficult to compile apache these days? :) I run it on centos 3 & 4. I rolled my own rpms with some other patches (lingerd comes to mind) and site-specific tweaks and installed them instead of httpd-2.0. No problems at all.
So are you saying that it does indeed work with 2.0 if you compile it yourself or that you're using it with Apache 1.x?
Cheers,
On Wed, 24 May 2006 07:08:07 -0400 Chris Mauritz chrism@imntv.com wrote:
So are you saying that it does indeed work with 2.0 if you compile it yourself or that you're using it with Apache 1.x?
I'm using it with apache 1.3.
Jure Pečar wrote:
On Wed, 24 May 2006 07:08:07 -0400 Chris Mauritz chrism@imntv.com wrote:
So are you saying that it does indeed work with 2.0 if you compile it yourself or that you're using it with Apache 1.x?
I'm using it with apache 1.3.
Ah....that makes it a non-starter for me. Like Jim, I'm rather fond of Apache 2.0 too.
Cheers,
On Wed, 24 May 2006 09:07:20 -0400 Chris Mauritz chrism@imntv.com wrote:
Ah....that makes it a non-starter for me. Like Jim, I'm rather fond of Apache 2.0 too.
Another httpd worth checking is lighttpd. It is faster than any apache and has native load balancing mechanism built in. I haven't yet played much with it though.
What, is it so difficult to compile apache these days? :) I run it on centos 3 & 4. I rolled my own rpms with some other patches (lingerd comes to mind) and site-specific tweaks and installed them instead of httpd-2.0. No problems at all.
Not at all, but I'm quite fond of apache 2.x, and I'd rather move on to other admin tasks which I enjoy more than watching the gentoo screensaver (gcc scroll for those of you not up on the gentoo bashing). But if someone is willing to put the work into maintaining it themselves, I have no problem with them doing so.
Hi there,
I'm currently involved in a Project that requires high-avaiability and load-balancing for web servers and database servers. In order to have a load-balancing webserver, I use LVS and I use dns round robin. Basically I have the hostname with 2 distinct IP, so I have some load-balancing right from the start, using DNS. Then I have 2 machines (low-end) , that run LVS, and this machines are high-avaiable , using heart-beat. As to storage , I have the machines using an iscsi share , shared by a machine running with a raid 1+0. Regarding the database servers, i use Oracle so perhaps this is off-topic for you :)
Best wishes, Bruno Sousa
-----Mensagem original----- De: centos-bounces@centos.org [mailto:centos-bounces@centos.org] Em nome de Mace Eliason Enviada: terça-feira, 23 de Maio de 2006 16:52 Para: CentOS mailing list Assunto: [CentOS] Load Balancing
Hi,
We are starting a new project, and are trying to decide the best way to proceed. We want to setup a LAMP configuration using Centos, something we have been doing in the past with great success.
The question is load balancing. We antisipate the potential for the system to receive 500,000 requests/ day with in the next year. We want to plan for that extra load now as we start the project. What would you suggest for setups for multiple servers for redundancy and load balancing?
I have setup MySQL replication and that works fine but what about the rest of the system. I know it is quite simple to setup with windows 2003 server.
Would a cluster be the way to go? Ideally we would like 2-? severs setup that are all identical and sharing the load as need be, and if one fails users would notice nothing.
I have also thought of just looking for a hosting company that offers load balancing servers and not worry about it but we like to have control.
Thanks for any suggestions
Mace _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
-
Bruno Sousa -
Charles Lacroix -
Chris Mauritz -
Dan Trainor -
Fabian Arrotin -
Jim Perrin -
Johnny Hughes -
Jure Pečar -
Karanbir Singh -
Mace Eliason -
Michael Metz [SpeedPartner] -
Tom Brown