Hello,
I am currently rsyncing the CentOS tree to one of our server. We are managing a few dozends of CentOS servers, so it surely will speed up updates and lead to less wasted bandwidth if we don't update from common public repositories any longer, but use an internal server instead, that only syncs once with the master server.
Now I'm in doubt wether I should make our own mirror public or not.
The first and main issue are the bandwidth costs. Can you give an estimate on how much traffic a typical European/German mirror generates per month?
The second issue is, that some kiddies might try to attack and hack our mirror to inject changed packages. Do you have statistics on this? Which FTP daemon do you recommend for a hardened anonymous-FTP only service?
Btw.: What do I have to change in the yum config on each server to use one specific repository server and not the mirrorlist system? Do I just have comment the mirrorlist line and uncomment the baseurl?
Regards Marten
On Fri, 25 Sep 2009, Marten Lehmann wrote:
Hello,
I am currently rsyncing the CentOS tree to one of our server. We are managing a few dozends of CentOS servers, so it surely will speed up updates and lead to less wasted bandwidth if we don't update from common public repositories any longer, but use an internal server instead, that only syncs once with the master server.
Now I'm in doubt wether I should make our own mirror public or not.
The first and main issue are the bandwidth costs. Can you give an estimate on how much traffic a typical European/German mirror generates per month?
The second issue is, that some kiddies might try to attack and hack our mirror to inject changed packages. Do you have statistics on this? Which FTP daemon do you recommend for a hardened anonymous-FTP only service?
Btw.: What do I have to change in the yum config on each server to use one specific repository server and not the mirrorlist system? Do I just have comment the mirrorlist line and uncomment the baseurl?
Since I'm not a German or European mirror, I don't have an answer to your bandwidth question.
As to the issue of being attacked, remember that the packages are signed, so if someone were to compromise your mirror, the changed package would not be signed and would give an error. (If your mirror is compromised, your other clients are still safe.)
vsftpd has a good reputation, and is the package provided by Red Hat and CentOS (disclaimer: I don't provide FTP service, just http and rsync).
To make your machines go straight to your mirror, yes, 1) comment out the mirrorlist, 2) uncomment the baseurl, and 3) change the baseurl to point to your mirror.
DR
Am 25.09.09 20:08, schrieb Marten Lehmann:
Now I'm in doubt wether I should make our own mirror public or not.
The first and main issue are the bandwidth costs. Can you give an estimate on how much traffic a typical European/German mirror generates per month?
http://centos.bio.lmu.de/mrtg/141.84.43.23_141.84.43.22.html
This is a german mirror, offering rsync, http and ftp, all isos including DVD. The lack of traffic you can see in July was a config problem :)
The second issue is, that some kiddies might try to attack and hack our mirror to inject changed packages. Do you have statistics on this? Which FTP daemon do you recommend for a hardened anonymous-FTP only service?
I use vsftpd with anonymous access only - and anonymous can only read.
Btw.: What do I have to change in the yum config on each server to use one specific repository server and not the mirrorlist system? Do I just have comment the mirrorlist line and uncomment the baseurl?
Yes, as the yum.conf manual page explains in length :)
Ralph
Hello,
http://centos.bio.lmu.de/mrtg/141.84.43.23_141.84.43.22.html
This is a german mirror, offering rsync, http and ftp, all isos including DVD. The lack of traffic you can see in July was a config problem :)
so usually you have an average of about 2 MB/s which means about 16MBit bandwidth?
Since HTTP seems to be the more widely used protocol for yum, which HTTP daemon do you use? I guess not Apache httpd with all its unnecassary features for a read only httpd?
Regards Marten
Am 25.09.09 21:06, schrieb Marten Lehmann:
so usually you have an average of about 2 MB/s which means about 16MBit bandwidth?
Looks like it. It peaks with large updates (OpenOffice or the 5.3 spike in April).
Since HTTP seems to be the more widely used protocol for yum, which HTTP daemon do you use? I guess not Apache httpd with all its unnecassary features for a read only httpd?
Why not? Apache is a great performer and you can get it to have a relatively small memory footprint if you throw out modules you don't need. I don't see a reason to install something from a different repository.
I think all the people talking about apache being bloated and non-performing never even tried. If it's good enough for HEANET, then it's good enough for me.
Ralph
Well said re:Apache - I've been involved in some websites running as high as 20-25 million page views PER DAY using Apache .. it performs for my needs just as well as other solutions. It's only as bloated as people make it by adding dozens of modules they don't require. Start with a stock install and add only what needed - works great for me...
Paul
-----Original Message----- From: centos-mirror-bounces@centos.org [mailto:centos-mirror-bounces@centos.org] On Behalf Of Ralph Angenendt Sent: September 25, 2009 6:01 PM To: centos-mirror@centos.org Subject: Re: [CentOS-mirror] Average mirror traffic
Am 25.09.09 21:06, schrieb Marten Lehmann:
so usually you have an average of about 2 MB/s which means about
16MBit
bandwidth?
Looks like it. It peaks with large updates (OpenOffice or the 5.3 spike in April).
Since HTTP seems to be the more widely used protocol for yum, which
HTTP
daemon do you use? I guess not Apache httpd with all its unnecassary features for a read only httpd?
Why not? Apache is a great performer and you can get it to have a relatively small memory footprint if you throw out modules you don't need. I don't see a reason to install something from a different repository.
I think all the people talking about apache being bloated and non-performing never even tried. If it's good enough for HEANET, then it's good enough for me.
Ralph _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
----------------------------------------------------------------------------
"The information transmitted is intended only for the person or entity to which it is addressed and contains confidential and/or privileged material. If you received this in error, please contact the sender immediately and then destroy this transmission, including all attachments, without copying, distributing or disclosing same. Thank you."
On Fri, Sep 25, 2009 at 08:28:13PM +0200, Ralph Angenendt wrote:
Am 25.09.09 20:08, schrieb Marten Lehmann:
Now I'm in doubt wether I should make our own mirror public or not.
The first and main issue are the bandwidth costs. Can you give an estimate on how much traffic a typical European/German mirror generates per month?
http://centos.bio.lmu.de/mrtg/141.84.43.23_141.84.43.22.html
This is a german mirror, offering rsync, http and ftp, all isos including DVD. The lack of traffic you can see in July was a config problem :)
Here is the data from another complete CentOS mirror in Germany.
http://ftp-stud.hs-esslingen.de/info/breakdown.php4?details=centos
Adrian
We are doing about 35Mb/s of traffic on our CentOS box here... account for about 5Mb/s of other traffic. Yes, we run Apache and the box experiences approximately 0.05 load on average.
Paul
-----Original Message----- From: centos-mirror-bounces@centos.org [mailto:centos-mirror-bounces@centos.org] On Behalf Of Ralph Angenendt Sent: September 25, 2009 2:28 PM To: centos-mirror@centos.org Subject: Re: [CentOS-mirror] Average mirror traffic
Am 25.09.09 20:08, schrieb Marten Lehmann:
Now I'm in doubt wether I should make our own mirror public or not.
The first and main issue are the bandwidth costs. Can you give an estimate on how much traffic a typical European/German mirror
generates
per month?
http://centos.bio.lmu.de/mrtg/141.84.43.23_141.84.43.22.html
This is a german mirror, offering rsync, http and ftp, all isos including DVD. The lack of traffic you can see in July was a config problem :)
The second issue is, that some kiddies might try to attack and hack
our
mirror to inject changed packages. Do you have statistics on this?
Which
FTP daemon do you recommend for a hardened anonymous-FTP only service?
I use vsftpd with anonymous access only - and anonymous can only read.
Btw.: What do I have to change in the yum config on each server to use
one specific repository server and not the mirrorlist system? Do I
just
have comment the mirrorlist line and uncomment the baseurl?
Yes, as the yum.conf manual page explains in length :)
Ralph _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
----------------------------------------------------------------------------
"The information transmitted is intended only for the person or entity to which it is addressed and contains confidential and/or privileged material. If you received this in error, please contact the sender immediately and then destroy this transmission, including all attachments, without copying, distributing or disclosing same. Thank you."
Hello,
We are doing about 35Mb/s of traffic on our CentOS box here...
that's definetely too much for us. We are really interested to be a CentOS mirror, but we cannot efford more than 10MBit average bandwidth for CentOS per month.
Is it possible to define in which ratio a mirror shall be listed for updates and downloads?
And is it possible to delist a mirror until the end of a month if the traffic has been too much for this certain month (e.g. due to a openoffice update)?
Regards Marten
Our traffic might be much higher than a typical mirror due our peering in many cities ... you might want feedback from other mirrors besides us ;)
-----Original Message----- From: centos-mirror-bounces@centos.org [mailto:centos-mirror-bounces@centos.org] On Behalf Of Marten Lehmann Sent: September-28-09 10:39 AM To: Mailing list for CentOS mirrors. Subject: Re: [CentOS-mirror] Average mirror traffic
Hello,
We are doing about 35Mb/s of traffic on our CentOS box here...
that's definetely too much for us. We are really interested to be a CentOS mirror, but we cannot efford more than 10MBit average bandwidth for CentOS per month.
Is it possible to define in which ratio a mirror shall be listed for updates and downloads?
And is it possible to delist a mirror until the end of a month if the traffic has been too much for this certain month (e.g. due to a openoffice update)?
Regards Marten _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
----------------------------------------------------------------------------
"The information transmitted is intended only for the person or entity to which it is addressed and contains confidential and/or privileged material. If you received this in error, please contact the sender immediately and then destroy this transmission, including all attachments, without copying, distributing or disclosing same. Thank you."
There are supposed to be many ways of restricting your throughput; you can just configure your mirror to cap your output at 10Mbps, for example. This at least is doable with iptables, if not at a "higher" level. I know some ftp daemons provide easy support for that, I don't know about web servers...
Personally, people can download all they want from my mirror....
Ahh, isn't University bandwidth great? :)
--Jim
On Mon, Sep 28, 2009 at 7:38 AM, Marten Lehmann lehmann@cnm.de wrote:
Hello,
We are doing about 35Mb/s of traffic on our CentOS box here...
that's definetely too much for us. We are really interested to be a CentOS mirror, but we cannot efford more than 10MBit average bandwidth for CentOS per month.
Is it possible to define in which ratio a mirror shall be listed for updates and downloads?
And is it possible to delist a mirror until the end of a month if the traffic has been too much for this certain month (e.g. due to a openoffice update)?
Regards Marten _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
On Mon, Sep 28, 2009 at 4:38 PM, Marten Lehmann lehmann@cnm.de wrote:
that's definetely too much for us. We are really interested to be a CentOS mirror, but we cannot efford more than 10MBit average bandwidth for CentOS per month.
We have 10Mbit mirrors, those normally aren't use for mirrorlist creation, though - which means that they probably will see less traffic.
Is it possible to define in which ratio a mirror shall be listed for updates and downloads?
No.
And is it possible to delist a mirror until the end of a month if the traffic has been too much for this certain month (e.g. due to a openoffice update)?
Also "no", because that would have to be done manually each time.
Ralph
-
Adrian Reber -
David Richardson -
Jim Kusznir -
Marten Lehmann -
Paul Stewart -
Ralph Angenendt