I would generally caution saying that, even the largest mirrors don't take these numbers lightly
- Centos' archive is 333G of data - Centos' primary mirroring is 162G of data - A single release (5.4) uses 32G of disk
Ignoring bandwidth (that's a lot harder to argue what's useful or not, though I'd personally say mirrors should start around 100mbps unless they are in a country without a mirror and than anything goes), and making the assumption that your only mirroring centos you now need ~500G of disk space and a machine with 32G - 24G of ram to be able to serve this well.
Why the disk space? That's obvious
Why the ram? This is something everyone forgets, but it basically boils down to the minute you have to start going to disk for your working set you will lose when faced with a horde of people trying to get data. If you have less memory than your working data set (in the case of Centos 5.4 it's 32G) than your going to be thrashing against disk, and ultimately no matter how much bandwidth you have you will be limited by the speed that you can get that data off of disk.
Now ram has gotten cheaper, thankfully, but I'm just talking about Centos on a mirror, most mirrors have more than just Centos on them. Case in point I have: - archlinux - centos - cpan - debian - fedora - fedora-epel - gentoo - gnu - LDP - mandriva - moblin - oldlinux - opensuse - redhat - slackware - suse - ubuntu
Want to guess what my *daily* working data set is? Want to guess what my working data set is when a release happens? Want to guess wat my working data set is when *MULTIPLE* releases happen simultaneously? the answer to some of those is: they don't make boxes with enough ram to support that yet, or at least that are within the wildest budgetary constraints of a mirror.
My argument about getting rid of the CD ISOs isn't about the disk space, though that's a valid point (good fast disk is expensive - a 450G 15K rpm sas drive currently runs $400+ /ea - thats expensive storage), my argument about killing the CD ISOs is one from a ram perspective, though looking at my disk usage right now I might be making the disk argument in the near future:
/dev/md0 5.5T 4.2T 1.3T 77% /home/mirrors
- John 'Warthog9' Hawley
Nuno Vieira - nfsi wrote:
Hey folks,
Who don't have enough bandwith nor storage space doesn't deserve to be a Official CentOS mirror.
My 100,00 EUR.
cheers,
Nuno Vieira nfsi telecom, lda.
[Email] nuno.vieira@nfsi.pt [Phone] +351 21 114 2315 [Phone] +351 21 142 2300 [Mobile] +351 91 925 5561 [Fax] +351 21 114 2301 [Web] http://www.nfsi.pt/
----- Original Message ----- From: "J.H." warthog9@kernel.org To: "Mailing list for CentOS mirrors." centos-mirror@centos.org Sent: Friday, 23 October, 2009 8:53:35 PM Subject: Re: [CentOS-mirror] New mirror
Jeff Sheltren wrote:
On Oct 23, 2009, at 10:22 AM, Nick Olsen wrote:
Never Thought of that.... I guess your right. Don't really see why ISO's shouldn't be carried though.
Disk space.
Some people (I won't name names, *cough* warthog *cough*) might argue that having ISO images is simply a replication of the packages we're already carrying on the mirror and that there should be a better way to handle stuff so that mirrors don't end up with multiple copies of what is essentially the same data.
I'm trying my best to kill those stupid ISO images - I mean I've got boot.kernel.org and I've done several installs / upgrades that way (including Centos I might add!), and as a general goal I want to eliminate as many needs to burn a cd for a task as I can.
That said I realize that I'm "not normal" and at best 5 years ahead of the big curve. Many people's internet connections are not as good as mine, and it's only 16mbps down / 2mbps up. Compare that to some of the other places on the planet with 10mbps symetric to 1000mbps symetric and mine pales.
I have no real expectation however that we will get rid of the ISOs anytime soon. I would *LOVE* if we could drop the CD ISOs completely from everything, but there's apparently a major backlash every time that happens (Fedora's done it a couple of times now). I'm kinda hoping that with boot.kernel.org and the DVD ISOs we might be able to finally kill the CD ISO itself off and save all of that space and eliminate that from the possible working set of data. Just my $0.02 though.
- John 'Warthog9' Hawley
CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror