On 07.11.2011 06:40, Matt Domsch wrote: > On Sat, Nov 05, 2011 at 04:51:43PM -0500, Ralph Angenendt wrote: >> Name: Is our primary key - so every mirror has to have a unique name in >> our DB > > MM has a "Site" that matches your URL to the Sponsor's website. Each > Site has a list of mirror Hosts. Each Site's name must be unique, and > Within each Site, the Hosts names must be unique. Okay, the site URL isn't unique per se, but unique per sponsor (if he has many hosts). >> location-minor: Country (or in the case of US and Canada: State) > > MM lets GeoIP handle this for us, to the country level. I haven't yet > added State-level - oftentimes it really doesn't match up with network > topology enough to care. That is only for representation on the website anyway. What's used for generating the lists is in cc: (Host.country). >> http,ftp,rsync: The URLs the mirror is at > > HostCategoryURL, two types: public (default), and private for other > downstream mirrors to use. Not sure these are actually used. Hmm? Does not compute: Those are the actual URLs of the mirror content. >> bandwidth: Actual bandwidth. Not needed. > > MM does need this, an integer value in Mbps (100 = 100Mbps uplink). Host.bandwidth_int. Okay. As this is free form for us, this needs normalizing, then. >> state: more detailed state > > ? The reason why it was disabled (lagging, non-responsive and so on. Nobody really uses that). >> contact-name: Name of the person running the mirror. Internal use for us. >> contact-tel: I cannot remember calling a mirroradmin :) >> contact-email: Our second unique field. I guess that will be used for login > > MM only knows about a user account name we list as the mirror admin. Hmmm. contact-email in that case. > In the Fedora world, this is the FAS account name. In RPMFusion, I > expect it's a local database built into TurboGears. Pretty thin on > info though, could add these other fields if we need to. I don't know if they are needed. But we need a user db, we have no general account system in our infrastructure. > MM has content Categories (Fedora Linux, Fedora EPEL, and historical > categories). Each Host has one or more Categories = HostCategory. > Each HostCategory has one or more HostCategoryURLs. The Categories > can be rooted at any arbitrary URL, but from the top of the Category > on down, a mirror has to maintain the upstream master directory > structure. Ummm. I need to digest that :) (but we require the mirrors to have the same structure, otherwise they won't show up in the mirrorlist yum uses). > You'll want to think about how you structure your content into > Categories. It works best if a Category is a distinct subtree, not > overlapping with other Categories. I guess something like Releases would be best here? 4, 5, 6, 7 ... We don't have anything else. > This is detected dynamically by MM, and exposed in the publiclist > chooser. Great, because entering and checking that can be a PITA :) >> dvd-iso: Does it carry them (always yes since 6) >> dvd*: The versions (6 is set to yes always) >> dvd-iso-host,rsync-dvd-host: No idea. Not used > > Again, dynamically detected. When switching to mirrormanager everyone will get the dvds anyway. We'll drop the double tree then. > Not used. MM uses GeoIP, and augments its mapping of countries to > continents with a CountryContinentRedirect. Little used, but it maps > say Israel to Europe instead of Asia, because it has better network > connectivity to Europe. Okay. We had rather longish discussions on this list about those mappings :) >> centos_code,priority: Not used >> use-in-mirror-list: Used: We don't really put 10Mbit-machines in EU or >> US or CA into the mirrorlist.txt which is handed out via yum > > Ah. We do, but they get listed at the top of mirrorlist.txt 1/100 as > often as a 1Gb mirror would. That's the weighted random sample based > on bandwidth. Yeah, that is fine. > I don't have the breakdown by continent in /publiclist. Wouldn't be > hard, but I hate mucking with that page - it took some major CSS > hacking to get it as readable as it is. :-) What Adrian pointed me too looks okay.Unless one of the mirror sponsors object (anyone still with us here?). >> Anything I actually overlooked? > > Do you have private mirrors in your database now? That maps to > Site.private and Host.private. No. And I am not sure if we want to allow them - but that is open to discussion. > Host.internet2 if a host is on Internet2 or related high-speed > educational/research network. We can look that up in MM's private copy > of the Internet2 route tables if needed - that's how I populated the > field the first time for Fedora too. Okay. I think I like that idea. > Host.internet2_clients if a host on I2, even if private, should be > listed for other I2 clients in the same country. By default set it > false, let mirror admins update it themselves. Yeah, that's fine, too. > Host.asn = AS Number > Host.asn_clients if a host should serve the whole ASN regardless of > netblocks set. Lets mirrors in places with many netblocks, but a > single ASN, get away with a single value here. Again, we can look > this up in our private copy of the worldwide routing table. Wonderful. > Host.countries_allowed = list of countries allowed. e.g. a mirror in > .il may want to only serve users in .il. Hmmm. Okay, I can understand that in countries with few mirrors. > Host.acl_ips = list of IP addresses or hostnames that will get put > into the /rsync_acl list. Other mirrors may wget /rsync_acl to get > that list, and use it in their own rsyncd.conf files. Only real > problem with it is anyone could sign up to be a private mirror, fill > this in, and then get early access to a pre-bitflip mirror via the > acl. Oh well... :) > I think that'll be enough to get going though. > > Be thinking about categories. At a glance, I think a single Category > "CentOS" would be fine. You could in theory do two Categories > "CentOS" and "CentOS ISOs" and rig up update-master-directory-list to > ignore /isos in your "CentOS" Category, and ignore everything but > /isos in the "CentOS ISOs" Category. but I don't think that will > buy you much, and it buys exactly nothing with C6 and newer. No, it probably won't. I was thinking in Releases, maybe, they don't fluctuate that fast as in Fedoraland. I am actually populating the machine now with a tree and will do some toying around with mirrormanager during this week, to see what I am up against. I might drop some questions on IRC :) Cheers and thanks, Ralph