On Jul 24, 2011, at 5:37 PM, Ljubomir Ljubojevic wrote:
Jeff Johnson wrote:
On Jul 24, 2011, at 4:35 PM, Ljubomir Ljubojevic wrote:
Oh, yeah, yum reads and process xml files, not actual files, so searches are fast because of it.
Here's something that might help you:
Using xml is a significant performance hit: see recent patches to yum/createrepo to use sqlite instead of xml … lemme find the check-in claim … here is the claim http://lists.baseurl.org/pipermail/rpm-metadata/2011-July/001353.html and quoting
Tested locally on repodata of 9000 pkgs.
Goes from 1.8-> 2GB of memory in use with the old createrepo code to 325MB of memory in use - same operation - performance-wise it is not considerably different. More testing will bear that out, though.
So -- if I believe those numbers -- there's *lots* of room for improvement in yum ripping out xml and replacing with a sqlite database. Note that createrepo != yum but some of the usage cases are similar. The general problem in yum (and smart and apt) is the high cost of the cache load, and the amount of aml that must be parsed/read in order to be cached. Adding a sqlite backing store which can just be used, not loaded, is a win.
You have mistaken createrepo with yum repomd data. Createrepo is for creating actual repository (I use mrepo).
I haven't (if you read what I said carefully). Meanwhile mrepo is nicely done, worth using if you have to babysit tonnes of package metadata. I like what Dag implements, sane and simple and useful.
Yum data (repomd, repoview) is different story. Every repository stores data in xml file packed with tar. They are unpacked in memory and xml data is parsed and put into internal database (and cache). It is very much possible that yum internally (for cache) uses sqlite database, haven't had the need to research. Using "yum -C <command>" will use yum cache rather then download repomd data again.
Please note that I'm speaking way way generally and from memory. What you gave me was a data point about how well yum performs, and yum is better than I would have guessed with 10+ repositories underneath it.
Anything else you read is pure crack smoke from me thinking out loud. I don't even agree with myself often ;-)
73 de Jeff
--
Ljubomir Ljubojevic (Love is in the Air) PL Computers Serbia, Europe
Google is the Mother, Google is the Father, and traceroute is your trusty Spiderman... StarOS, Mikrotik and CentOS/RHEL/Linux consultant _______________________________________________ CentOS-devel mailing list CentOS-devel@centos.org http://lists.centos.org/mailman/listinfo/centos-devel