[CentOS-devel] mock using yum .repo file?
Ljubomir Ljubojevic
office at plnet.rs
Sun Jul 24 20:35:29 UTC 2011
Jeff Johnson wrote:
> You are correct that the scaling depends on the number of packages
> not the number of repositories.
>
> However the solution to a distributed lookup scaling problem *does* depend
> on the number of places that have to be searched as well as the cost of a
> failed lookup. If you have to look in a large number of repositories to ensure
> that some packages does NOT exist anywhere, well there are ways to do that efficiently.
>
> And none of the right solutions to the increasing cost of a failed lookup
> are implemented in yum afiak.
>
> I was hoping to get an estimate of how bad the scaling problem actually
> is from an objective wall clock time seat-of-the-pants measurement.
>
> Meanwhile I'm happy that you've found a workable solution for
> your purposes. I'm rather more interested in what happens when
> there hundreds of repositories and 10's of thousands of packages
> that MUST be searched.
>
> I suspect that yum will melt into a puddle if/when faced with depsolving
> on that scale. Not that anyone needs depsolving on the scale of
> hundreds of repos and 10's of thousands of packages in the "real world",
> but that isn't a proper justification for not considering the cost
> of a failed lookup carefully which (from what you are telling me)
> you are already seeing, and dealing with by enabling/disabling repositories
> and inserting a high priority repository that is also acting as a de facto
> cache and "working set" for the most useful packages.
>
Here are 2 speed tests. Repos are on 100Mbps LAN. First is with all
repositories enabled and (left from CentOS 5 and old style RPMForge)
aTrpms has higher priority over RepoForge. Second is with only safe
RepoForge and other default (in my sistem) repositories enabled:
[root at mama ~]# yum clean all
Loaded plugins: fastestmirror, priorities, refresh-packagekit
Cleaning up Everything
Cleaning up list of fastest mirrors
[root at mama ~]# time yum install vlc --enablerepo=* --showduplicates
Loaded plugins: fastestmirror, priorities, refresh-packagekit
Loading mirror speeds from cached hostfile
2233 packages excluded due to repository priority protections
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package vlc.x86_64 0:1.1.10-71.el6 set to be updated
--> Processing Dependency: bitstream-vera-serif-fonts for package:
vlc-1.1.10-71.el6.x86_64
<snip>
--> Finished Dependency Resolution
Error: Package: vlc-1.1.10-71.el6.x86_64 (plc-atrpms-stable)
Requires: libmodplug.so.0()(64bit)
You could try using --skip-broken to work around the problem
You could try running: rpm -Va --nofiles --nodigest
real 1m9.365s
user 0m36.585s
sys 0m2.830s
[root at mama ~]# time yum install vlc --showduplicates
Loaded plugins: fastestmirror, priorities, refresh-packagekit
Loading mirror speeds from cached hostfile
952 packages excluded due to repository priority protections
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package vlc.x86_64 0:1.1.11-1.el6.rf set to be updated
--> Processing Dependency: libavformat.so.52(LIBAVFORMAT_52)(64bit) for
package: vlc-1.1.11-1.el6.rf.x86_64
<snip>
--> Finished Dependency Resolution
Error: Package: vlc-1.1.11-1.el6.rf.x86_64 (plc-repoforge)
Requires: libmodplug.so.0()(64bit)
You could try using --skip-broken to work around the problem
You could try running: rpm -Va --nofiles --nodigest
real 0m30.869s
user 0m25.619s
sys 0m1.261s
Difference is double, but time difference is only 30sec on low budget PC
(1GB RAM, AMD Sempron 64-bit, 40GB IDE HDD).
And first run had to download yum repo files, second was from cache.
If you have some specific stres test I would be happy to run it.
Oh, yeah, yum reads and process xml files, not actual files, so searches
are fast because of it.
Here are repolist data (1=all repos, 2=default, 3 = default with no
rpmforge and epel):
[root at mama ~]# yum clean all
Loaded plugins: fastestmirror, priorities, refresh-packagekit
Cleaning up Everything
Cleaning up list of fastest mirrors
[root at mama ~]# time yum repolist --enablerepo=*
Loaded plugins: fastestmirror, priorities, refresh-packagekit
Loading mirror speeds from cached hostfile
2233 packages excluded due to repository priority protections
<snip>
repo id repo name
status
playonlinux 2+6
plc-adobe-linux 0
plc-atrpms-stable 1,636+382
plc-atrpms-testing 43+67
plc-c6-testing 0
plc-centosplus 42
plc-elrepo 92
plc-elrepo-extras 0
plc-elrepo-fasttrack 0
plc-elrepo-kernel 0
plc-elrepo-testing 31
plc-epel 6,080
plc-extras 0
plc-fasttrack 0
plc-kb-el6-ext 0
plc-kb-el6-ext-test 0
plc-kb-el6-misc 0
plc-kb-el6-misc-test 0
plc-os 6,019
plc-releases 0
plc-remi 114+223
plc-remi-test 2+76
plc-repoforge 2,835+1,106
plc-repoforge-buildtools 0
plc-repoforge-dag 0
plc-repoforge-extras 25+357
plc-rpmfusion-free-updates 0
plc-rpmfusion-free-updates-testing 70
plc-rpmfusion-nonfree-updates 0
plc-rpmfusion-nonfree-updates-testing 70
plc-sernet-samba 0
plc-updates 1,042
plc-virtualbox 10
plc-virtualmin-universal 134
plnet-archive 0
plnet-compiled 28
plnet-downloaded 89+16
plnet-releases 0
plnet-replace 0
plnet-test 0
repolist: 18,364
real 0m29.675s
user 0m15.167s
sys 0m1.633s
[root at mama ~]# yum clean all
Loaded plugins: fastestmirror, priorities, refresh-packagekit
Cleaning up Everything
Cleaning up list of fastest mirrors
[root at mama ~]# time yum repolist
Loaded plugins: fastestmirror, priorities, refresh-packagekit
Determining fastest mirrors
<snip>
952 packages excluded due to repository priority protections
repo id status
playonlinux 2+6
plc-adobe-linux 0
plc-centosplus 42
plc-elrepo 92
plc-elrepo-kernel 0
plc-epel 6,080
plc-extras 0
plc-os 6,019
plc-releases 0
plc-repoforge 3,011+930
plc-updates 1,042
plc-virtualbox 10
plc-virtualmin-universal 134
plnet-compiled 28
plnet-downloaded 89+16
plnet-releases 0
repolist: 16,549
real 0m28.276s
user 0m14.475s
sys 0m1.658s
[root at mama ~]# yum clean all
Loaded plugins: fastestmirror, priorities, refresh-packagekit
Cleaning up Everything
Cleaning up list of fastest mirrors
[root at mama ~]# time yum repolist --disablerepo=*repoforge*
--disablerepo=*epel*
Loaded plugins: fastestmirror, priorities, refresh-packagekit
Determining fastest mirrors
<snip>
17 packages excluded due to repository priority protections
repo id status
playonlinux 2+6
plc-adobe-linux 0
plc-centosplus 42
plc-elrepo 92
plc-elrepo-kernel 0
plc-extras 0
plc-os 6,019
plc-releases 0
plc-updates 1,042
plc-virtualbox 10
plc-virtualmin-universal 134
plnet-compiled 28
plnet-downloaded 94+11
plnet-releases 0
repolist: 7,463
real 0m19.810s
user 0m8.663s
sys 0m1.227s
--
Ljubomir Ljubojevic
(Love is in the Air)
PL Computers
Serbia, Europe
Google is the Mother, Google is the Father, and traceroute is your
trusty Spiderman...
StarOS, Mikrotik and CentOS/RHEL/Linux consultant
More information about the CentOS-devel
mailing list