I know of a few techniques for minimizing server bandwidth when bittorent is not an option.
1) If you have the beta isos, you can rename them as centos 5 isos and then rsync from a mirror that allows rsync (such as kernel.org). Then only changes from beta to centos 5 plus some overhead is downloaded. I suppose you could use this technique with bittorrent too.
2) Instead of downloading both cd and dvd isos, just download the cd isos and use a script to create the dvd iso. You can rsync like step 1 if the dvd md5sum differs.
A cd to dvd script is at url below (although I haven't tried it yet). http://www.tuxforums.org/forum/viewtopic.php?t=446
On 4/12/07, Dale Sykora Dale.Sykora@7core.biz wrote:
A cd to dvd script is at url below (although I haven't tried it yet). http://www.tuxforums.org/forum/viewtopic.php?t=446
The script does not take care of repodata. After all files have been combined, the repodata direcory needs to be recompiled. This can be done by inserting this line:
/usr/bin/createrepo -g repodata/comps.xml ./
just after the line cd $DVD and before the line mkisofs.
Akemi
Dale Sykora wrote:
I know of a few techniques for minimizing server bandwidth when bittorent is not an option.
- If you have the beta isos, you can rename them as centos 5 isos and
then rsync from a mirror that allows rsync (such as kernel.org). Then only changes from beta to centos 5 plus some overhead is downloaded. I suppose you could use this technique with bittorrent too.
if one file near the beginning of the ISO changes size by even a block, then the whole rest of the ISO will be different, this will gain you nothing.
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of John R Pierce Sent: Thursday, April 12, 2007 2:27 PM To: CentOS mailing list Subject: Re: [CentOS] minimize download bandwidth techniques
Dale Sykora wrote:
I know of a few techniques for minimizing server bandwidth when bittorent is not an option.
- If you have the beta isos, you can rename them as centos
5 isos and
then rsync from a mirror that allows rsync (such as
kernel.org). Then
only changes from beta to centos 5 plus some overhead is
downloaded.
I suppose you could use this technique with bittorrent too.
if one file near the beginning of the ISO changes size by even a block, then the whole rest of the ISO will be different, this will gain you nothing.
I used to download K12LTSP isos this way between beta versions and it seemed to speed up the process. I suppose your mileage may vary.
On Thu, 12 Apr 2007, John R Pierce wrote:
Dale Sykora wrote:
I know of a few techniques for minimizing server bandwidth when bittorent is not an option.
- If you have the beta isos, you can rename them as centos 5 isos and
then rsync from a mirror that allows rsync (such as kernel.org). Then only changes from beta to centos 5 plus some overhead is downloaded. I suppose you could use this technique with bittorrent too.
if one file near the beginning of the ISO changes size by even a block, then the whole rest of the ISO will be different, this will gain you nothing.
With bittorrent, that is perhaps true. Rsync will be able to handle that difference. But if most RPMs are rebuild that it will have not much matching data.
Wojtek
Wojtek.Pilorz wrote:
On Thu, 12 Apr 2007, John R Pierce wrote:
Dale Sykora wrote:
I know of a few techniques for minimizing server bandwidth when bittorent is not an option.
- If you have the beta isos, you can rename them as centos 5 isos and
then rsync from a mirror that allows rsync (such as kernel.org). Then only changes from beta to centos 5 plus some overhead is downloaded. I suppose you could use this technique with bittorrent too.
if one file near the beginning of the ISO changes size by even a block, then the whole rest of the ISO will be different, this will gain you nothing.
With bittorrent, that is perhaps true. Rsync will be able to handle that difference. But if most RPMs are rebuild that it will have not much matching data.
By way of example, I had a recent daily build of Debian Etch (mid March). When Debian released (must be a blue moon!) earlier this week, I rsynced the three DVD images. rsync did it really quickly, and claimed over 1 mbyte/sec on an ADSL 256k incoming link.
Another example; a few years ago I had a broken ISO that rsync would not fix, it was the same size and timestamp as the original, and checksumming was forbidden.
I pruned a little off by copying about 1k less with dd, then rsync did fix it. (wonders why touch didn't work. hmm.)
-
Akemi at gmail -
Dale Sykora -
John R Pierce -
John Summerfield -
Wojtek.Pilorz