[CentOS-devel] Importing CentOS-6 Sources into git.centos.org

[Nico Kadel-Garcia]

On Thu, Aug 21, 2014 at 1:54 PM, Stephen John Smoogen <smooge at gmail.com> wrote:

> On 20 August 2014 19:29, Nico Kadel-Garcia <nkadel at gmail.com> wrote:
>>
>> On Wed, Aug 20, 2014 at 5:24 PM, Karanbir Singh <mail-lists at karan.org>

>> > I'd like to be able to offer rsync from there as well, but there are a
>> > few challenges that need resolved first. For the binary content cache,
>> > we can likely run the rsync instance from the backup machine so there is
>> > no network load on the production box. For the git repos its a bit
>> > harder since there are private or working-in-progress repos in there as
>> > well, and we need to find a way to mask those out.
>> >
>> > Certainly worth trying to get to.
>>
>> Use GPG signed git tags to assure provenance, and the repository can
>> be safely cloned. Rsyncing a git repo is like rsyncing a CVS or
>> Subversion reository. Even small changes in the midst of the rsync
>> operation can corrupt the underlying database.
>
>
> Could you please cut down the broken record please? There is no sign there
> will be GPG signed git tags in the near future and your constant harping on
> it is not going to make it happen any faster.

Sorry it bothers you. I didn't bring up using rsync to make mirrors.
I'm trying to get across the provenance and propagation problem.

Besides the potential corrupt snapshot problem, there's the inevitable
discrepancies between the mirrors and git.centos.org itself. Content
is likely to differ in small ways among the mirrors, due to the rsync
based snapshot being in the past.  I assume that some of the
individual repos are changing during the overall rsumc update period,
unless they're all done in parallel, which would be *really* nasty.

Unless.... Is there a top level directory to use for an rsync mirror?
That's going to be a pretty bulky rsync operation, with over 6000
subdirectories and the amount of churn in any of the modified git
repos.

Anyway, verification of the consistency of all the mirrored
repositories becomes awkward. There's also the lack of site
verification in the unencrypted and unsigned rsync protocol, which I'd
not even thought about for git.centos.org. That puts it right into the
"people cloning from each other's unsecured repos locally" world, in
this case cloning from the rsync mirrors. And it directly brings up
the "verify the provenance of local repos" problem that was discounted
by some when I brought up the problem earlier.

Several folks did bring up the point of "git.centos.org has an SSL
key, what's not secure about it?"  If we're using rsync mirrors, we're
relying on someone else's mirror site to be secure, as well. And we're
probably relying on unencrypted rsync to git.centos.org, itself, to
support those mirrors. And we're once again open to someone polluting
the data stream with a fake repo.

No, *if* our friends at git.centos.org want to help protect that data
stream for mirror clients, they can consider using something like
rsync with ssh keys and the old "validate-rsync.sh" script as a
ForceCommand. A site that wants to be a mirror would need a relevant
private SSH key, and unlocking it for rsync use is their problem. But
that would at least help assure provenance between the mirror sites
and git.centos.org.

The repercussions of using rsync for this start adding up pretty fast.