[CentOS-devel] OT: os/ and kickstart/ seem identical on Centos 8?

Fri Sep 27 06:01:50 UTC 2019
Simon Matter <simon.matter at invoca.ch>

> On Thu, 26 Sep 2019 at 05:29, Simon Matter via CentOS-devel
> <centos-devel at centos.org> wrote:
>>
>> > On 26/09/2019 10:06, Kaj Niemi wrote:
>> >> Hi,
>> >>
>> >>
>> >> Is it on purpose that under AppStream, BaseOS, PowerTools (but not
>> under
>> >> extras, fasttrack and centosplusplus) there is an os and a kickstart
>> >> directory for each architecture? On a quick glance the directory
>> layout
>> >> and contents seem rather identical.
>> >>
>> >> Asking as “we” do a local mirror of releases and started wondering
>> about
>> >> the amount of disk space 8 takes.
>> >>
>> >
>> > Hi,
>> >
>> > kickstart is a snapshot of os at GA/release time.
>> > So that permits people to deploy with exactly same content without
>> > having a moving target, as BaseOS/AppStream will be including updates
>> as
>> > they land.
>> >
>> > WRT disk space, hopefully, as explained on how to mirror correctly
>> > (https://wiki.centos.org/HowTos/CreatePublicMirrors) , you use -H for
>> > rsync, as we heavily use hardlinks in our trees so that it reduces
>> space
>> > and also used bandwidth to sync mirror content
>>
>> Hi,
>>
>> I was just wondering why it wasn't implemented with symlinks? They have
>> the advantage that they can be recognized easily IMHO. You could then go
>> a
>> step further and even create an updates/ directory with only symlinks to
>> the corresponding files. It would allow to still have an updates/
>> directory which some of us are missing now.
>>
>
> From dealing with mirroring issues a long time, large number of
> symlinks in a directory have broken various filesystems that various
> systems use. You end up with sites with missing symlinks, broken
> symlinks or other anomalies. If you hardlink the mirror might download
> the file twice but it will be there. If you symlink hundreds of files,
> you are debugging client issues to a mirror you don't control and you
> have to make your check script even more complicated to confirm that
> things are there versus just look to be there.

Interesting, I never saw issues with symlinks. Was it a problem with
filesystems or a problem with the sync tool?

A bit OT here but: I do have problems syncing large filesystems which
contain large number of hardlinks using rsync. I get the following error
and don't know how to go on. That's with rsync -aH:

building file list ... ERROR: out of memory in hashtable_node [sender]
rsync error: error allocating core memory buffers (code 22) at util.c(117)
[sender=3.0.6]

I've built latest rsync to try but it fails the same way. Is the only way
to fix this to add more memory to the systems?

Thanks,
Simon