[CentOS-devel] Bringing glibc 2.34 snapshots to Fedora 35 and CentOS Stream 9

Wed Jul 7 16:59:38 UTC 2021
Florian Weimer <fweimer at redhat.com>

This is a heads-up that the first import of a glibc 2.34 snapshot will
land in the compose soon (glibc-2.33.9000-36.el9).  I believe we have
solution for the dependency issues mentioned below, with additional
versioned dependencies that are autogenerated.

Thanks,
Florian

* Florian Weimer:

> TL;DR: glibc 2.34 snapshots are coming. libpthread as a separate library
> creates problems, so we are removing it.  There may be some
> hickups. Full updates with “dnf update” are recommended.
>
>
> We expect that we will soon be able to import glibc upstream snapshots
> regularly into Fedora 35 and CentOS Stream 9.  We did such regular
> imports for the past couple of Fedora rawhide releases, but the
> situation will be slightly different, as explained below.  The snapshots
> will also land in CentOS Stream 9, after a delay as the result of a
> different testing pipeline.
>
> glibc 2.33 and earlier accidentally provided an very generic ROP gadget
> in the statically linked startup code that is present in every program
> (even dynamically linked ones).  We have removed the ROP gadget and
> moved that particular initialization code into the dynamically linked
> part, but that means that the interface between the startup code and the
> dynamically loaded glibc parts had to change.  As a result, any program
> linked against glibc 2.34 will not run with glibc 2.33 or any earlier
> version.  (Some of you may remember the memcpy at GLIBC_2.14 issue, it was
> similar.)  This version requirement will be properly reflected in RPM
> dependencies.
>
> glibc 2.34 removes libpthread as a separate library.  This is based on
> the observation that most processes load libpthread anyway.  (On this
> system, I count 141 out of 159 processes that load libpthread.)  One
> particularly thorny issue is that certain NSS modules depend on
> libpthread, and if the main program is not linked (indirectly) against
> libpthread, loading such NSS modules effectively loads libpthread via
> dlopen, which is something that glibc has never supported well.
> Furthermore, the availability of some pthread_* functions without
> -lpthread has been a source of confusion to developers, resulting in
> both overlinking and underlinking.
>
> We started the libpthread transition in glibc 2.33 when we added the
> __libc_single_threaded variable as a replacement for the weak symbol
> hacks that some libraries use to detect single-threaded processes.  (For
> example, libstdc++ used to do this to avoid using atomic instructions in
> std::shared_ptr.)  Backwards compatibility for dynamically linked
> binaries should be preserved (as usual), but we know of one issue on
> ppc64le, where the weak symbol hacks resulted in slightly corrupt
> binaries.  Upstream glibc did not accept the proposed backwards
> compatibility enhancement so far.  Instead, we will rebuild affected
> distribution binaries with a binutils version which handles weak symbols
> slightly differently, avoiding the corruption.  As we plan to perform
> these rebuilds before the new glibc lands in the buildroot, the
> requirement to upgrade to the new binaries before or at the same time of
> the upgrade to the glibc upstream snapshot will NOT be reflected in RPM
> dependencies.  A full upgrade using “dnf update” will work, however.
>
> In addition to the removal of libpthread, I also hope to remove libdl
> and librt.  This means that new GLIBC_2.34 symbols will be added to
> libc.so.6 (e.g., timer_create at GLIBC_2.34).  The librt removal in
> particular will probably not land with the first imported snapshot.
> This is unfortunate because RPM does not have a way to express this:
> there's just a blanket Provides: libc.so.6(GLIBC_2.34), which will
> already be included in the first snapshot that adds the symbol
> __lib_start_main@@GLIBC_2.34 (whether or not the RPM package includes
> timer_create at GLIBC_2.34 as well).  Some tools in the fedpkg/centpkg/mock
> sphere try to install builds from the Koji buildroot into installations
> from composes, without upgrading everything to the current buildroot, so
> it's possible that glibc won't be upgraded to match the requirements of
> RPMs directly imported from the buildroot.  Developers encountered
> similar issues with glibc snapshot imports (e.g., around the symbol
> pthread_getattr_np).  Our RPM infrastructure does not have per-symbol
> dependencies, so there isn't much we can do about it at the packaging
> level.  It's a transitory issue during rawhide/CentOS Stream 9
> development; the finished release will add all GLIBC_2.34 symbols in one
> upgrade, so end users won't see it in a Fedora 34 to Fedora 35 upgrade
> or a CentOS Stream 8 to CentOS Stream 9 upgrade.
>
> We think there is value in providing access to these snapshots early,
> and will try to make the transitions as smooth as possible, within the
> constraints outlined above.