Hello,
We’ve always been able to install Storage SIG built Ceph packages without using EPEL9 but when we are testing with the Reef release it looks like we need a newer libarrow that hasn’t been rebuild in Storage SIG since newer thrift 0.15 was introduced.
We currently have parquet-libs-9.0.0-1.el9s in Storage SIG [1] but that has been built against thrift 0.14 so requires libthrift-0.14.0.so()(64bit) but we updated thrift to 0.15 a while ago [2] but newer rebuilt libarrow.
The rebuild in EPEL9 happend way back in parquet-libs-9.0.0-9.el9 with [3], perhaps we can rebuild libarrow in Storage SIG against current thrift as well to fix this?
@kkeithle: any chance you could start a new build for libarrow in Storage SIG?
- package librgw2-2:18.2.7-1.el9s.x86_64 from cloud requires libparquet.so.900()(64bit), but none of the providers can be installed - cannot install the best candidate for the job - nothing provides libthrift-0.14.0.so()(64bit) needed by parquet-libs-9.0.0-1.el9s.x86_64 from cloud
/Tobias
[1] https://cbs.centos.org/koji/rpminfo?rpmID=387534 [2] https://cbs.centos.org/koji/buildinfo?buildID=50735 [3] https://src.fedoraproject.org/rpms/libarrow/c/e1d0139e9685b9ddf29a84a4ba8021...
On Thu, Jun 12, 2025 at 4:45 AM Tobias Urdin - Binero < tobias.urdin@binero.com> wrote:
Hello,
We’ve always been able to install Storage SIG built Ceph packages without using EPEL9 but when we are testing with the Reef release it looks like we need a newer libarrow that hasn’t been rebuild in Storage SIG since newer thrift 0.15 was introduced.
Up until ceph-quincy or so on el8s and el9s epel was not used because CBS did not allow external repos like EPEL to be used.
When external repos like EPEL were added I switched from using dependencies built in CBS, including (lib)arrow and parquet-libs, to using the EPEL versions. Looks like I didn't get ceph-18.2.7 completely untangled.
I just respun ceph-18.2.7 using only EPEL for dependencies. Everything I tried, including librgw2, installs on my Stream9 box.
The new build is on some mirrors already, ymmv.
Hello,
Just so that I understand it correctly, EPEL 9 is now required to use Storage SIG packaged Ceph and is not expected (nor intended) to work without EPEL?
I wonder if it’s just tagging thrift 0.15.0 [1] with storage9s-ceph-reef-* tags that is missing and rebuilding libarrow for parquet-libs to use that would be something that could be done?
/Tobias
[1] https://cbs.centos.org/koji/buildinfo?buildID=50735 [2] https://cbs.centos.org/koji/buildinfo?buildID=40573
On 16 Jun 2025, at 22:03, Kaleb Keithley kkeithle@redhat.com wrote:
On Thu, Jun 12, 2025 at 4:45 AM Tobias Urdin - Binero <tobias.urdin@binero.commailto:tobias.urdin@binero.com> wrote: Hello,
We’ve always been able to install Storage SIG built Ceph packages without using EPEL9 but when we are testing with the Reef release it looks like we need a newer libarrow that hasn’t been rebuild in Storage SIG since newer thrift 0.15 was introduced.
Up until ceph-quincy or so on el8s and el9s epel was not used because CBS did not allow external repos like EPEL to be used.
When external repos like EPEL were added I switched from using dependencies built in CBS, including (lib)arrow and parquet-libs, to using the EPEL versions. Looks like I didn't get ceph-18.2.7 completely untangled.
I just respun ceph-18.2.7 using only EPEL for dependencies. Everything I tried, including librgw2, installs on my Stream9 box.
The new build is on some mirrors already, ymmv.
--
Kaleb
Hi,
Since 31-Mar-2022[1] Ceph Reef packages for 9Stream were intended (planned) to be built with EPEL, including using EPEL for runtime. Apache Arrow (libarrow, parquetlibs, et. al.) were packaged in EPEL9 on 08-Jun-2022[2]. Pre-release builds of Ceph Reef (18.1.x) were built in CBS and released to testing repos starting 14-Jun-2023[3]. The first GA build of Ceph Reef (18.2.0) was built and released on 04-Aug-2023[4]
The Arrow packages were available in EPEL well in advance of the release of Ceph Reef and it's evident that my intention was always to use EPEL. (Many people wanted to use EPEL, but found that there were incompatible discrepancies between the packages in EPEL and in CBS. Hence the decision to only use EPEL and stop providing dependencies in CBS.)
Yes, libarrow 9 does exist in CBS for 9Stream, possibly for Ceph Quincy, TBH I don't remember. It was never intended to be used by Ceph Reef. I don't know, nor do I remember, how it got tagged into Ceph Reef builds. It was obviously a mistake. I'm sorry it wasn't discovered earlier.
Ceph Reef (18.2.y), Ceph Squid (19.2.y), and soon Ceph Tentacle (20.x.y) all require EPEL.
If requiring EPEL doesn't work for you, I'm sorry.
[1] https://pagure.io/centos-infra/issue/725 [2] https://koji.fedoraproject.org/koji/buildinfo?buildID=1977931 [3] https://cbs.centos.org/koji/buildinfo?buildID=44367 [4] https://cbs.centos.org/koji/buildinfo?buildID=50233
On Mon, Jun 16, 2025 at 4:57 PM Tobias Urdin - Binero < tobias.urdin@binero.com> wrote:
Hello,
Just so that I understand it correctly, EPEL 9 is now required to use Storage SIG packaged Ceph and is not expected (nor intended) to work without EPEL?
I wonder if it’s just tagging thrift 0.15.0 [1] with storage9s-ceph-reef-* tags that is missing and rebuilding libarrow for parquet-libs to use that would be something that could be done?
/Tobias
[1] https://cbs.centos.org/koji/buildinfo?buildID=50735 [2] https://cbs.centos.org/koji/buildinfo?buildID=40573
On 16 Jun 2025, at 22:03, Kaleb Keithley kkeithle@redhat.com wrote:
On Thu, Jun 12, 2025 at 4:45 AM Tobias Urdin - Binero < tobias.urdin@binero.com> wrote:
Hello,
We’ve always been able to install Storage SIG built Ceph packages without using EPEL9 but when we are testing with the Reef release it looks like we need a newer libarrow that hasn’t been rebuild in Storage SIG since newer thrift 0.15 was introduced.
Up until ceph-quincy or so on el8s and el9s epel was not used because CBS did not allow external repos like EPEL to be used.
When external repos like EPEL were added I switched from using dependencies built in CBS, including (lib)arrow and parquet-libs, to using the EPEL versions. Looks like I didn't get ceph-18.2.7 completely untangled.
I just respun ceph-18.2.7 using only EPEL for dependencies. Everything I tried, including librgw2, installs on my Stream9 box.
The new build is on some mirrors already, ymmv.
--
Kaleb
Thanks for clearing it up Kaleb! Then we know how to proceed moving forward with newer versions.
Best regards Tobias
On 17 Jun 2025, at 00:20, Kaleb Keithley kkeithle@redhat.com wrote:
Hi,
Since 31-Mar-2022[1] Ceph Reef packages for 9Stream were intended (planned) to be built with EPEL, including using EPEL for runtime. Apache Arrow (libarrow, parquetlibs, et. al.) were packaged in EPEL9 on 08-Jun-2022[2]. Pre-release builds of Ceph Reef (18.1.x) were built in CBS and released to testing repos starting 14-Jun-2023[3]. The first GA build of Ceph Reef (18.2.0) was built and released on 04-Aug-2023[4]
The Arrow packages were available in EPEL well in advance of the release of Ceph Reef and it's evident that my intention was always to use EPEL. (Many people wanted to use EPEL, but found that there were incompatible discrepancies between the packages in EPEL and in CBS. Hence the decision to only use EPEL and stop providing dependencies in CBS.)
Yes, libarrow 9 does exist in CBS for 9Stream, possibly for Ceph Quincy, TBH I don't remember. It was never intended to be used by Ceph Reef. I don't know, nor do I remember, how it got tagged into Ceph Reef builds. It was obviously a mistake. I'm sorry it wasn't discovered earlier.
Ceph Reef (18.2.y), Ceph Squid (19.2.y), and soon Ceph Tentacle (20.x.y) all require EPEL.
If requiring EPEL doesn't work for you, I'm sorry.
[1] https://pagure.io/centos-infra/issue/725 [2] https://koji.fedoraproject.org/koji/buildinfo?buildID=1977931 [3] https://cbs.centos.org/koji/buildinfo?buildID=44367 [4] https://cbs.centos.org/koji/buildinfo?buildID=50233
On Mon, Jun 16, 2025 at 4:57 PM Tobias Urdin - Binero <tobias.urdin@binero.commailto:tobias.urdin@binero.com> wrote: Hello,
Just so that I understand it correctly, EPEL 9 is now required to use Storage SIG packaged Ceph and is not expected (nor intended) to work without EPEL?
I wonder if it’s just tagging thrift 0.15.0 [1] with storage9s-ceph-reef-* tags that is missing and rebuilding libarrow for parquet-libs to use that would be something that could be done?
/Tobias
[1] https://cbs.centos.org/koji/buildinfo?buildID=50735 [2] https://cbs.centos.org/koji/buildinfo?buildID=40573
On 16 Jun 2025, at 22:03, Kaleb Keithley <kkeithle@redhat.commailto:kkeithle@redhat.com> wrote:
On Thu, Jun 12, 2025 at 4:45 AM Tobias Urdin - Binero <tobias.urdin@binero.commailto:tobias.urdin@binero.com> wrote: Hello,
We’ve always been able to install Storage SIG built Ceph packages without using EPEL9 but when we are testing with the Reef release it looks like we need a newer libarrow that hasn’t been rebuild in Storage SIG since newer thrift 0.15 was introduced.
Up until ceph-quincy or so on el8s and el9s epel was not used because CBS did not allow external repos like EPEL to be used.
When external repos like EPEL were added I switched from using dependencies built in CBS, including (lib)arrow and parquet-libs, to using the EPEL versions. Looks like I didn't get ceph-18.2.7 completely untangled.
I just respun ceph-18.2.7 using only EPEL for dependencies. Everything I tried, including librgw2, installs on my Stream9 box.
The new build is on some mirrors already, ymmv.
--
Kaleb
--
Kaleb
-
Kaleb Keithley -
Tobias Urdin - Binero