Hi,
I've already asked this on the dpdk-users list (https://mails.dpdk.org/archives/users/2022-April/006395.html) without getting any answer, so I thought maybe I'd also try centos-devel, since this is a CentOS and dpdk question.
We're using a CentOS 7 based distro that is obviously RPM-based, and we package the DPDK with it, in order to run on some of our x86-64 hardware boards. Now, we currently have DPDK 20.11 in our image and are looking to upgrade to 21.11, and I'm hitting a build error about elftools not being found:
Program python3 found: YES (/usr/bin/python3)
../buildtools/meson.build:49:8: ERROR: Problem encountered: missing python module: elftools
The fix seems to be to install pyelftools from pip:
pip3 install pyelftools
But that only works for building directly with meson / ninja. That doesn't work when building through rpmbuild. I've struggled to understand this, until I found this RedHat bug: https://bugzilla.redhat.com/show_bug.cgi?id=1937494. It seems Fedora changed the python3 search path so that /usr/local/lib/python is not taken into account if RPM_BUILD_ROOT is defined (see https://fedoraproject.org/wiki/Changes/Making_sudo_pip_safe); so basically, when building with rpmbuild, pip-installed packages are ignored.
I think I can understand the rationale of the Fedora change when you want to ensure that dependencies are properly set: packages installed using pip are not recorded as RPM dependencies, and the package might break at runtime. However, as far as I can tell, pyelftools seems to be a *build-time dependency*; removing the elftools check in the meson script leads to a build failure. There's no pyelftools RPM for CentOS 7 that I could find, so I don't see another option than installing the package from pip. Also, the python change in Fedora doesn't seem to allow for any workaround. The only thing that I can think of is ugly as hell: symlink everything under /usr/local/python in /usr/lib/python for the DPDK build. I'm not even sure we need pyelftools at runtime; in our build of DPDK-20.11, dpdk-pmdinfo.py doesn't even run (because of missing pyelftools :) ).
Are there any other people struggling with this? Is there any more elegant approach that I haven't thought of?
Thanks, Stefan.
On Fri, Apr 29, 2022 at 7:50 AM Stefan Puiu stefan.puiu@gmail.com wrote:
Hi,
I've already asked this on the dpdk-users list (https://mails.dpdk.org/archives/users/2022-April/006395.html) without getting any answer, so I thought maybe I'd also try centos-devel, since this is a CentOS and dpdk question.
We're using a CentOS 7 based distro that is obviously RPM-based, and we package the DPDK with it, in order to run on some of our x86-64 hardware boards. Now, we currently have DPDK 20.11 in our image and are looking to upgrade to 21.11, and I'm hitting a build error about elftools not being found:
Program python3 found: YES (/usr/bin/python3)
../buildtools/meson.build:49:8: ERROR: Problem encountered: missing python module: elftools
The fix seems to be to install pyelftools from pip:
pip3 install pyelftools
But that only works for building directly with meson / ninja. That doesn't work when building through rpmbuild. I've struggled to understand this, until I found this RedHat bug: https://bugzilla.redhat.com/show_bug.cgi?id=1937494. It seems Fedora changed the python3 search path so that /usr/local/lib/python is not taken into account if RPM_BUILD_ROOT is defined (see https://fedoraproject.org/wiki/Changes/Making_sudo_pip_safe); so basically, when building with rpmbuild, pip-installed packages are ignored.
I think I can understand the rationale of the Fedora change when you want to ensure that dependencies are properly set: packages installed using pip are not recorded as RPM dependencies, and the package might break at runtime. However, as far as I can tell, pyelftools seems to be a *build-time dependency*; removing the elftools check in the meson script leads to a build failure. There's no pyelftools RPM for CentOS 7 that I could find, so I don't see another option than installing the package from pip. Also, the python change in Fedora doesn't seem to allow for any workaround. The only thing that I can think of is ugly as hell: symlink everything under /usr/local/python in /usr/lib/python for the DPDK build. I'm not even sure we need pyelftools at runtime; in our build of DPDK-20.11, dpdk-pmdinfo.py doesn't even run (because of missing pyelftools :) ).
Are there any other people struggling with this? Is there any more elegant approach that I haven't thought of?
pyelftools is available in EPEL 7, but it's built for python2. Looking at the spec file, it seems like it will build just fine for python3, so you might be able to munge it and rebuild on your system to have a python3-pyelftools RPM to install.
josh
Hi Josh,
On Fri, Apr 29, 2022 at 3:03 PM Josh Boyer jwboyer@redhat.com wrote:
On Fri, Apr 29, 2022 at 7:50 AM Stefan Puiu stefan.puiu@gmail.com wrote:
Hi,
I've already asked this on the dpdk-users list (https://mails.dpdk.org/archives/users/2022-April/006395.html) without getting any answer, so I thought maybe I'd also try centos-devel, since this is a CentOS and dpdk question.
We're using a CentOS 7 based distro that is obviously RPM-based, and we package the DPDK with it, in order to run on some of our x86-64 hardware boards. Now, we currently have DPDK 20.11 in our image and are looking to upgrade to 21.11, and I'm hitting a build error about elftools not being found:
Program python3 found: YES (/usr/bin/python3)
../buildtools/meson.build:49:8: ERROR: Problem encountered: missing python module: elftools
The fix seems to be to install pyelftools from pip:
pip3 install pyelftools
But that only works for building directly with meson / ninja. That doesn't work when building through rpmbuild. I've struggled to understand this, until I found this RedHat bug: https://bugzilla.redhat.com/show_bug.cgi?id=1937494. It seems Fedora changed the python3 search path so that /usr/local/lib/python is not taken into account if RPM_BUILD_ROOT is defined (see https://fedoraproject.org/wiki/Changes/Making_sudo_pip_safe); so basically, when building with rpmbuild, pip-installed packages are ignored.
I think I can understand the rationale of the Fedora change when you want to ensure that dependencies are properly set: packages installed using pip are not recorded as RPM dependencies, and the package might break at runtime. However, as far as I can tell, pyelftools seems to be a *build-time dependency*; removing the elftools check in the meson script leads to a build failure. There's no pyelftools RPM for CentOS 7 that I could find, so I don't see another option than installing the package from pip. Also, the python change in Fedora doesn't seem to allow for any workaround. The only thing that I can think of is ugly as hell: symlink everything under /usr/local/python in /usr/lib/python for the DPDK build. I'm not even sure we need pyelftools at runtime; in our build of DPDK-20.11, dpdk-pmdinfo.py doesn't even run (because of missing pyelftools :) ).
Are there any other people struggling with this? Is there any more elegant approach that I haven't thought of?
pyelftools is available in EPEL 7, but it's built for python2. Looking at the spec file, it seems like it will build just fine for python3, so you might be able to munge it and rebuild on your system to have a python3-pyelftools RPM to install.
Just to make sure I understand. I found the spec in the EPEL source: https://dl.fedoraproject.org/pub/epel/7/source/tree/Packages/p/pyelftools-0....
I see it has some conditionals to only enable the python3 part for RHEL >= 8. I guess you're suggesting for me to try to enable the python3 part also for RHEL 7?
Thanks, Stefan.
josh
CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
On Fri, Apr 29, 2022 at 3:03 PM Josh Boyer jwboyer@redhat.com wrote:
On Fri, Apr 29, 2022 at 7:50 AM Stefan Puiu stefan.puiu@gmail.com wrote:
Hi,
I've already asked this on the dpdk-users list (https://mails.dpdk.org/archives/users/2022-April/006395.html) without getting any answer, so I thought maybe I'd also try centos-devel, since this is a CentOS and dpdk question.
We're using a CentOS 7 based distro that is obviously RPM-based, and we package the DPDK with it, in order to run on some of our x86-64 hardware boards. Now, we currently have DPDK 20.11 in our image and are looking to upgrade to 21.11, and I'm hitting a build error about elftools not being found:
Program python3 found: YES (/usr/bin/python3)
../buildtools/meson.build:49:8: ERROR: Problem encountered: missing python module: elftools
The fix seems to be to install pyelftools from pip:
pip3 install pyelftools
But that only works for building directly with meson / ninja. That doesn't work when building through rpmbuild. I've struggled to understand this, until I found this RedHat bug: https://bugzilla.redhat.com/show_bug.cgi?id=1937494. It seems Fedora changed the python3 search path so that /usr/local/lib/python is not taken into account if RPM_BUILD_ROOT is defined (see https://fedoraproject.org/wiki/Changes/Making_sudo_pip_safe); so basically, when building with rpmbuild, pip-installed packages are ignored.
I think I can understand the rationale of the Fedora change when you want to ensure that dependencies are properly set: packages installed using pip are not recorded as RPM dependencies, and the package might break at runtime. However, as far as I can tell, pyelftools seems to be a *build-time dependency*; removing the elftools check in the meson script leads to a build failure. There's no pyelftools RPM for CentOS 7 that I could find, so I don't see another option than installing the package from pip. Also, the python change in Fedora doesn't seem to allow for any workaround. The only thing that I can think of is ugly as hell: symlink everything under /usr/local/python in /usr/lib/python for the DPDK build. I'm not even sure we need pyelftools at runtime; in our build of DPDK-20.11, dpdk-pmdinfo.py doesn't even run (because of missing pyelftools :) ).
Are there any other people struggling with this? Is there any more elegant approach that I haven't thought of?
pyelftools is available in EPEL 7, but it's built for python2. Looking at the spec file, it seems like it will build just fine for python3, so you might be able to munge it and rebuild on your system to have a python3-pyelftools RPM to install.
I tried modifying the pyelftools spec to build for python3 as well, but now it seems a dependency is missing:
stefan@spuiu-vm:~/rpmbuild$ rpmbuild -bb SPECS/pyelftools.spec error: Failed build dependencies: python3-construct is needed by pyelftools-0.22-0.5.git20130619.a1d9681.el7.x86_64
So maybe that's the reason why the python3 package is only enabled for rhel >= 8.
Thanks for the suggestion, Stefan.
josh
CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
-
Josh Boyer -
Stefan Puiu