I'm having a major frustration with curl.
When building curl, if libssl.so.10 is present the curl binary WILL link against it.
If curl is configured with an ssl option - the library WILL link against it.
If you change the curl configuration options to use a different TLS library (e.g. nss like CentOS does) the curl binary and library will still link against the OpenSSL library.
There's definitely something funny about curl's ./configure -
If you disable features but they are still pulled in by mock as dependencies for the build environment, the curl library will respect your configure options and won't link against those features (except it will for libssl.so.10 if ANY tls option is chosen) but the binary will link against the libraries if it is there. EVEN IF THE DEVEL PACKAGE WITH HEADER FILES IS NOT PRESENT.
There is something very broken about how curl builds. If I was a skilled blackhat, I might look for ways that causes it to be exploitable, because the building of curl doesn't do what the user expects.
I tried building curl creating a mock build environment where openssl is forbidden. There's a bug in mock.
In both base and updates I have
exclude=openssl*
I had to rebuild many packages against LibreSSL to get that to work.
That btw is what I'm trying to do with curl - build it against LibreSSL and it does, but also links against libssl.so.10 and there is the problem - it's not safe to have a library (or binary) that links against both OpenSSL and LibreSSL.
With the presence of those excludes - mock does prevent the installation of openssl packages *in some cases* but it allows it others.
rpm depends upon curl and curl from the CentOS packages depends upon libssl.so.10 and mock pulls in rpm and thus pulls in curl and thus pulls in openssl-libs and so if building curl in mock - it will link against openssl.
I went through everything in the mock buildroot with ldd and curl is the ONLY package installed that has anything linked against openssl.
I tried building an intermediate curl for mock to pull in without any TLS capabilities - it works for the library but the curl binary still links against openssl.
I tried building an intermediary RPM package that doesn't require curl - but something else in the build system is pulling in curl resulting in libssl.so.10 being installed.
I wish mock didn't have this bug as if it actually respected the excludes on base and updates, it would tell me what packages are pulling in openssl-libs but unfortunately there are cases where the excludes are not respected.
This is really frustrating.
I tried looking through the curl buildsystem to see if I could patch that but it seems messy to me and I can't find why the binary links against libraries you disable with configure and I can't see why the library always links against openssl if any TLS is chosen.
It's very frustrating.
-=- No other package I've rebuilt against LibreSSL has this problem.
With curl its a big problem.
It definitely should not be linking against libraries it doesn't even have the right headers for.
On 08/03/2016 05:11 PM, Alice Wonder wrote:
I'm having a major frustration with curl.
When building curl, if libssl.so.10 is present the curl binary WILL link against it.
*snip*
Go ahead and ldd on the CentOS curl binary and library - you will see openssl linked even though the spec file has --disable-ssl and --enable-nss
It's clearly broken.
On 08/03/2016 05:20 PM, Alice Wonder wrote:
On 08/03/2016 05:11 PM, Alice Wonder wrote:
I'm having a major frustration with curl.
When building curl, if libssl.so.10 is present the curl binary WILL link against it.
*snip*
Go ahead and ldd on the CentOS curl binary and library - you will see openssl linked even though the spec file has --disable-ssl and --enable-nss
It's clearly broken.
And building the CentOS curl package doesn't even BuildRequires the openssl-devel package.
It's linking against a library it doesn't have the headers for.
That's broken.
On 08/03/2016 05:23 PM, Alice Wonder wrote:
On 08/03/2016 05:20 PM, Alice Wonder wrote:
On 08/03/2016 05:11 PM, Alice Wonder wrote:
I'm having a major frustration with curl.
When building curl, if libssl.so.10 is present the curl binary WILL link against it.
*snip*
Go ahead and ldd on the CentOS curl binary and library - you will see openssl linked even though the spec file has --disable-ssl and --enable-nss
It's clearly broken.
And building the CentOS curl package doesn't even BuildRequires the openssl-devel package.
It's linking against a library it doesn't have the headers for.
That's broken.
I haven't looked at how curl is built, butit is likely that the build links against some other package that is, in turn, built against OpenSSL.
You would not need the openssl-devel package to do that, only the runtime libraries.
It looks like that package could be libssh2...
T.
Once upon a time, Thomas Eriksson thomas.eriksson@slac.stanford.edu said:
I haven't looked at how curl is built, butit is likely that the build links against some other package that is, in turn, built against OpenSSL.
You would not need the openssl-devel package to do that, only the runtime libraries.
It looks like that package could be libssh2...
Yes, that's what it is. libcurl links libssh2 which links libssl.
There's nothing broken about mock pulling in mandatory requirements. If you don't want curl to use libssl, you'll need to disable SSH support.
ldd does not just show the direct dependencies of the file you run it against; it resolves all deps needed to load the file.
On 08/03/2016 05:33 PM, Thomas Eriksson wrote:
On 08/03/2016 05:23 PM, Alice Wonder wrote:
On 08/03/2016 05:20 PM, Alice Wonder wrote:
On 08/03/2016 05:11 PM, Alice Wonder wrote:
I'm having a major frustration with curl.
When building curl, if libssl.so.10 is present the curl binary WILL link against it.
*snip*
Go ahead and ldd on the CentOS curl binary and library - you will see openssl linked even though the spec file has --disable-ssl and --enable-nss
It's clearly broken.
And building the CentOS curl package doesn't even BuildRequires the openssl-devel package.
It's linking against a library it doesn't have the headers for.
That's broken.
I haven't looked at how curl is built, butit is likely that the build links against some other package that is, in turn, built against OpenSSL.
You would not need the openssl-devel package to do that, only the runtime libraries.
It looks like that package could be libssh2...
It's not libssh2 because I built libssh2 against LibreSSL and tested it with ldd and it doesn't use OpenSSL nor pull it in.
And in trying to create a curl that doesn't link against anything TLS I put --disable-libssh2 into the configure.
The curl library respects that configure switch when building, the curl binary does not - it links against it anyway (mock pulls it in for other things, pulling in the version I built against OpenSSL)
I also built custom OpenSSH against LibreSSL (which required ripping out all the fips stuff) for the mock build too - it also isn't pulling in OpenSSL libs.
Something in the curl build will always link the binary against OpenSSL if the openssl-lib package is present, and will always link the library against OpenSSL if any TLS option is enabled in the configure.
This happens even openssl-devel is not installed in the mock build environment.
On 08/03/2016 05:45 PM, Alice Wonder wrote:
On 08/03/2016 05:33 PM, Thomas Eriksson wrote:
On 08/03/2016 05:23 PM, Alice Wonder wrote:
On 08/03/2016 05:20 PM, Alice Wonder wrote:
On 08/03/2016 05:11 PM, Alice Wonder wrote:
I'm having a major frustration with curl.
When building curl, if libssl.so.10 is present the curl binary WILL link against it.
*snip*
Go ahead and ldd on the CentOS curl binary and library - you will see openssl linked even though the spec file has --disable-ssl and --enable-nss
It's clearly broken.
And building the CentOS curl package doesn't even BuildRequires the openssl-devel package.
It's linking against a library it doesn't have the headers for.
That's broken.
I haven't looked at how curl is built, butit is likely that the build links against some other package that is, in turn, built against OpenSSL.
You would not need the openssl-devel package to do that, only the runtime libraries.
It looks like that package could be libssh2...
It's not libssh2 because I built libssh2 against LibreSSL and tested it with ldd and it doesn't use OpenSSL nor pull it in.
*snip*
It might be getting -lcrypto from libssh2 but then it should link against LibreSSL because that's what the libssh2 for the build environment is linked against.
I suspect it might do the right thing and link against libressl if the openssl lib wasn't present.
But despite the mock excludes, it gets pulled in by something.
The excludes are in both base and updates. And work if trying to build a package that directly wants openssl or has a dependency that directly wants openssl.
It seems the excludes fail when it is a dependency of a dependency.
On 08/03/2016 05:54 PM, Alice Wonder wrote:
On 08/03/2016 05:45 PM, Alice Wonder wrote:
On 08/03/2016 05:33 PM, Thomas Eriksson wrote:
On 08/03/2016 05:23 PM, Alice Wonder wrote:
On 08/03/2016 05:20 PM, Alice Wonder wrote:
On 08/03/2016 05:11 PM, Alice Wonder wrote:
I'm having a major frustration with curl.
When building curl, if libssl.so.10 is present the curl binary WILL link against it.
*snip*
Go ahead and ldd on the CentOS curl binary and library - you will see openssl linked even though the spec file has --disable-ssl and --enable-nss
It's clearly broken.
And building the CentOS curl package doesn't even BuildRequires the openssl-devel package.
It's linking against a library it doesn't have the headers for.
That's broken.
I haven't looked at how curl is built, butit is likely that the build links against some other package that is, in turn, built against OpenSSL.
You would not need the openssl-devel package to do that, only the runtime libraries.
It looks like that package could be libssh2...
It's not libssh2 because I built libssh2 against LibreSSL and tested it with ldd and it doesn't use OpenSSL nor pull it in.
*snip*
*snip*
Take the CentOS curl src.rpm package and add exit 1 to the beginning of %check so that it exits quickly w/o removing the BUILDROOT in mock.
Start changing the configure switches disabling stuff and look at the library and the curl binary with ldd.
You'll see the configure switches reduce what the library links against but not what the binary links against. It's configure is clearly broken.
I tried disabling all the RHEL patches but the behavior is the same, it's not caused by a patch.
On 08/03/2016 06:05 PM, Alice Wonder wrote:
On 08/03/2016 05:54 PM, Alice Wonder wrote:
On 08/03/2016 05:45 PM, Alice Wonder wrote:
On 08/03/2016 05:33 PM, Thomas Eriksson wrote:
On 08/03/2016 05:23 PM, Alice Wonder wrote:
On 08/03/2016 05:20 PM, Alice Wonder wrote:
On 08/03/2016 05:11 PM, Alice Wonder wrote: > I'm having a major frustration with curl. > > When building curl, if libssl.so.10 is present the curl binary WILL > link > against it.
*snip*
Go ahead and ldd on the CentOS curl binary and library - you will see openssl linked even though the spec file has --disable-ssl and --enable-nss
It's clearly broken.
And building the CentOS curl package doesn't even BuildRequires the openssl-devel package.
It's linking against a library it doesn't have the headers for.
That's broken.
I haven't looked at how curl is built, butit is likely that the build links against some other package that is, in turn, built against OpenSSL.
You would not need the openssl-devel package to do that, only the runtime libraries.
It looks like that package could be libssh2...
It's not libssh2 because I built libssh2 against LibreSSL and tested it with ldd and it doesn't use OpenSSL nor pull it in.
*snip*
*snip*
*snip*
One thing left to try - a race condition - start building an intermediate curl and remove the libssl.so.10 from the buildroot with a shell after mock creates the builroot but before curl is building.
If cure then successfully builds, I should have an intermediary curl binary I can put in my mock sources to build the real curl so that libssl.so.10 isn't pulled in.
I don't like doing that.
I do suspect it is a flaw in how the curl build systems handles -lcrypto
Once upon a time, Alice Wonder alice@domblogger.net said:
Something in the curl build will always link the binary against OpenSSL if the openssl-lib package is present, and will always link the library against OpenSSL if any TLS option is enabled in the configure.
No, it doesn't. You can see this by looking at the RPM dependencies; at least on CentOS 7 and Fedora 23, they don't depend on libssl.so at all. If either curl or libcurl.so were linked against libssl.so, the RPM would require it.
I just tried building the CentOS 7 RPM of curl, changing the SPEC to remove the libssh2 settings, and I got a curl and libcurl that do not in any way depend on libssl. It is most definitely libssh2.so that requires libssl.so.
If you want help, you need to stop repeating your claims and show some actual commands run, output, versions, etc.
On 08/03/2016 06:13 PM, Chris Adams wrote:
Once upon a time, Alice Wonder alice@domblogger.net said:
Something in the curl build will always link the binary against OpenSSL if the openssl-lib package is present, and will always link the library against OpenSSL if any TLS option is enabled in the configure.
No, it doesn't. You can see this by looking at the RPM dependencies; at least on CentOS 7 and Fedora 23, they don't depend on libssl.so at all. If either curl or libcurl.so were linked against libssl.so, the RPM would require it.
I just tried building the CentOS 7 RPM of curl, changing the SPEC to remove the libssh2 settings, and I got a curl and libcurl that do not in any way depend on libssl. It is most definitely libssh2.so that requires libssl.so.
If you want help, you need to stop repeating your claims and show some actual commands run, output, versions, etc.
[alice@pern ~]$ ldd /usr/bin/curl |grep crypto libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f4524390000) libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007f45234ca000) [alice@pern ~]$ rpm -qf /lib64/libcrypto.so.10 openssl-libs-1.0.1e-51.el7_2.5.x86_64 [alice@pern ~]$
That's with stock CentOS packages
On 08/03/2016 06:17 PM, Alice Wonder wrote: *snip*
[alice@pern ~]$ ldd /usr/bin/curl |grep crypto libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f4524390000) libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007f45234ca000) [alice@pern ~]$ rpm -qf /lib64/libcrypto.so.10 openssl-libs-1.0.1e-51.el7_2.5.x86_64 [alice@pern ~]$
That's with stock CentOS packages
That is also what I get when building curl in mock with
exlude=openssl*
and test the built binary with ldd - even with libssh and openssh built against LibreSSL (which uses libcrypto.so.37 instead of .10)
Once upon a time, Alice Wonder alice@domblogger.net said:
[alice@pern ~]$ ldd /usr/bin/curl |grep crypto libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f4524390000) libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007f45234ca000)
What do the following show:
ldd /usr/bin/curl ldd /usr/lib64/libcurl.so.4 ldd /usr/lib64/libssh2.so.1
I can only guess you don't have the libssh2 you think you have. I just replaced curl/libcurl on a CentOS 7 system with my version built without ssh2 support:
$ ldd /usr/bin/curl linux-vdso.so.1 => (0x00007fff4b9fc000) libcurl.so.4 => /lib64/libcurl.so.4 (0x00007f001623f000) libssl3.so => /lib64/libssl3.so (0x00007f0015ffd000) libsmime3.so => /lib64/libsmime3.so (0x00007f0015dd5000) libnss3.so => /lib64/libnss3.so (0x00007f0015aaf000) libnssutil3.so => /lib64/libnssutil3.so (0x00007f0015883000) libplds4.so => /lib64/libplds4.so (0x00007f001567e000) libplc4.so => /lib64/libplc4.so (0x00007f0015479000) libnspr4.so => /lib64/libnspr4.so (0x00007f001523b000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f001501e000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f0014e1a000) libz.so.1 => /lib64/libz.so.1 (0x00007f0014c04000) libc.so.6 => /lib64/libc.so.6 (0x00007f0014842000) libidn.so.11 => /lib64/libidn.so.11 (0x00007f001460f000) libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f00143c3000) libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f00140dd000) libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f0013eab000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f0013ca7000) liblber-2.4.so.2 => /lib64/liblber-2.4.so.2 (0x00007f0013a97000) libldap-2.4.so.2 => /lib64/libldap-2.4.so.2 (0x00007f0013844000) librt.so.1 => /lib64/librt.so.1 (0x00007f001363b000) /lib64/ld-linux-x86-64.so.2 (0x00007f00164a6000) libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f001342c000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f0013228000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f001300d000) libsasl2.so.3 => /lib64/libsasl2.so.3 (0x00007f0012df0000) libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f0012bca000) libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f0012993000) libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f0012732000) liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f001250c000) libfreebl3.so => /lib64/libfreebl3.so (0x00007f0012309000)
No libssl.so, no libcrypto.so.
On 08/03/2016 06:30 PM, Chris Adams wrote:
Once upon a time, Alice Wonder alice@domblogger.net said:
[alice@pern ~]$ ldd /usr/bin/curl |grep crypto libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f4524390000) libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007f45234ca000)
What do the following show:
ldd /usr/bin/curl ldd /usr/lib64/libcurl.so.4 ldd /usr/lib64/libssh2.so.1
I can only guess you don't have the libssh2 you think you have.
It's worse than I thought.
I just very carefully went through the mock build root.
openssl-libs is indeed not even installed.
There is no libcrypto.so.10 installed in the mock build root.
But the binary built by mock still links to it.
So it is linking against a library not even in the mock build root.
That shouldn't happen.
[alice@pern root]$ pwd /var/lib/mock/awel-7-curl-x86_64/root
(to show I'm in the build root - I put exit 1 in %check)
[alice@pern root]$ ls usr/lib64/ |grep crypto libcrypto.so.38 libcrypto.so.38.0.0 libk5crypto.so.3 libk5crypto.so.3.1
[alice@pern root]$ ldd builddir/build/BUILDROOT/curl-7.29.0-26.el7_2.awel.libre.0.x86_64/usr/bin/curl |grep crypto libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007fb6e00a5000) libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007fb6df3e8000) [alice@pern root]$
Once upon a time, Alice Wonder alice@domblogger.net said:
[alice@pern root]$ ldd builddir/build/BUILDROOT/curl-7.29.0-26.el7_2.awel.libre.0.x86_64/usr/bin/curl |grep crypto libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007fb6e00a5000) libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007fb6df3e8000)
Here is your problem - unless you specify an LD_LIBRARY_PATH, you are getting the mock root installed libcurl, not your newly-built libcurl.
On 08/03/2016 06:40 PM, Chris Adams wrote:
Once upon a time, Alice Wonder alice@domblogger.net said:
[alice@pern root]$ ldd builddir/build/BUILDROOT/curl-7.29.0-26.el7_2.awel.libre.0.x86_64/usr/bin/curl |grep crypto libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007fb6e00a5000) libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007fb6df3e8000)
Here is your problem - unless you specify an LD_LIBRARY_PATH, you are getting the mock root installed libcurl, not your newly-built libcurl.
So when building curl, it links curl against the libcurl in the buildroot and not against the libcurl it just compiled?
No other packages I know of do that.
I've built a love of packages against LibreSSL (I really like it) and ldd on the resulting binaries and libraries *always* point to the LibreSSL library - even when openssl-libs is installed in the buildroot.
It seems to me to be a bug if curl requires a bootstrap build in order to change what libraries the binary links against but I will try that.
Once upon a time, Alice Wonder alice@domblogger.net said:
So when building curl, it links curl against the libcurl in the buildroot and not against the libcurl it just compiled?
No other packages I know of do that.
No, that is not what it does. If you posted the full ldd output like I asked, rather than grepping out bits, you'd see it loading the system libcurl.
When you run ldd against an arbitrary binary, ldd uses the system directories to resolve dependencies. The libcurl you just built is not in any of those directories, but the system libcurl (that is linked against libssh2, which pulls in libssl/libcrypto) is. The dynamic linker uses the library it can find.
If you want to use a different library, you have to specify the path. If you look at curl.spec, you will see it set LD_LIBRARY_PATH before running tests, so that it gets its newly-built libcurl dependency.
There is absolutely nothing magic or broken in curl, its build setup, or mock.
On 08/03/2016 06:57 PM, Chris Adams wrote:
Once upon a time, Alice Wonder alice@domblogger.net said:
So when building curl, it links curl against the libcurl in the buildroot and not against the libcurl it just compiled?
No other packages I know of do that.
No, that is not what it does. If you posted the full ldd output like I asked, rather than grepping out bits, you'd see it loading the system libcurl.
When you run ldd against an arbitrary binary, ldd uses the system directories to resolve dependencies. The libcurl you just built is not in any of those directories, but the system libcurl (that is linked against libssh2, which pulls in libssl/libcrypto) is. The dynamic linker uses the library it can find.
If you want to use a different library, you have to specify the path. If you look at curl.spec, you will see it set LD_LIBRARY_PATH before running tests, so that it gets its newly-built libcurl dependency.
There is absolutely nothing magic or broken in curl, its build setup, or mock.
Okay I think I got it.
You are saying that ldd is recursive and since curl links against libcurl running ldd on the curl binary also results in the libraries that it libcurl on the system is linked against.
I didn't realize ldd was recursive. I may have known that at one point (been using linux since MK Linux DR3 and building RPMs since 1999), but have a head injury results in memory problems with pieces of knowledge I don't frequently use.
Thank you, that makes sense.
On 08/03/2016 07:53 PM, Alice Wonder wrote:
On 08/03/2016 06:57 PM, Chris Adams wrote:
Once upon a time, Alice Wonder alice@domblogger.net said:
So when building curl, it links curl against the libcurl in the buildroot and not against the libcurl it just compiled?
No other packages I know of do that.
No, that is not what it does. If you posted the full ldd output like I asked, rather than grepping out bits, you'd see it loading the system libcurl.
When you run ldd against an arbitrary binary, ldd uses the system directories to resolve dependencies. The libcurl you just built is not in any of those directories, but the system libcurl (that is linked against libssh2, which pulls in libssl/libcrypto) is. The dynamic linker uses the library it can find.
If you want to use a different library, you have to specify the path. If you look at curl.spec, you will see it set LD_LIBRARY_PATH before running tests, so that it gets its newly-built libcurl dependency.
There is absolutely nothing magic or broken in curl, its build setup, or mock.
Okay I think I got it.
You are saying that ldd is recursive and since curl links against libcurl running ldd on the curl binary also results in the libraries that it libcurl on the system is linked against.
I didn't realize ldd was recursive. I may have known that at one point (been using linux since MK Linux DR3 and building RPMs since 1999), but have a head injury results in memory problems with pieces of knowledge I don't frequently use.
Thank you, that makes sense.
From google search, it looks like not all versions of ldd are recursive.
The symptoms of this problem though suddenly make sense if it is (ldd in buildroot on curl binary different from library)
-
Alice Wonder -
Chris Adams -
Thomas Eriksson