Find libaries in non-standard places by considering the original RPATH (e.g., libpulsecommon-9.0.so)

[my-tien]

One of my binary’s dependencies is libpulsecommon-9.0.so. ldd outputs correctly:

libpulsecommon-9.0.so => /usr/lib/pulseaudio/libpulsecommon-9.0.so (0x00007f10a344b000)

When I run linuxdeployqt the first time, libpulsecommon-9.0.so is present in the newly created /lib folder. These are the relevant log outputs:

[…]
ERROR: ldd outputLine: "\tlibpulsecommon-9.0.so => not found"
[…]
Log:  copied: "/usr/lib/pulseaudio/libpulsecommon-9.0.so"
Log:  to "./lib//libpulsecommon-9.0.so"
[…]

But for another reason I ran linuxdeployqt twice, the second time libpulsecommon-9.0.so is missing and the output now reads:

[…]
Log: Deploying libraries found inside: ("./AppRun", "./lib/libQt5Python27.so", "./lib/libpulsecommon-9.0.so", "./lib/libQt5Python27_QtAll.so", "./lib/libQt5Xml.so.5", "./lib/libXrender.so.1", "./lib/libXau.so.6", "./lib/libpcre16.so.0", "./lib/libQt5Quick.so.5", "./lib/libgstreamer-1.0.so.0", "./lib/libsqlite3.so.0", "./lib/libglapi.so.0", "./lib/libffi.so.6", "./lib/libvorbis.so.0", "./lib/libgraphite2.so.3", "./lib/libgstapp-1.0.so.0", "./lib/libQt5OpenGL.so.5", "./lib/libsndfile.so.1", "./lib/libQt5Widgets.so.5", "./lib/libwebp.so.6", "./lib/libQt5PrintSupport.so.5", "./lib/libxcb-glx.so.0", "./lib/libsnappy.so.1", "./lib/libcap.so.2", "./lib/libjson-c.so.2", "./lib/libQt5Qml.so.5", "./lib/libQt5Sensors.so.5", "./lib/libutil.so.1", "./lib/libQt5XmlPatterns.so.5", "./lib/libQt5Help.so.5", "./lib/libharfbuzz.so.0", "./lib/libsystemd.so.0", "./lib/libQt5Gui.so.5", "./lib/libxml2.so.2", "./lib/libxcb-sync.so.1", "./lib/libicudata.so.57", "./lib/libQt5MultimediaWidgets.so.5", "./lib/libpulse.so.0", "./lib/libXdmcp.so.6", "./lib/libgcrypt.so.20", "./lib/libXdamage.so.1", "./lib/libquazip5.so.1", "./lib/libQt5Network.so.5", "./lib/libXfixes.so.3", "./lib/libXext.so.6", "./lib/libjpeg.so.8", "./lib/libQt5Svg.so.5", "./lib/libcrypto.so.1.0.0", "./lib/libQt5Python27.so", "./lib/libasyncns.so.0", "./lib/liborc-0.4.so.0", "./lib/libproxy.so.1", "./lib/libgsttag-1.0.so.0", "./lib/libbz2.so.1.0", "./lib/libgmodule-2.0.so.0", "./lib/libQt5Concurrent.so.5", "./lib/libQt5Multimedia.so.5", "./lib/libogg.so.0", "./lib/liblz4.so.1", "./lib/libgstbase-1.0.so.0", "./lib/libFLAC.so.8", "./lib/libpulsecommon-9.0.so", "./lib/libQt5Sql.so.5", "./lib/libgstvideo-1.0.so.0", "./lib/libxcb-dri3.so.0", "./lib/libX11-xcb.so.1", "./lib/libicuuc.so.57", "./lib/libgstpbutils-1.0.so.0", "./lib/libdbus-1.so.3", "./lib/libGLU.so.1", "./lib/libQt5Python27_QtAll.so", "./lib/libQt5CLucene.so.5", "./lib/libXcomposite.so.1", "./lib/libxcb-present.so.0", "./lib/libpython2.7.so.1.0", "./lib/libQt5WebChannel.so.5", "./lib/libxcb-dri2.so.0", "./lib/libXxf86vm.so.1", "./lib/libvorbisenc.so.2", "./lib/libxshmfence.so.1", "./lib/libQt5WebKitWidgets.so.5", "./lib/libfreetype.so.6", "./lib/libicui18n.so.57", "./lib/libpng16.so.16", "./lib/liblzma.so.5", "./lib/libQt5WebKit.so.5", "./lib/libQt5Positioning.so.5", "./lib/libssl.so.1.0.0", "./lib/libQt5Core.so.5", "./lib/libgstaudio-1.0.so.0", "./lib/libxslt.so.1")
[…]
ERROR: findDependencyInfo: ""
[…]
Log:  copied: "/usr/lib/pulseaudio/libpulsecommon-9.0.so"
Log:  to "./lib//libpulsecommon-9.0.so"
[…]
ERROR: file copy failed from "/path/to/linuxdeployqt_test/./lib/libpulsecommon-9.0.so"
ERROR:  to "./lib//libpulsecommon-9.0.so"
[…]

I noticed that libpulsecommon-9.0.so appears twice in the list of libraries found in ./AppRun.

[probonopd]

Are you saying that running linuxdeployqt for a second time deletes the library from the created lib/ directory? Now that would be strange...

[my-tien]

Yes, I noticed it and I believe these lines indicate just that:

ERROR: file copy failed from "/path/to/linuxdeployqt_test/./lib/libpulsecommon-9.0.so"
ERROR:  to "./lib//libpulsecommon-9.0.so"

But I have to mention that from my observations different things can happen even though I start out with a fresh folder. When I tried to reproduce this issue today, I ended up with results reverse to the ones described above (so first run no libpulsecommon, second run it appears) and on a third fresh try it fails to find libpulsecommon altogether.

[ScarlettGatelyMoore]

I am also having issues with libpulsecommon. It is very inconsistant.... sometimes it finds it sometimes it does not.

http://aci.pangea.pub/job/mpipeline-blinken-appimage/job/master/53/parsed_console/

I have it set on verbose.

[probonopd]

@ScarlettGatelyClark @my-tien do you still encounter this?
Could it be that the location it is trying to copy from is a symlink?

[ThorstenBux]

If it is of any help, I can verify that this still occurs with the CI build from commit: c21b171

[probonopd]

@ThorstenBux just to verify, if you run linuxdeployqt once, then libpulsecommon is bundled inside the AppDir, and when you run it twice, it is no longer there?

[ThorstenBux]

No actually the other way round.
When I run it the first time I get the response:
ERROR: ldd outputLine: "\tlibpulsecommon-8.0.so => not found"

When I run it the second time there is no error at all and the resulting .appImage works.

(When trying to run the first resulting .appImage it doesn't start at all)

[probonopd]

OK thanks for the clarification @ThorstenBux. Yes, for now it is recommended to run linuxdeployqt twice.

[DKMudrechenko]

Encountered the same problem when packaging my application, running twice did not help. It was solved using suggestion from #105, namely export LD_LIBRARY_PATH=/path-to-libpulse-lib/:$LD_LIBRARY_PATH and then running linuxdeployqt

[probonopd]

I think the proper thing would be to check every library for pre-existing rpaths to subdirectories of /lib or /usr, and rewrite them to relative rpaths instead of just replacing them. Pull requests in this direction would be highly welcome.

[tresf]

I think the proper thing would be to check every library for pre-existing rpaths to subdirectories of /lib or /usr, and rewrite them to relative rpaths instead of just replacing them.

I'm running into a similar issue with Ubuntu 14.04. Running the tool again doe not correct the problem for me either.

• First, linuxdeployqt successfully copies libpulse-simple.so.0 from /usr/lib/x86_64-linux-gnu/ to <binary-directory>/../lib
• At some point, it errors with ERROR: ldd outputLine: "libpulsecommon-4.0.so => not found"

However, when I ldd the OS version of libpulse-simple.so.0, it does not immediately appear to use rpaths. Researching this issue shows ldd may hide this information.

libpulsecommon-4.0.so => /usr/lib/x86_64-linux-gnu/pulseaudio/libpulsecommon-4.0.so (0x00007f75c85b1000)

And now I ldd the copied version residing in <binary-directory>/../lib I'm seeing a duplicate entry.

- libpulsecommon-4.0.so => not found
  libpulsecommon-4.0.so => /usr/lib/x86_64-linux-gnu/pulseaudio/libpulsecommon-4.0.so (0x00007f4a48bd1000)

I'm going to attempt @DKMudrechenko's export LD_LIBRARY_PATH recommendation as a stop-gap, but is there a better CLI method to help troubleshoot rpaths?

[probonopd]

The question is why (= by what mechanism) libpulsecommon-4.0.so gets found on the original system inside /usr/lib/x86_64-linux-gnu/pulseaudio/ in the first place. I have to admit I don't fully understand how this works, i.e., why a library is even being searched for in this directory. Normally libraries reside (and are searched for) in places like /usr/lib/x86_64-linux-gnu/ but not subdirectories thereof. PulseAudio seems to be one of a few notable exceptions.

Any insights welcome.

[tresf]

why (= by what mechanism) libpulsecommon-4.0.so gets found on the original system inside /usr/lib/x86_64-linux-gnu/pulseaudio/ in the first place.

Are you sure it does? If it's never referenced directly, can't it just rely on the relationship inside libpulse-simple.so, which clearly points to the correct location?

[AlfredoCubitos]

the problem is that the libraries in a subdirectory of /usr/lib64 are not found by default calling /lib/ld-linux.so.2 which is used by ldd
The solution is to put this path into your /etc/ld.so.conf file
The best way to do this, is to call ldconfig <path to the library> as root

[probonopd]

@AlfredoCubitos on the build system, ldd does actually find such libraries prior to running linuxdeployqt, presumably because the binaries/libraries have an embedded RPATH set and/or are using ldopen().

So, as the title implies, linuxdeployqt should consider the original RPATH when rewriting it. Then you will not need to modify files in /etc as root.

[AlfredoCubitos]

hm, in my case ldd shows libpulsecommon-9.0.so twice.
the first one is not found
and the second one with the correct path
linuxdeployqt exits with an error after the first time
RPATH from the original library shows $ORIGIN/../../lib

So I assume the linker searches in the the standard library path and where are the libraries located defined in /etc/ld.so.conf. If the library is not found, a recursion searches in the subdirs .
During run time it seems that this doesn't matter because finally the library is found, but linuxdeployqt exit when it gets not found.

Of course, when you can fix this, modifying /etc/ld.so.conf isn't necessary ;-)
But for now it solves the problem.

There a lot discussions about this issue see here

[probonopd]

The problem is that libpulse* libraries have RPATH set by default, pointing to a subdirectory of the normal libraries directory. For example, on openSUSE systems, this is /usr/lib64/pulseaudio but on other distributions it is a different directory.

linux@linux:~> find /usr/ -name "libpulse*.so*" -exec patchelf --print-rpath {}  \;

/usr/lib64/pulseaudio
/usr/lib64/pulseaudio
/usr/lib64/pulseaudio
/usr/lib64/pulseaudio
/usr/lib64/pulseaudio
/usr/lib64/pulseaudio
/usr/lib64/pulseaudio
/usr/lib64/pulseaudio

(When deploying libraries to the AppDir, linuxdeployqt removes such pre-existing RPATHs and replaces them with relative ones.)

As a workaround, you can export LD_LIBRARY_PATH=/usr/lib64/pulseaudio (or your distribution's equivalent) before running linuxdeployqt, but a proper solution needs to be implemented.

[probonopd]

Closed in 9a93e03