Correctly handle libraries in subdirectories

[probonopd]

@M-Mueller writes in #216 (comment):

This is still an issue if a library as an RPATH. For example if you have the following structure:

bin/foo
lib/libfoo.so
lib/libfoo/libbar.so

libfoo.so has an RPATH of $ORIGIN/libfoo and links against libbar.so.

linuxdeployqt will first set the RPATH of libfoo.so to $ORIGIN and then complain that ldd cannot find libbar.so. I think the order in this case should be changed as well.

---

I had attempted to fix this by adding lib/libfoo.so to LD_LIBRARY_PATH in 9a93e03 but apparently it is not working. Any help appreciated.

[M-Mueller]

I tried this again with a minimal example and the above case actually works. However, our structures is actually a bit more complicated (I think QtCreator does it similarly):

bin/demo
lib/libspam.so
lib/spam/libfoo.so
lib/spam/libbar.so
lib/spam/libbaz.so

In this case, everything in lib/spam/ is consider a plugin for libspam.so and is loaded through dlopen during runtime, i.e. libspam.so does not link against anything in lib/spam/. The plugins also have direct dependencies on each other: libfoo.so links against libbar.so and libbar.so against libbaz.so. All plugins have a RPATH of $ORIGIN.

When linuxdeployqt encounters lib/spam/libfoo.so, it see lib/spam/libbar.so as a dependency and copies it to lib/libbar.so. Subsequently it tries to check the dependencies of lib/libbar.so, which fails since it cannot find libbaz.so anymore.

I honestly don't know what the best way of fixing this is. Even if the dependency resolution would work, libbar.so and libbaz.so would end up twice in the package, which is not that nice imo.

What do you think about another option, that would not copy dependencies that are already part of the package and would only prepend to the original RPATH instead of overriding it? So basically keeping the structure of the original package and only copy dependencies from outside.

[probonopd]

only prepend to the original RPATH instead of overriding it?

The original RPATH could be absolute (this is what I see most of the time) or relative (starting with $ORIGIN). I think whenever we see an RPATH that is already starting with $ORIGIN we can assume that someone already made the thing relocateable, and skip the RPATH rewriting for that file.

This leaves the question what to do with pre-existing absolute RPATHs. Not sure how this "prepend" should work but we should not get the situation where a library inside the AppImage leads the linker to look up nonstandard places outside of the AppImage, e.g. it should not start to try loading libraries from /usr/lib/spam/ on the host system if some happen to be there.

[M-Mueller]

(this is what I see most of the time)

Interesting, I would have expected it the other way around. Anyway, I would just remove any RPATH that does not start with $ORIGIN. An absolute RPATH doesn't make sense in a relocateable package. If it is really required, it can still be set after running linuxdeployqt.

I will try to create a pull request for this and see if this actually solves my problem.

[probonopd]

I mean, if there was an absolute RPATH to begin with, then we could try to rewrite it into a relative one. If we already find a relative one, then just leave it alone.

Does this make sense?

[M-Mueller]

Hm, I think this would only work for RPATHs starting with /usr. CMake, for example, sometimes sets an RPATH of /home/user/externals/caffe/1.0.0-rc3/lib if dependencies are outside the default locations. Since the libs get copied by linuxdeployqt, the RPATH is not needed any more in this case.

I think rewriting the absolute RPATH is too complicated and probably not worth the effort.

[probonopd]

Since in your case you are doing a very special thing (which you as the author are familiar with) it would probably be OK to just run linuxdeployqt -bundle-non-qt-apps ..., then manually run patchelf -set-rpath $ORIGIN/... as you see fit, and finally run linuxdeployqt -appimage .... The second linuxdeployqt run should not change the already-edited rpaths, since it skips rewriting rpaths when it encounters ones starting with $. Does this make sense?

[RolandHughes]

I used to use chrpath to change the rpath QtWebEngineProcess and other files. I don't know if patchelf has the same restriction (believe it should.) You cannot insert a longer RAPTH into a file than was originally built in. $ORIGIN didn't exist (or I didn't know about it) last time I wrote this stuff. Qt certainly didn't have any documentation on it, but now it has lots. Be careful with $ORIGIN. When it comes to "plugins," not always what you think. Depending on how the thing is launched it could be the plugin directory and not the /bin where the primary executable is. Relative paths get really horrible then.

Along these lines and the lines of:

#311
#34

I'm currently experimenting with porting a Qt Webengine multi-arch debian to use AppImage. Under Ubuntu Mate 18.04 32-bit still getting bit by the nss stuff, but, might have work around. The real killer is linuxdeployqt doesn't appear to have any multi-arch support.

I would like to suggest a few things to ease overall pain.

-include-lib-dir
for all versions of linuxdeployqt. Should accept a list of comma or colon separated directories which will be lifted as-is for the AppImage. The directory structure and content would be copied before dependancy checking starts. This would solve the /nss issue for many and might solve this particular bug as well since the files would already be in their correct locations - in theory.

As long as the land of Linux remains complete anarchy (yes, I skimmed through https://gitlab.com/probono/platformissues and don't expect the anarchy to end any time soon, they all want to be the profitable one) developers are going to need a method of forcing in directory structures along with additional files.

For the 32-bit version it would be nice to see -multi-arch but you may be limited via snapd itself there. What I envision for that feature is a complete packaging of 32-bit libraries such that the .AppImage can be dropped onto a 64-bit machine and just run. No need to install platform specific multi-arch libraries.

[M-Mueller]

You cannot insert a longer RAPTH into a file than was originally built in

Actually with patchelf you can, since it automatically adjusts the size of the executable.

Be careful with $ORIGIN. When it comes to "plugins," not always what you think. Depending on how the thing is launched it could be the plugin directory and not the /bin where the primary executable is.

$ORIGIN is always expanded to the directory containing the program or shared object. E.g. the $ORIGIN in the RPATH of a shared library will always be the directory of the shared library and never of the executable that loaded the library. However, there can be some other issues with RPATH vs RUNPATH (see here).

for all versions of linuxdeployqt. Should accept a list of comma or colon separated directories which will be lifted as-is for the AppImage. The directory structure and content would be copied before dependancy checking starts. This would solve the /nss issue for many and might solve this particular bug as well since the files would already be in their correct locations - in theory.

In this case you could move the affected folders out of the package into a temporary directory, call linuxdeployqt without the -appimage argument to collect all dependencies, move the affected folders back and call appimagetool to create the final AppImage.

[RolandHughes]

You cannot insert a longer RAPTH into a file than was originally built in

Actually with patchelf you can, since it automatically adjusts the size of the executable.

Ah! So it is much better than chrpath. That's the one I was thinking of.

Be careful with $ORIGIN. When it comes to "plugins," not always what you think. Depending on how the thing is launched it could be the plugin directory and not the /bin where the primary executable is.

$ORIGIN is always expanded to the directory containing the program or shared object. E.g. the $ORIGIN in the RPATH of a shared library will always be the directory of the shared library and never of the executable that loaded the library. However, there can be some other issues with RPATH vs RUNPATH (see here).

What you said is what I've believed for some time until I started working with QWebEngine. The plugins don't always behave in such manner. Perhaps some of it is due to items in the link you provided.

for all versions of linuxdeployqt. Should accept a list of comma or colon separated directories which will be lifted as-is for the AppImage. The directory structure and content would be copied before dependancy checking starts. This would solve the /nss issue for many and might solve this particular bug as well since the files would already be in their correct locations - in theory.

In this case you could move the affected folders out of the package into a temporary directory, call linuxdeployqt without the -appimage argument to collect all dependencies, move the affected folders back and call appimagetool to create the final AppImage.

One could, but, it would sure be nice to provide them in the command.

I actually was looking at (not trying, just thinking about) copying them into the tree under lib so it would be:
lib
lib/nss
lib/blah
before creating the image.

[probonopd]

Our philosophy is not to add additional switches like -include-lib-dir if possible. Adding any such switches only increases the likelihood of "clever" users to break things even more than they are already broken, running weird configurations, and working around issues in attempts to make one-off solutions rather than fixing things properly for everyone.

Instead, we should make linuxdeployqt intelligent enough to "just work" without the need of manual fiddling around. To facilitate this, we can consider to support only very specific build hosts, e.g., Ubuntu 14.04 using Qt from https://launchpad.net/~beineri - in an effort to reduce variants that all need to be tested and supported. That's anyhow what I am doing currently.