Shell with tools in appimage

[codan]

Hello all and @probono! Thank you for your answers in these threads! It helps to understand AppImage concepts!

Could you please advise me in my case. I’m trying to create portable stand alone shell along with couple of tools. It consists of:

1. zsh shell — in general it requires home directory with .zshrc file with settings
2. OMZ themes — in general it’s just a directory in home dir and .zshrc contains source and settings of the files
3. python 3 with packages — installation via conda for example as described in AppImage/Bundling-Python-apps
4. autojump — python based tool for zsh — in general it’s just a directory in home dir and .zshrc contains source and settings of the files AND it can write cache to ~/.local

I already made first step and tried to create zsh AppImage via linuxdeploy/linuxdeploy-plugin-conda. It was executed perfect and zsh was running but I faced with issues:

1] When I for experiment delete AppDir and try to use zsh AppImage it show the error:

zsh: failed to load module zsh/zle: /tmp/test/AppDir/usr/conda/lib/zsh/5.6.2/zsh/zle.so: cannot open shared object file: No such file or directory

2] If I for experiment didn’t delete AppDir then zsh worked and I tried to include OMZ in .zshrc and I’ve faced with the errors:

$ mkdir /tmp/test/shell_home
$ git clone https://github.com/robbyrussell/oh-my-zsh/
$ echo '...connect OMZ...' > /tmp/test/shell_home/.zshrc
$ export ZDOTDIR=/tmp/test/shell_home 
$ ./zsh-x86_64.AppImage
/tmp/test/shell_home/oh-my-zsh/oh-my-zsh.sh:72: compinit: function definition file not found
/tmp/test/shell_home/oh-my-zsh/lib/completion.zsh:23: command not found: zstyle
/tmp/test/shell_home/oh-my-zsh/lib/key-bindings.zsh:20: command not found: bindkey
/tmp/test/shell_home/oh-my-zsh/plugins/git/git.plugin.zsh:119: command not found: compdef

It looks like OMZ expects zsh in the current env and probably should be packed with zsh in the AppImage.

So I’m here and want to ask advice and start from basic questions to understand:

1. Can I pack all 1-4 items from my list in one AppImage and use from shell which run as AppImage OR I should create separate AppImages for zsh, python and autojump and run it together thru configs and scripting?

2. What is the best way to pack shell (zsh) into AppImage? If it helps: my experiments shows that zsh has absolute paths for libs inside zsh executable and should be rebuilded from source with appropriate PREFIX for non standard directory.

3. For python I’ve read Bundling-Python-apps and choose conda. Any advice on this with my case?

4. OMZ and autojump — it’s external scripts but they expect environment around. Should I pack them inside AppImage or for example I can place it to AppImage.home and write entrypoint script which will fix paths and then run zsh?

What use case I expect at the end of story: I have portable shell with tools — it’s one or couple of AppImages with scripting files around which I can place on any OS which supported AppImage and run without installation.

Thanks for answers!

[probono]

Hello @codan, welcome to AppImage.

What you are trying to bundle is much more than what we normally call an “application” - it is more a set of semi-related tools and components. As a rough guide, an .AppImage is supposed to do what a single-file (“portable”) .exe would do on Windows and, more closely, what an .app would do on the Mac.

That being said, with some patience and debugging of the AppDir and trial and error it should be possible to do what you are intending.

Some notes:

@codan

zsh: failed to load module zsh/zle: /tmp/test/AppDir/usr/conda/lib/zsh/5.6.2/zsh/zle.so: cannot open shared object file: No such file or directory

This means that zsh is trying to load zle.so from inside the AppImage (which is good - it would be worse if it was trying to load it from /usr in the system). Why does it fail, though? Does the file exist at /tmp/test/AppDir/usr/conda/lib/zsh/5.6.2/zsh/zle.so? What happens if you run ldd /tmp/test/AppDir/usr/conda/lib/zsh/5.6.2/zsh/zle.so?

OMZ themes — in general it’s just a directory in home dir and .zshrc contains source and settings of the files

Since what is inside the AppImage does not reside in the locations it is probably expected to be by the application in ($HOME and it subdirectories), you must make sure that the code inside the AppDir (AppRun or something called by it) will copy these files into $HOME and it subdirectories before the main payload application gets launched.

Same for

@codan

autojump — python based tool for zsh — in general it’s just a directory in home dir and .zshrc contains source and settings of the files AND it can write cache to ~/.local

To your questions:

@codan

Can I pack all 1-4 items from my list in one AppImage and use from shell which run as AppImage OR I should create separate AppImages for zsh, python and autojump and run it together thru configs and scripting?

Either way is theoretically possible, but packing everything together in 1 AppImage will probably be much easier for you and your users.

@codan

What is the best way to pack shell (zsh) into AppImage[/quote]

I can’t say because I really have absolute no experience with it, neither as a user nor from understanding what goes on inside it. Normally we recommend “upstream packaging” where the original author of an application (or, say, shell in this case) who understands how the application works (and can make adjustments to the code if required) packages it as an AppImage and tests it to ensure it works exactly as intended.

@codan

If it helps: my experiments shows that zsh has absolute paths for libs inside zsh executable and should be rebuilded from source with appropriate PREFIX for non standard directory.

This is the first thing that would need to be changed in its code. An application packaged inside an AppImage cannot rely on some absolute paths, because each time an AppImage executes, the paths are different. This means that an application shall load its resources never from compiled-in absolute paths, but from paths relative to the main executable. This is called “relocatablility”.

https://github.com/limbahq/binreloc is one possibility for creating relocatable applications and libraries.

@codan

For python I’ve read Bundling-Python-apps and choose conda. Any advice on this with my case?

For "python 3 with packages " this may well be the way to go - or use something like this:

niess/linuxdeploy-plugin-python

@codan

OMZ and autojump — it’s external scripts but they expect environment around. Should I pack them inside AppImage or for example I can place it to AppImage.home and write entrypoint script which will fix paths and then run zsh?

I don’t know how zsh expects things, one way is what I described above - make sure these things are “copied out” into $HOME before running zsh. Maybe zsh has also some environment variables that you could set before running zsh, pointing zsh to the location of these things inside the AppDir (relative to the location of the AppDir itself because this location will be changing).

Hope this makes sense. Good luck!

[codan]

Thank you for the fast answer!

This means that zsh is trying to load zle.so from inside the AppImage (which is good - it would be worse if it was trying to load it from /usr in the system). Why does it fail, though? Does the file exist at /tmp/test/AppDir/usr/conda/lib/zsh/5.6.2/zsh/zle.so ?

What I did:

1. I built AppImage via linuxdeploy/linuxdeploy-plugin-conda and got AppDir and zsh.AppImage which works
2. Next I delete AppDir with expectation that AppImage will work but I’ve got the error above. I suppose that conda rebuild zsh with PREFIX="/tmp/test/AppDir" and when I run AppImage zsh expects to get the lib from this path but I deleted it and this is the cause of error.

What happens if you run ldd /tmp/test/AppDir/usr/conda/lib/zsh/5.6.2/zsh/zle.so ?

$ ldd /tmp/test/AppDir/usr/conda/lib/zsh/5.6.2/zsh/zle.so                                                                                                    linux-vdso.so.1 (0x00007fffac3f5000)
        libtinfo.so.6 => /tmp/test/AppDir/usr/conda/lib/zsh/5.6.2/zsh/../../../libtinfo.so.6 (0x00007f01f6902000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f01f637e000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f01f676f000)

Since what is inside the AppImage does not reside in the locations it is probably expected to be by the application in ( $HOME and it subdirectories), you must make sure that the code inside the AppDir (AppRun or something called by it) will copy these files into $HOME and it subdirectories before the main payload application gets launched.

Am I understand correctly that I should place the files to zsh.AppImage.home, then set env variables to this dir and then try to run zsh.AppImage? If so I do right way.

https://github.com/limbahq/binreloc

Will research. Thanks!

Normally we recommend “upstream packaging”

Got it. I’m going to write to zsh mailing list or smth like that.

What you are trying to bundle is much more than what we normally call an “application”

I know This isn’t first try to do it But I found AppImage more perspective than others methods like build from sources every time on every system

[probono]

I suppose that conda rebuild zsh with PREFIX="/tmp/test/AppDir" and when I run AppImage zsh expects to get the lib from this path but I deleted it and this is the cause of error.

That may very well be the case. Run grep -r /tmp/test/AppDir /tmp/.mount_* while the AppImage is running to see if this path is hardcoded somewhere inside the contents of the AppImage. If it is, then it needs to be changed. Absolute paths to things that are supposed to be inside the AppImage will not work.

Am I understand correctly that I should place the files to zsh.AppImage.home , then set env variables to this dir and then try to run zsh.AppImage ? If so I do right way.

No. Please read again what I wrote

I found AppImage more perspective than others methods like build from sources every time on every system

Thanks I think so too.

[codan]

Run grep -r /tmp/test/AppDir /tmp/.mount_* while the AppImage is running to see if this path is hardcoded

You mean grep -r /tmp/test/ /tmp/test/AppDir/? It match 775 files and one of them is zsh binary )

That’s it. To be continued…

[codan]

For information. Here is the thread in ZSH mailing list about portable rootless ZSH - https://www.zsh.org/mla/workers/2019/msg00841.html