Experimenting with squashfs instead of ISO9660.
Switching from zisofs to squashfs would:
Significantly save space, up to 45% (e.g., 121 MB rather than 218 MB for QtCreator-5.7.0-x86_64.AppImage when using a 1 MB block size with -comp xz -b 1M but that makes application launches of large apps noticeably slower); or 27% (159 MB rather than 218 MB) with great performance using default gzip2 compression
Allow us to use less arcane generation tools, i.e., mksquashfs instead of xorriso
Significantly speed up AppImage generation time, around 40% (e.g., 1:38 instead of 2:37 for QtCreator-5.7.0-x86_64.AppImage)
There is now also appimagetool, a convenient tool that will replace AppImageAssistant. It can be used to generate AppImages in the new experimental format.
Have added the type 2 image format to the spec draft.
In contrast to the earlier ISO9660-based AppImage format, the new type 2 format:
Is not tied to any particular filesystem format (so there might be others in the future); right now my implementation uses squashfs rather than ISO9660
Does not use a fixed offset to determine where the filesystem image starts, but specifies that the filesystem image starts right after the end of the ELF (however long it is). This means that the runtime does not need to fit into a predetermined amount of space, which is future-proof
Embeds update information in an ELF section inside the AppImage rather than at a predetermined, fixed offset
Allows for embedding a digital signature in an ELF section inside the AppImage rather than a .asc file next to the AppImage
Along with the capabilities of the new features comes a new set of tools that will eventually replace AppImageAssistant, AppImageExtract and friends.
appimagetool right now can:
Generate an AppImage from an AppDir
Automatically detect the architecture
Embed update information inside the AppImage
Calculate the update information string for Bintray repositories
Sign the AppImage with GPG and embed the signature inside the AppImage
validate right now can:
Check the digital signature inside an AppImage to ensure that the AppImage is coming from the original author and has not been tampered with
I am considering - let me know what you think:
Upload to Bintray
Integrate validate into AppImageUpdate
Type 2 AppImages have also learned quite a few new tricks:
--appimage-offset: Print the byte at which the squashfs filesystem begins, for mounting with -o loop,offset=...
--appimage-version: Print the version (git hash) of appimagetool which was used to generate this AppImage
--appimage-updateinformation: Display the update information that is embedded in the AppImage
--appimage-signature: Display the digital signature embedded in the AppImage
--appimage-mount: Only mount the AppImage without executing its payload
--appimage-extract: Only extract the AppImage without executing its payload
--appimage-icon: Only extract the icon of the AppImage without executing its payload
Note that these are not set in stone yet and are subject to change.
wget https://transfer.sh/UQfeE/validate
chmod a+x ./validate
./validate ./XChat_IRC-x86_64.AppImage
(...)
gpg: Signature made Sun 25 Sep 2016 10:41:24 PM CEST using RSA key ID 86C3DFDD
gpg: Good signature from "Testkey" [ultimate]
This looks like great step forward to me. When I have some spare time I will definitely be trying this out with SER Player and will report back.
I had a thought that may or may not be sensible. When we create an AppImage we bundle all the libraries that may be required for any of the platforms we want to support. However, on some platforms many of these libraries are actually present and we are duplicating them on the hard drive (not such a big issue) and in memory.
Would it be possible for the AppImage to check which libraries are available on the host system and delete these duplicated libraries from the bundle so that the system versions are used instead? Though this may create problems if the system libraries are changed to different versions or removed. Anyway, just a thought.
Would it be possible for the AppImage to check which libraries are
available on the host system and delete these duplicated libraries from
the bundle so that the system versions are used instead?
Right now AppImages are read-only; you could unpack an AppImage, delete some libraries, and then repack it. But the result would then be closely coupled to your particular system rather than portable between different machines with different distributions and versions.
I boot into a different Linux distribution each time, and still like to use the same set of apps that I keep around as AppImages.
Wow, nice and sweet! ISO9660 format seemed unfit for this purpose, and AppImage format was very strange to me (but convenient, anyway!). Using squashfs and making it a true ELF (without ISO hacks) is a great leap forward! I have experimented with AppImage Type 1 and will definitely use Type 2 in my project.
Some security considerations:
In AppImage 1 you could easily extract the contents without executing the ELF part. In AppImage 2 you can extract the contents, but you must get the squashfs offset from somewhere.
So, in AppImage 1 the file manager could extract the icon using its system-wide ISO extractor (any of them).
And in AppImage 2 the file manager will have to execute the binary (potentially dangerous!).
Can you point me to the documentation for the format of the AppDir when using appimagetool instead of AppImageAssistant? I have been round all the documentation multiple times but cannot find it and the AppDir that my script builds does not work with appimagetool.
Also, is appimagetool able to generate 32-bit AppImages? I can only find a 64-bit prebuilt version.
Thanks for the reply @probono, very reassuring as always.
This is the output I get when I try to create an AppImage from my AppDir:
cgarry@hp-mate-laptop:~/software/ser-player-clean/linux_appimage/temp$ …/appimagetool-x86_64.AppImage -v ser-player.AppDir
WARNING: appstreamcli is missing, please install it if you want to use AppStream metadata
WARNING: appstream-util is missing, please install it if you want to use AppStream metadata
Desktop file: /home/cgarry/software/ser-player-clean/linux_appimage/temp/ser-player.AppDir/ser-player.desktop
Name: SER Player
Icon: /usr/share/app-install/icons/ser-player.png
Exec: ser-player.wrapper %f
Comment: A video player and processing utility for SER files
Type: Application
Categories: Video;AudioVideo
File used for determining architecture: /home/cgarry/software/ser-player-clean/linux_appimage/temp/ser-player.AppDir/usr/lib/libXi.so.6
Arch: 86_64
App name for filename: SER_Player
dest_path: SER_Player-x86_64.AppImage
DESTINATION not specified, so assuming SER_Player-x86_64.AppImage
/home/cgarry/software/ser-player-clean/linux_appimage/temp/ser-player.AppDir should be packaged as SER_Player-x86_64.AppImage
/usr/share/app-install/icons/ser-player.png{.png,.svg,.svgz,.xpm} not present but defined in desktop file
I also want to quickly mention that since I have been providing my app as an AppImage I have received many messages from Linux users thanking me for supporting their particular distribution. Thanks for all your work on AppImage which has made this possible.
Thanks for that, I also had to remove the ‘/app-install/’ part from the path to make it work. I am not actually sure what that is doing there!
I am now able to build an AppImage using the AppImage version of appimagetool. However, if I try and use a version of appimagetool that I have built myself from the latest sources then it fails. I will post details of this once I am back on my Linux machine.