I apologize in advanced for the dense post.
I’d like to deploy the django webapp scancode.io (https://github.com/nexB/scancode.io) as an AppImage. As a first attempt, I want to run this using the django development server (running the webserver using manage.py runserver) using an sqlite database. I’ve made the AppImage’s entrypoint to be scancode.io’s management.py script, where you can invoke django management commands. I’ve tried to create AppImages using pkg2appimage and appimage-builder, but I have not had much success. I would appreciate any help with the issues I’ve encountered using these tools.
I used pkg2appimage and appimage-builder in a Docker container created from the following Dockerfile
FROM ubuntu:20.04
# Build tools and deps
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
libexpat1 \
gtk-update-icon-cache \
ca-certificates \
gcc \
libpq-dev \
python3-dev \
wget \
python3-pip \
file \
desktop-file-utils \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
WORKDIR /tmp
RUN wget https://github.com/AppImage/AppImageKit/releases/download/continuous/appimagetool-x86_64.AppImage && \
sed 's|AI\x02|\x00\x00\x00|g' -i /tmp/appimagetool-x86_64.AppImage && \
chmod +x /tmp/appimagetool-x86_64.AppImage && \
cd /opt && /tmp/appimagetool-x86_64.AppImage --appimage-extract && \
mv squashfs-root appimage-tool.AppDir && \
ln -s /opt/appimage-tool.AppDir/AppRun /usr/bin/appimagetool && \
rm /tmp/appimagetool-x86_64.AppImage
WORKDIR /tmp
RUN wget https://github.com/AppImageCrafters/appimage-builder/releases/download/Continuous/appimage-builder-1.1.1.dev32+g2709a3b-x86_64.AppImage -O appimage-builder.AppImage && \
sed 's|AI\x02|\x00\x00\x00|g' -i /tmp/appimage-builder.AppImage && \
chmod +x /tmp/appimage-builder.AppImage && \
cd /opt && /tmp/appimage-builder.AppImage --appimage-extract || true && \
mv squashfs-root appimage-builder.AppDir && \
ln -s /opt/appimage-builder.AppDir/AppRun /usr/bin/appimage-builder && \
rm /tmp/appimage-builder.AppImage
WORKDIR /tmp
RUN wget https://github.com/AppImageCommunity/pkg2appimage/releases/download/continuous/pkg2appimage-1807-x86_64.AppImage -O pkg2appimage.AppImage && \
sed 's|AI\x02|\x00\x00\x00|g' -i /tmp/pkg2appimage.AppImage && \
chmod +x /tmp/pkg2appimage.AppImage && \
cd /opt && /tmp/pkg2appimage.AppImage --appimage-extract || true && \
mv squashfs-root pkg2appimage.AppDir && \
ln -s /opt/pkg2appimage.AppDir/AppRun /usr/bin/pkg2appimage && \
sed 's|AI\x02|\x00\x00\x00|g' -i /opt/pkg2appimage.AppDir/usr/bin/appimagetool && \
rm /tmp/pkg2appimage.AppImage
WORKDIR /
I used pkg2appimage to create an AppImage, but I have the issue where I cannot run the AppImage on systems whose glibc versions are lower than 2.29, due to how Python 3.8 was compiled for Ubuntu 20.04. In my pkg2appimage recipe, I use the packages from Ubuntu 20.04 to install Python and other system dependencies. I tried to use an older version (18.04), but the provided versions of Python were too old to with scancode.io. The recipe and build instructions I used can be found here: (https://github.com/nexB/scancode.io/tree/scancode.io-appimage/etc/scripts/appimage-build)
I’ve also tried to use appimage-builder, where I run into some problems using the following recipe:
version: 1
script:
# Remove any previous build
- rm -rf AppDir | true
# Make opt
- mkdir -p AppDir/opt/scancodeio
# Download scancode.io and extract
- wget -c https://github.com/nexB/scancode.io/archive/main.tar.gz -O - | tar xz --directory=./AppDir/opt/scancodeio --strip-components=1
# Make .env file
- echo SCANCODEIO_DB_ENGINE=\"django.db.backends.sqlite3\" >> AppDir/opt/scancodeio/.env
- echo SECRET_KEY=\"BWzNxgGEfMuj2p40InQ0gsb23q6rAFijSmuS0aT13sVT4iWFwr\" >> AppDir/opt/scancodeio/.env
# Make usr and icons dirs
- mkdir -p AppDir/usr/share/icons/hicolor/256x256/apps/
- cp python.png AppDir/usr/share/icons/hicolor/256x256/apps/scancodeio.png
AppDir:
path: AppDir
after_bundle: |
AppDir/usr/bin/python3 -m pip install --system AppDir/opt/scancodeio
AppDir/usr/bin/python3 -m pip install --system --upgrade spdx-tools==0.7.0a3
AppDir/usr/bin/python3 -m pip install --system --upgrade packaging==21.3
app_info:
id: com.nexb.scancodeio
name: scancodeio
icon: scancodeio
version: 0.1.0
# Set the python executable as entry point
exec: usr/bin/python3
# Set the application main script path as argument. Use '$@' to forward CLI parameters
exec_args: "$APPDIR/opt/scancodeio/manage.py $@"
apt:
arch: amd64
sources:
- sourceline: 'deb [arch=amd64] http://archive.ubuntu.com/ubuntu/ focal main restricted universe multiverse'
key_url: 'http://keyserver.ubuntu.com/pks/lookup?op=get&search=0x871920D1991BC93C'
include:
- python3
- python3-dev
- python3-pip
- python-is-python3
- bzip2
- xz-utils
- zlib1g
- libxml2-dev
- libxslt1-dev
- libpopt0
exclude: []
runtime:
env:
# Set python home
# See https://docs.python.org/3/using/cmdline.html#envvar-PYTHONHOME
PYTHONHOME: '${APPDIR}/usr'
# Path to the site-packages dir or other modules dirs
# See https://docs.python.org/3/using/cmdline.html#envvar-PYTHONPATH
PYTHONPATH: '${APPDIR}/usr/lib/python3.8/site-packages'
test:
fedora:
image: appimagecrafters/tests-env:fedora-30
command: ./AppRun --help
use_host_x: false
debian:
image: appimagecrafters/tests-env:debian-stable
command: ./AppRun --help
use_host_x: false
arch:
image: appimagecrafters/tests-env:archlinux-latest
command: ./AppRun --help
use_host_x: false
centos:
image: appimagecrafters/tests-env:centos-7
command: ./AppRun --help
use_host_x: false
ubuntu:
image: appimagecrafters/tests-env:ubuntu-xenial
command: ./AppRun --help
use_host_x: false
AppImage:
update-information: ''
sign-key: None
arch: x86_64
First, appimage-builder can’t build an AppImage and gives the following error:
INFO:main:Running AppImage creation
INFO:AppImagePrimer:Creating squashfs from AppDir
Traceback (most recent call last):
File "/opt/appimage-builder.AppDir/usr/bin/appimage-builder", line 8, in <module>
sys.exit(__main__())
File "/opt/appimage-builder.AppDir/usr/lib/python3.8/site-packages/appimagebuilder/__main__.py", line 50, in __main__
invoker.execute(commands)
File "/opt/appimage-builder.AppDir/usr/lib/python3.8/site-packages/appimagebuilder/invoker.py", line 29, in execute
command()
File "/opt/appimage-builder.AppDir/usr/lib/python3.8/site-packages/appimagebuilder/commands/create_appimage.py", line 27, in __call__
self.primer.prime()
File "/opt/appimage-builder.AppDir/usr/lib/python3.8/site-packages/appimagebuilder/modules/prime/appimage_primer.py", line 46, in prime
payload_path = self._make_squashfs(self.context.app_dir)
File "/opt/appimage-builder.AppDir/usr/lib/python3.8/site-packages/appimagebuilder/modules/prime/appimage_primer.py", line 90, in _make_squashfs
subprocess.run(command, check=True)
File "/opt/appimage-builder.AppDir/usr/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/opt/appimage-builder.AppDir/usr/bin/mksquashfs', '/data/AppDir', '/data/AppDir.squashfs', '-root-owned', '-noappend', '-reproducible', '-comp', 'xz']' died with <Signals.SIGBUS: 7>.
However, the AppDir is present and I can use appimagetool to create an AppImage from it.
The second issue that I run into is that when I run ./AppDir/AppRun, I’m able to see the available Django management commands listed and run some of them (like migrate), but when I try to run the webserver using runserver, I get this error:
$ ./AppDir/AppRun runserver
INFO Synchronizing QUEUED and RUNNING Runs with their related Jobs...
Traceback (most recent call last):
File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/opt/scancodeio/manage.py", line 6, in <module>
command_line()
File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/opt/scancodeio/scancodeio/__init__.py", line 45, in command_line
execute_from_command_line(sys.argv)
File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/local/lib/python3.8/dist-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
utility.execute()
File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/local/lib/python3.8/dist-packages/django/core/management/__init__.py", line 440, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/local/lib/python3.8/dist-packages/django/core/management/base.py", line 402, in run_from_argv
self.execute(*args, **cmd_options)
File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/local/lib/python3.8/dist-packages/django/core/management/commands/runserver.py", line 74, in execute
super().execute(*args, **options)
File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/local/lib/python3.8/dist-packages/django/core/management/base.py", line 448, in execute
output = self.handle(*args, **options)
File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/local/lib/python3.8/dist-packages/django/core/management/commands/runserver.py", line 111, in handle
self.run(**options)
File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/local/lib/python3.8/dist-packages/django/core/management/commands/runserver.py", line 118, in run
autoreload.run_with_reloader(self.inner_run, **options)
File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/local/lib/python3.8/dist-packages/django/utils/autoreload.py", line 682, in run_with_reloader
exit_code = restart_with_reloader()
File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/local/lib/python3.8/dist-packages/django/utils/autoreload.py", line 274, in restart_with_reloader
p = subprocess.run(args, env=new_environ, close_fds=False)
File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/lib/python3.8/subprocess.py", line 489, in run
with Popen(*popenargs, **kwargs) as process:
File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/lib/python3.8/subprocess.py", line 854, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/lib/python3.8/subprocess.py", line 1592, in _execute_child
self._posix_spawn(args, executable, env, restore_signals,
File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/lib/python3.8/subprocess.py", line 1543, in _posix_spawn
self.pid = os.posix_spawn(executable, args, env, **kwargs)
FileNotFoundError: [Errno 2] No such file or directory: '/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/bin/python3'
I get the same error when I build the AppDir using appimagetool. I’m not quite sure what’s going on here, if I had messed something up with the Python path.