Encountering Difficulties Packaging Django App in AppImage

I apologize in advanced for the dense post.

I’d like to deploy the django webapp scancode.io (https://github.com/nexB/scancode.io) as an AppImage. As a first attempt, I want to run this using the django development server (running the webserver using manage.py runserver) using an sqlite database. I’ve made the AppImage’s entrypoint to be scancode.io’s management.py script, where you can invoke django management commands. I’ve tried to create AppImages using pkg2appimage and appimage-builder, but I have not had much success. I would appreciate any help with the issues I’ve encountered using these tools.

I used pkg2appimage and appimage-builder in a Docker container created from the following Dockerfile

FROM ubuntu:20.04

# Build tools and deps
RUN apt-get update \
 && apt-get install -y --no-install-recommends \
    libexpat1 \
    gtk-update-icon-cache \
    ca-certificates \
    gcc \
    libpq-dev \
    python3-dev \
    wget \
    python3-pip \
    file \
    desktop-file-utils \
 && apt-get clean \
 && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

WORKDIR /tmp
RUN wget https://github.com/AppImage/AppImageKit/releases/download/continuous/appimagetool-x86_64.AppImage && \
    sed 's|AI\x02|\x00\x00\x00|g' -i /tmp/appimagetool-x86_64.AppImage && \
    chmod +x /tmp/appimagetool-x86_64.AppImage && \
    cd /opt && /tmp/appimagetool-x86_64.AppImage --appimage-extract && \
    mv squashfs-root appimage-tool.AppDir && \
    ln -s /opt/appimage-tool.AppDir/AppRun /usr/bin/appimagetool && \
    rm /tmp/appimagetool-x86_64.AppImage

WORKDIR /tmp
RUN wget https://github.com/AppImageCrafters/appimage-builder/releases/download/Continuous/appimage-builder-1.1.1.dev32+g2709a3b-x86_64.AppImage -O appimage-builder.AppImage && \
    sed 's|AI\x02|\x00\x00\x00|g' -i /tmp/appimage-builder.AppImage && \
    chmod +x /tmp/appimage-builder.AppImage && \
    cd /opt && /tmp/appimage-builder.AppImage --appimage-extract || true && \
    mv squashfs-root appimage-builder.AppDir && \
    ln -s /opt/appimage-builder.AppDir/AppRun /usr/bin/appimage-builder && \
    rm /tmp/appimage-builder.AppImage

WORKDIR /tmp
RUN wget https://github.com/AppImageCommunity/pkg2appimage/releases/download/continuous/pkg2appimage-1807-x86_64.AppImage -O pkg2appimage.AppImage && \
    sed 's|AI\x02|\x00\x00\x00|g' -i /tmp/pkg2appimage.AppImage && \
    chmod +x /tmp/pkg2appimage.AppImage && \
    cd /opt && /tmp/pkg2appimage.AppImage --appimage-extract || true && \
    mv squashfs-root pkg2appimage.AppDir && \
    ln -s /opt/pkg2appimage.AppDir/AppRun /usr/bin/pkg2appimage && \
    sed 's|AI\x02|\x00\x00\x00|g' -i /opt/pkg2appimage.AppDir/usr/bin/appimagetool && \
    rm /tmp/pkg2appimage.AppImage

WORKDIR /

I used pkg2appimage to create an AppImage, but I have the issue where I cannot run the AppImage on systems whose glibc versions are lower than 2.29, due to how Python 3.8 was compiled for Ubuntu 20.04. In my pkg2appimage recipe, I use the packages from Ubuntu 20.04 to install Python and other system dependencies. I tried to use an older version (18.04), but the provided versions of Python were too old to with scancode.io. The recipe and build instructions I used can be found here: (https://github.com/nexB/scancode.io/tree/scancode.io-appimage/etc/scripts/appimage-build)

I’ve also tried to use appimage-builder, where I run into some problems using the following recipe:

version: 1
script:
  # Remove any previous build
  - rm -rf AppDir  | true
  # Make opt
  - mkdir -p AppDir/opt/scancodeio
  # Download scancode.io and extract
  - wget -c https://github.com/nexB/scancode.io/archive/main.tar.gz -O - | tar xz --directory=./AppDir/opt/scancodeio --strip-components=1
  # Make .env file
  - echo SCANCODEIO_DB_ENGINE=\"django.db.backends.sqlite3\" >> AppDir/opt/scancodeio/.env
  - echo SECRET_KEY=\"BWzNxgGEfMuj2p40InQ0gsb23q6rAFijSmuS0aT13sVT4iWFwr\" >> AppDir/opt/scancodeio/.env
  # Make usr and icons dirs
  - mkdir -p AppDir/usr/share/icons/hicolor/256x256/apps/
  - cp python.png AppDir/usr/share/icons/hicolor/256x256/apps/scancodeio.png

AppDir:
  path: AppDir

  after_bundle: |
    AppDir/usr/bin/python3 -m pip install --system AppDir/opt/scancodeio
    AppDir/usr/bin/python3 -m pip install --system --upgrade spdx-tools==0.7.0a3
    AppDir/usr/bin/python3 -m pip install --system --upgrade packaging==21.3

  app_info:
    id: com.nexb.scancodeio
    name: scancodeio
    icon: scancodeio
    version: 0.1.0
    # Set the python executable as entry point
    exec: usr/bin/python3
    # Set the application main script path as argument. Use '$@' to forward CLI parameters
    exec_args: "$APPDIR/opt/scancodeio/manage.py $@"

  apt:
    arch: amd64
    sources:
      - sourceline: 'deb [arch=amd64] http://archive.ubuntu.com/ubuntu/ focal main restricted universe multiverse'
        key_url: 'http://keyserver.ubuntu.com/pks/lookup?op=get&search=0x871920D1991BC93C'

    include:
      - python3
      - python3-dev
      - python3-pip
      - python-is-python3
      - bzip2
      - xz-utils
      - zlib1g
      - libxml2-dev
      - libxslt1-dev
      - libpopt0
    exclude: []

  runtime:
    env:
      # Set python home
      # See https://docs.python.org/3/using/cmdline.html#envvar-PYTHONHOME
      PYTHONHOME: '${APPDIR}/usr'
      # Path to the site-packages dir or other modules dirs
      # See https://docs.python.org/3/using/cmdline.html#envvar-PYTHONPATH
      PYTHONPATH: '${APPDIR}/usr/lib/python3.8/site-packages'


  test:
    fedora:
      image: appimagecrafters/tests-env:fedora-30
      command: ./AppRun --help
      use_host_x: false
    debian:
      image: appimagecrafters/tests-env:debian-stable
      command: ./AppRun --help
      use_host_x: false
    arch:
      image: appimagecrafters/tests-env:archlinux-latest
      command: ./AppRun --help
      use_host_x: false
    centos:
      image: appimagecrafters/tests-env:centos-7
      command: ./AppRun --help
      use_host_x: false
    ubuntu:
      image: appimagecrafters/tests-env:ubuntu-xenial
      command: ./AppRun --help
      use_host_x: false

AppImage:
  update-information: ''
  sign-key: None
  arch: x86_64

First, appimage-builder can’t build an AppImage and gives the following error:

INFO:main:Running AppImage creation
INFO:AppImagePrimer:Creating squashfs from AppDir
Traceback (most recent call last):
  File "/opt/appimage-builder.AppDir/usr/bin/appimage-builder", line 8, in <module>
    sys.exit(__main__())
  File "/opt/appimage-builder.AppDir/usr/lib/python3.8/site-packages/appimagebuilder/__main__.py", line 50, in __main__
    invoker.execute(commands)
  File "/opt/appimage-builder.AppDir/usr/lib/python3.8/site-packages/appimagebuilder/invoker.py", line 29, in execute
    command()
  File "/opt/appimage-builder.AppDir/usr/lib/python3.8/site-packages/appimagebuilder/commands/create_appimage.py", line 27, in __call__
    self.primer.prime()
  File "/opt/appimage-builder.AppDir/usr/lib/python3.8/site-packages/appimagebuilder/modules/prime/appimage_primer.py", line 46, in prime
    payload_path = self._make_squashfs(self.context.app_dir)
  File "/opt/appimage-builder.AppDir/usr/lib/python3.8/site-packages/appimagebuilder/modules/prime/appimage_primer.py", line 90, in _make_squashfs
    subprocess.run(command, check=True)
  File "/opt/appimage-builder.AppDir/usr/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/opt/appimage-builder.AppDir/usr/bin/mksquashfs', '/data/AppDir', '/data/AppDir.squashfs', '-root-owned', '-noappend', '-reproducible', '-comp', 'xz']' died with <Signals.SIGBUS: 7>.

However, the AppDir is present and I can use appimagetool to create an AppImage from it.

The second issue that I run into is that when I run ./AppDir/AppRun, I’m able to see the available Django management commands listed and run some of them (like migrate), but when I try to run the webserver using runserver, I get this error:

$ ./AppDir/AppRun runserver
INFO Synchronizing QUEUED and RUNNING Runs with their related Jobs...
Traceback (most recent call last):
  File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/opt/scancodeio/manage.py", line 6, in <module>
    command_line()
  File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/opt/scancodeio/scancodeio/__init__.py", line 45, in command_line
    execute_from_command_line(sys.argv)
  File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/local/lib/python3.8/dist-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
    utility.execute()
  File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/local/lib/python3.8/dist-packages/django/core/management/__init__.py", line 440, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/local/lib/python3.8/dist-packages/django/core/management/base.py", line 402, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/local/lib/python3.8/dist-packages/django/core/management/commands/runserver.py", line 74, in execute
    super().execute(*args, **options)
  File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/local/lib/python3.8/dist-packages/django/core/management/base.py", line 448, in execute
    output = self.handle(*args, **options)
  File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/local/lib/python3.8/dist-packages/django/core/management/commands/runserver.py", line 111, in handle
    self.run(**options)
  File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/local/lib/python3.8/dist-packages/django/core/management/commands/runserver.py", line 118, in run
    autoreload.run_with_reloader(self.inner_run, **options)
  File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/local/lib/python3.8/dist-packages/django/utils/autoreload.py", line 682, in run_with_reloader
    exit_code = restart_with_reloader()
  File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/local/lib/python3.8/dist-packages/django/utils/autoreload.py", line 274, in restart_with_reloader
    p = subprocess.run(args, env=new_environ, close_fds=False)
  File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/lib/python3.8/subprocess.py", line 489, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/lib/python3.8/subprocess.py", line 854, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/lib/python3.8/subprocess.py", line 1592, in _execute_child
    self._posix_spawn(args, executable, env, restore_signals,
  File "/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/lib/python3.8/subprocess.py", line 1543, in _posix_spawn
    self.pid = os.posix_spawn(executable, args, env, **kwargs)
FileNotFoundError: [Errno 2] No such file or directory: '/home/jono/nexb/projects/scancode.io-appimage/test3/AppDir/usr/bin/python3'

I get the same error when I build the AppDir using appimagetool. I’m not quite sure what’s going on here, if I had messed something up with the Python path.