AppImage vs Snapcraft vs Flatpak

2019-11-19
Previous post: NEXTSTEP 3.3 on a SPARCstation 10
Next post: Replacing a missing CRT TV Remote Control with an Arduino

This is a rant-sized report of honestly tring both Flatpak and Snapcraft. And only name-dropping AppImage. On Linux, I'm a GNOME user and use Ubuntu (best "standard" Linux distro/Debian repos) and Fedora (best upstream GNOME and freedesktop.org support) on my machines. I'm obviously biased.

Warning: Walls of text! What follows is a dump of a few hours spent with this last weekend. YOTLD.

Updated 2019-11-22.


I recently cleaned up and updated an old GTK+ tool I still use every now and then for breaking up audio files. It's written in C and apart from GTK uses libao and mpg123 as its dependencies, so it's quite a small, nifty utility.

Since I want that tool available on macOS and Windows too, I set up Travis CI to build packages (Travis CI has macOS build machines, and Windows builds are done using Fedora in a Docker image, since Fedora has nice packaging for most of mingw-w64).

As a baseline, the Windows release (that ships with all required libs thanks to Martin Preisler's wonderful mingw-bundledlls and a custom "copy icons from the Adwaita icon theme but only the ones I think the app uses" script) is ~ 9.35 MiB.

The Linux/BSD app packaging situation is always a little tricky. You can rely on distributions packaging the update eventually, or you could add the .spec file (for RPM, although different RPM-based distros have different rules and naming conventions) and debian/ folder (for DEB), but even then, you might need to build against really old versions of libraries (to be maximally compatible) or have builds for different OS versions. Static linking isn't really an option here.

There's basically 3 competing "standards":

AppImage

AppImage has been around since 2004 as "klik", around 2011 seems to be when the current tech was created (according to Wikipedia).

I skip AppImage here, since the community seems to be promoting the solution a bit too heavily and apart from tooling (which is important!) it's not much better (in terms of sandboxing) than shipping binaries with dependencies in a relocatable tarball.

Seriously, it's good stuff, but would need more tooling in the sandboxing and "avoid shipping everything and the kitchen sink" area to be a serious contender.

Snapcraft

According to Wikipedia, snapd/Snappy has been around since 2014.

I tried snapcraft; it has not one but two different tutorials: Here and there. The second one is asking for "your current level of expertise", which tries to be friendly to newbies, but in reality it's making it hard to navigate. Why not have a Single Tutorial that covers all bases? At least the docs are better, and I like the fact that it uses YAML (Flatpak uses JSON). The snapcraft.yaml reference is fine, but if I want to find out what base: core18 means, I click through to core18, and it doesn't tell me anything (yes it's some base thing -- what does it contain and what is it good for?!). If there was a kind of "gui" base that includes GTK and related libraries, a snap built with that base could be quite compact (similar to what "platforms" in Flatpak are?).

The tutorial doesn't really mention possible values for plugin (in my case, Meson), and source-type (in my case, local) but hey, I figured it out somehow.

Also, the tutorial uses DOSBox as an example (this is probably fine), but then it has custom environment variables like DISABLE_WAYLAND: '1' in there that tells you more about the Year Of The Linux Desktop than anything else. If the Canonical engineers would have just picked another package or spent the time to fix up DOSBox to work with Wayland, then the YOTLD might come sooner than "always next year".

One upside of snapcraft is that it can work on macOS and Windows too, so you could build snaps on non-Linux machines. Canonical uses its own Multipass utility for this (yay NIH!), which is basically the same as Docker, but with a Ubuntu/Canonical flavor (I do think that multipass has a bit nicer CLI interface than Docker for spinning up local dev VMs, though -- when it works). snapcraft uses Multipass by default to get a clean build environment, but I had trouble running it in Travis CI (which only has Ubuntu 18.04, and it doesn't properly work if using e.g. Ubuntu 19.10 via Docker).

And right now, my local multipass (on Ubuntu) seems to be broken, it works until the Launched: snapcraft-<packagename> line and then does nothing. Tool-specific things. Hard to debug.

But when it did work, the package was ~ 34 MiB compressed, with the biggest file being libicudata.so.60.2 at ~ 25.7 MiB, followed by libgtk-3.so.0.220.30 coming in at ~ 7 MiB. Yes, every snap has to ship its own ICU. Apparently nobody at Canonical thought it would be a good idea to put ICU into the "base" image? Then again, maybe I just did something wrong and wouldn't need to ship those libs in the snap? I would imagine that even most server apps would (indirectly) need ICU? ICU seems to be fine for base. Put it in base!

After the first successful build of the snap, installing and running the snap failed, but the instructions in this thread helped here, and theming (on Ubuntu) seems to work fine. But why can't snapping GTK GUI apps be part of the official docs?

I don't get why I need to specify all stage-packages, the tool even helpfully lists libs that I missed -- why do I need to add it explicitly and it can't figure it out similar to dpkg-shlibdeps on Debian? And why do I need to list X11/Wayland libs when I don't really care (in this app) about which backend GTK happens to be using?

My app uses the Meson build system, and support for Meson is built into snapcraft, but I needed to specify meson-parameters: ["--prefix=/usr"] to make it work (can't Snap do that automatically? Flatpak seems to be able to figure that detail out for me automatically).

Some positive points: The file format is just a squashfs, and one can use unsquashfs -l <filename>.snap to list the file contents. Also, the filename is as nice as what Debian tools build, meaning <packagename>_<version>_<arch>.snap. Sweet!

It also seems to be super picky about "owning" the snap/ folder in the project, up to the point of complaining if any other files are in that folder than what it expects (it tells you to use snap/local/ for any "custom" files).

So no moving packaging-related files off to e.g. packaging/snap/ by default. However, you can create a snap symlink to whatever subfolder you have and it will be fine. I do wish snapcraft would optionally allow specifying the snapcraft.yaml file on the command line instead of enforcing its file structure, but it's more or less a minor issue that can be worked around with symlinks for now.

The snapcraft.yaml file I came up with is 94 lines and mostly lists run-time shared library dependencies (stage-packages). For whatever reason, I also needed to override the pull step and sed the Icon= line in the .desktop file to contain the full path to the icon file, I don't get why snap can't figure out the full icon path from the icon name, after all, the icon-theme-spec has been around for some time now.

Bad: Snapcraft "owns" $HOME/snap, which appears in the file manager and ls by default, which is bad. There is bug 1575053 for this, but it's basically the snap engineers arguing why it could be a good idea and everybody else telling them what year it is ("It's 2017", "It's 2019", ...).

I started a thread on the snapcraft Forum about how to shrink the package size, it's still not perfect, but first tries resulted in a package size of ~ 344 KiB, which is quite acceptable (app + libao + libmpg123), but snapcraft complains that some libraries I link against (they should come from the GNOME snap) are not shipped with the snap, hm...

Flatpak

Flatpak started out in 2007 as "glick", with work on xdg-app beginning in 2014, which was renamed to Flatpak in 2016.

It uses JSON, and while I prefer YAML for something I type manually, JSON is Just Fine. The Flatpak JSON file I came up with is 35 lines, and there's basically no boilerplate code, all meat. Sweet!

One weird thing is that the sandboxing options (allowing access to the Wayland, X11 and PulseAudio sockets) are added to "finish-args", which doesn't sound intuitive. About the sandboxing permissions: I also don't want to specify both Wayland and X11, I want to say "this uses GTK, so whatever the current display manager, I want to have access to that".

The defaults are good, there is really good documentation and the package is super small (120 KiB, including libao). GTK is not shipped in the package, because it's in the Freedesktop Platform Flatpak, which is installed as a dependency and shared between all installed apps (this is A Good Thing!). Interestingly, mpg123 is also shipped with the Freedesktop Platform, so I didn't even need to ship it, which was a bit of a suprise (I wouldn't have expected it, but I don't mind having to ship less, but that brings up the point of where do I see a list of files and libraries shipped with the freedesktop platform?).

As of this writing, freedesktop-sdk bug#886 prevents me from installing the Flatpak, but forcing it with --no-deps makes it work without issues.

Flatpak is based on ostree and has the concept of repos, which makes it kind of hard to "inspect" the contents of the Flatpak. Snap is much nicer with unsquashfs -l in this regard. I have found some hints that one can unpack the ostree bundle, but it's quite tedious, there should be an easy way to inspect the contents of a Flatpak bundle before installing (and also to verify the contents and paths look good after building the package).

Here's a super annoying big issue: How and where do I specify the version of the package? I haven't found a way do to this yet, all it says is "master".

In a similar vein, in order to build a "package" file, I first have to build the Flatpak into a local repo and then export the bundle from there? I'm sure there was some thinking behind that, but the Snap way of outputting a .snap file by default (that can then be put into a repo in a second step) seems more natural to me.

How do I figure out which freedesktop.org platform versions are on which distro? Is it OK to go with 19.08 or should I stick to 18.08 for now? My suspicion is that it does not matter, and it will just install the runtime in parallel if needed, so I'll just pick the most recent / most convenient.

Good: User data is stored in ~/.var/app/<packagename>, so it's hidden by default, but easy to access if needed (and easy to calculate per-app disk usage, as it's separated).

Distribution Packages

I still hope that there's a place for distribution packages. The leaf packages ("apps") do need some kind of sandboxing, as sandboxing is good and important to have (not only to protect the "root" user, but protect the user's data, as usually there's more privacy-related juicy info in $HOME than in all of the system folders).

Even with Flatpak, there's still some duplication (e.g. the GTK version from the distro package and the Freedesktop Platform Flatpak), but it's better than Snap's "let every snap ship with its own version of ICU" or AppImage's "just run this executable, it contains Everything" approach.


Continue reading: Replacing a missing CRT TV Remote Control with an Arduino
Thomas Perl · 2019-11-19