Bug#793057: Upstream advice on packaging Godot
Hi there,
Godot upstream maintainer here [0], as well as its packager for Mageia
[1] and (soon, pending review) Fedora/RHEL [2] [3]. I thought I'd
chime in to give some more tips about packaging Godot from a fellow
packager's point of view with a good understanding of the upstream
setup :)
Sorry, it's going to be a long post, but Godot is a complex
cross-platform project and packaging it efficiently requires some
explanations on how things are setup.
## Things to build and package
For the reference, here are some tips I sent to debian-devel-games and
fedora-games last year [4]. The packaging structure I proposed back
then is still relevant, albeit I decided to drop the idea to package
the Linux export templates as it's too much hassle for little gain,
when users need to download the upstream templates anyway for other
platforms.
So I suggest that you package Godot this way:
- "godot" package, optimized build of the editor (`p=x11 tools=yes`);
- "godot-runner" package, optimized build of the Linux export template
(`p=x11 tools=no`) - it can then be used directly to run any Godot
game (both from source or from a data pack), which is the best way to
package libre Godot games in the future like Tanks of Freedom,
Minilens or The Diasporic Crypt;
- (optionally) "godot-server" package (`p=server tools=yes`), which is
a headless build of the editor which can be used to do command line
operations in a non-graphical environment (e.g. automating exports to
various platforms), but more importantly power a game's server (as of
Godot 2.1.x I don't know of libre multiplayer game which would need it
though).
- (optionally) "godot-demos" with the upstream demos [5], all MIT licensed.
Regarding your discussion on what version to package, I encourage to
package the stable version (2.1.4), potentially with some backported
patches from the 2.1 branch (see below).
When Godot 3.0 comes out, it might make sense to provide both
(especially if you're packaging some libre games which are not ported
to Godot 3.0+ yet).
How to name the packages is up to you, for Mageia and Fedora my Godot
2.1.4 packaged is "godot". When Godot 3.0 is released, I'll fork it to
a new "godot2" package, and will upgrade the main "godot" package to
version 3.0.
## Licensing and copyright
For the current Godot 2.1.4 release, I've done a lot of work to
properly document the licenses and copyright statements of all source
files included in Godot's source repository. This is all listed in
COPYRIGHT.txt [6] using the Debian machine-readable format spec, and
should be a good base for your own debian/copyright after editing it
to reflect unbundling of vendored libraries (see below).
Note that for vendored thirdparty libraries, I stopped at their "main"
copyright statement and license (typically from COPYING, LICENSE.txt
or the header of their main code files), and did not try to redo all
the work they should be doing themselves to documented code snippets
they took from other license-compatible sources.
## Unbundling thirdparty libraries
For convenience, both to be able to build out of the box for all the
platforms it supports, and to be able to make portable Linux binaries
as export templates, we vendor the libraries that Godot relies upon
[7].
You cannot however simply delete this folder, as some of the
thirdparty code included there is not packaged in Debian (and probably
doesn't need to be). Everything that can be unbundled has a
`builtin_${name}=no` SCons option (you can list them all with `scons
--help`, or check the SConscript [8]).
As we use the bundled stuff by default so that Godot users who build
their own templates from source can have portable binaries, you'll
have to specify the various "builtin_${name}=no" arguments manually.
In the future I intend to add a "packaging" option that will do it
automatically, but haven't got to doing it yet.
Here's how I do it for Mageia/Fedora:
```
# Windows-specific
rm -rf thirdparty/rtaudio
# Use system libraries when available
system_libs=""
for lib in freetype glew libogg libpng libtheora libvorbis libwebp
openssl opus zlib; do
system_libs+="builtin_"$lib"=no "
rm -rf thirdparty/$lib
done
```
Then build Godot binaries with `scons $system_libs <other options>`.
### Caveats
After the above step, there are still some vendored MIT-compatible
libraries. See this comment [9] for details about each of them. In
particular:
- libmpcdec and speex could be unbundled, but would require some dev
work. Since they've been removed from the master branch (upcoming
Godot 3.0), I didn't bother. You are however free to remove them and
disable the modules that use them with `module_mpc_enabled=no
module_speex_enabled=no`.
- minizip sadly has some custom modifications which I haven't managed
to get rid of yet, so it's advised to build against the custom code
(the changes are documented in patches included in the source folder)
- the certs bundle needs an update upstream, but it's basically just
the regular Mozilla certs bundle as packaged by distros. It's needed
at built time as it's converted to a header and included directly in
the Godot binaries (which are self-contained), so you can remove it
and symlink it to the corresponding openssl-friendly Debian certs
bundle before building.
I'm of course interested in any feedback or patches that you may have
to improve the current state - the more stuff we can unbundle without
causing troubles for self-compiling users and other OSes, the better.
## Building
Up to the current 2.1.4 release, Godot has three noteworthy caveats
for building:
- Compiling release builds with GCC 6+ leads to potential crashes -
this was a complex issue involving improvement alignment and
preventing any kind of null pointer dereferencing, and it was recently
fixed in the master branch. The changes are quite complex and we
decided not to try to backport them to the 2.1 branch as the focus is
on master and the upcoming 3.0 release.
In the meantime, the 2.1.x releases should be compiled either with GCC
5, or with any version of Clang - only GCC 6+ got stricter about what
we were doing wrong.
- OpenSSL 1.1.0 was not supported in 2.1.4, but there's a later patch
that adds support for it, so you should probably cherry-pick it [10].
- The misnomer `debug_release=yes` option (not to confuse with the
`target=release_debug` which is an optimized binary with -DDEBUG) adds
debugging symbol to the binary - if you handle stripping manually to
extract debuginfo during packaging, you should probably define it.
When building, you can define the `BUILD_REVISION` environment
variable to override the default `custom_build`.
`BUILD_REVISION=debian` is probably a good pick, the resulting version
string will be "2.1.4.stable.debian".
## Dist stuff
The 2.1.4 release lacked an upstream .desktop file, .appdata.xml and
man page. I've added them recently to the master and the 2.1 branches,
you can cherry-pick these commits to get them (in `misc/dist/linux`):
[11] [12] [13]. To apply [11] you'd need to fix a merge conflict, but
you can instead use the patches from my Fedora package [14] which are
already rediffed against the 2.1.4 release.
## Asset Library
I know this might be against Debian policies, so I prefer to mention
it upfront. Godot "calls home" for its Asset Library feature [15],
which is a repository of libre licensed assets, plugins, code
snippets, demos, etc. which can be downloaded directly from the
editor. The downloads themselves are not hosted by Godot Engine, but
retrieved directly from public forges such as GitHub, GitLab, Gogs,
etc. - the content is moderated manually and validated using the
sha256 of the zip archive for a given commit/tag, so if the remote
repo changes after moderation, the download is invalidated.
If this is an issue, the Asset Library can easily be disabled with a
simple patch, I can provide one upon request.
--
That's about it for this already pretty long email :)
As mentioned above, I'm a packager myself and quite aware of Linux
packaging policies (I actually came to Godot initially with the intent
to package it for Mageia, and ended up as the project manager and
releng :)), so I'd be glad to discuss what needs to be improved and to
review patches.
Feel free to contact me either via this bug report, or on the
debian-devel-games ML, or in private if need be.
Librement,
Rémi Verschelde / Akien
[0] https://github.com/akien-mga
[1] https://madb.mageia.org/package/show/source/1/name/godot
[2] https://gitlab.com/snippets/1676357
[3] https://copr.fedorainfracloud.org/coprs/akien/godot/
[4] https://lists.debian.org/debian-devel-games/2016/02/msg00016.html
[5] https://github.com/godotengine/godot-demo-projects
[6] https://github.com/godotengine/godot/blob/2.1/COPYRIGHT.txt
[7] https://github.com/godotengine/godot/tree/2.1/thirdparty
[8] https://github.com/godotengine/godot/blob/2.1/SConstruct#L151
[9] https://gitlab.com/snippets/1676357#note_41250096
[10] https://github.com/godotengine/godot/commit/8d246dfc1d787756d42b3026cb4935bb37d0e49b
[11] https://github.com/godotengine/godot/commit/893ebc54fceae13fe7a0427be5a11601ef30e601
[12] https://github.com/godotengine/godot/commit/514c8ab66cd3137c5d8f0f4f1ea045e12876685e
[13] https://github.com/godotengine/godot/commit/e25c37afa12d96b58a6a2f923d6b077905247f46
[14] http://copr-dist-git.fedorainfracloud.org/cgit/akien/godot/godot.git/tree/
[15] https://godotengine.org/asset-library/
Reply to: