Involving free-threaded Python in Debian (Was: Free-threaded python3.13t packaging in Debian?)
Hi,
> Python 3.13 is released with experimental free-threaded (or no-gil) mode [1].
> Upstream calls it python3.13t. I wonder whether Debian plans to package it?
Now we're in the forky cycle and Python 3.14 is out. Free-threaded interpreter
is no longer experimental. Many useful packages, e.g. numpy, pandas, have now
supported it [1]. So I think it is time for some discussion again.
Xu Chen mentioned that he had some previous email with Stefanor, so I'm looping
him into this thread.
Per Python documentation [2][3], the free-threaded version (python3.14t) must
be built alongside the regular CPython (python3.14), along with its standard
libraries. So we could treat it as yet another independent Python version and
separately package it as python3.14t, python3.14t-stdlib, python3.14t-dev, etc.
The issue is now how to handle other Python packages (and also applications),
especially those with native code (e.g. C or Rust extension), which require
explicit opt-in to be used by 3.14t. Otherwise, GIL will still be enabled.
There are some possible options (maybe more?):
(a) Let python3.14 and python3.14t share the same dist-packages dir, and build all
python packages only for python3.14. When importing non-pure packages from
dist-packages, python3.14t enables GIL. Users could choose freely from both,
and benefit from nogil feature in their scripts and virtual environments.
After unification of interpreters, we will need to rebuild all packages with
free-threaded support anyway.
This requires minimal effort from the toolchain maintainer now (looks like Xu Chen
has already done some work on this) and still provides nogil option for users.
Then we have to run a large transition on all python packages when free-threaded
Python becomes default, which might happen around ~2028, in the next release cycle
(b) Use separate dist-packages dir for python3.14 and python3.14t, and build all
python packages for both versions (e.g. python3-foo, python3t-foo). This will
lead to extra maintainence work now: we need to modify pybuild to pass the
`Py_GIL_DISABLED` flag when building free-threaded versions; for pure Python
packages, adding only one line of 'Provides' is enough; for complex software
like PyTorch, there might be more work. And I have not figured out a solution
for packages that install executable files into bin.
Such co-existence will surely put more burden on the archive, yet gives package
maintainers and users more time to test and adapt. Some system tools / packages
could have the boost brought by free-threaded Python. However this requires more
than one transition -- one to rebuild all packages for free-threaded Python, and
then remove all non-virtual python3t-foo packages.
(c) Similar to (a), but build for python3.14t by default.
This is somehow aggressive, as many packages are not ready yet. And I'm not sure
what the behavior of GIL-enabled Python importing free-threaded packages will be like.
This removes the need for a later transition or a large number of package addition
/ removal in the archive.
What do you think?
[1]: https://hugovk.github.io/free-threaded-wheels/
[2]: https://docs.python.org/3.14/howto/free-threading-python.html#the-global-interpreter-lock-in-free-threaded-python
[3]: https://peps.python.org/pep-0703/
Thanks,
Shengqi Chen
Reply to: