designing dh_pytorch for PyTorch reverse dependencies

["M. Zhou" <lumin@debian.org>]

Hi folks,

I'm writing down my draft design for dh_pytorch here in order to hear
some comments or feedbacks. If you maintain a package that depends
on python3-torch or relevant pytorch packages, please let me have
your attention.

We need a custom dh module for pytorch because we have multiple
pytorch variants in our repository, which might or might not
fill the reverse dependency requirements of its reverse dependencies.
PyTorch is somewhat similar to numpy, but we have to implement
something different than dh_numpy.

For instance, a reverse dependency might be agnostic to the actual
variant of pytorch. An example is src:pytorch-ignite. It provides
a higher-level abstract to the pytorch API, and contains no arch
specific file. So it can depend on any pytorch variant, e.g.,
  Depends: python3-torch | python3-torch-cuda | python3-torch-...

A reverse dependency might be depending on a particular variant
as well. An example is src:pytorch-vision. It compiles binaries
against libtorch2.0 (with B-D: libtorch-dev). In that case,
the cpu variant (python3-torch) will not be able to fill the
dependency requirement.

My proposed solution for automatically filling in the pytorch
dependency is as follows:

Firstly, every pytorch variant will provide a virtual package
  python3-torch-api-2.0
to declare that the package provides the high level API of pytorch.

Then it is the dh_pytorch processing logic in pseudo code:
(dh_pytorch is inserted before dh_gencontrol)

for each binary package $pkg {

  if $pkg.architecture is all {
    # does not require specific variant
    append the following to python3:Depends
    "python3-torch | python3-torch-api-2.0"

  } else {
    # for arch-any package

    if $pkg.substvars contains "libtorch*" {
      # this variant-specific dependency
      do nothing to python3:Depends.
      Just use the default dependency.
    
    } else {
      # does not contain "libtorch*" dependency
      # that indivates variant-agnostic package.
      append the following to python3:Depends
      "python3-torch | python3-torch-api-2.0"
  }
}

The $pkg.substvars files are generated by dpkg-shlibdeps.


As a result, the src:pytorch-ignite, a variant-agnostic reverse
dependency, will be able to depend on any variant that provides
python3-torch-api-2.0, including python3-torch, python3-torch-cuda,
python3-torch-rocm, etc.

Meanwhile, the src:pytorch-vision, a variant-specific reverse
depenedency, will simply require python3-torch-cuda.
(I will upload the cuda version of torchvision as
python3-torchvision-cuda in the future)

I don't have much experience in writing custom debhelper modules.
Any comments or suggestions are welcome.
The perl implementation is work in progress.

Thanks