Re: Cross-building with python3-numpy
Hi Dima,
Quoting Dima Kogan (2023-01-16 19:41:29)
> Helmut Grohne <helmut@subdivi.de> writes:
> > On Sat, Jan 14, 2023 at 03:20:22PM -0800, Dima Kogan wrote:
> >> Unsurprisingly, added python3-numpy:any to the Build-Depends complains a
> >> LOT.
> > The :any annotation is only permitted for packages that happen to be
> > tagged Multi-Arch: allowed. python3-numpy isn't thus tagged, which makes
> > it fairly obvious why it complains.
> Yes. I see that now. Two notes:
>
> I was looking at the wiki page, and I think I misinterpreted it to mean
> that most of the time you want Build-Depends:something:any. Here's the
> line that made me think that:
>
> https://wiki.debian.org/CrossBuildPackagingGuidelines#Use_.27:any.27_qualifier_for_dependencies
I'd like to improve the wiki page. Would it help if that heading had an added
"if possible" at the end? Because if you read on, then the first sentence in
the paragraph below says, that this only applies to dependencies on
Multi-Arch:allowed packages.
> Based on the table on that page, I'm now thinking that most of the time
> you want Build-Depends:something and the Multi-Arch tags in the dependee
> should be set correctly to determine the right architecture to pull in.
Yes, no M-A field, M-A:same and M-A:foreign are used so that the dependee can
decide which architecture is the correct one. So most of the time, the dependee
setting its M-A field correctly will do the right thing. But sometimes (and the
dependee cannot know that) you want the build-architecture package and then
(unless your dependee is M-A:foreign) you either use :native or with
M-A:allowed packages you use :any.
But the wiki page already says the above, no? What is it missing?
> I think the wiki should be edited, but I'm not feeling confident-enough
> in my understanding of this at the moment to be making changes.
But that's helpful because by understanding what you did not understand yet we
can improve the docs, so thank you for bringing it up!
> Another question. You just said that python3-numpy isn't
> Multi-Arch:allowed. How did you know that? Normally I would "apt show
> python3-numpy", but that command doesn't show Multi-Arch tags
The Multi-Arch field can have four values: "no", "same", "foreign" and
"allowed". If the field is missing, then the default value is assumed, which is
"no". There will be some documentation for this in dpkg soon:
https://git.hadrons.org/cgit/debian/dpkg/dpkg.git/tree/doc/spec/multiarch.txt?h=pu/doc-multiarch-spec
> >> - One part of being set-up for cross-building is that python3-numpy should
> >> be Multi-Arch:foreign. But how do I query this? "apt show python3-numpy"
> >> doesn't include the Multi-Arch tags.
> >
> > python3-numpy is a Python extension and certainly should not be
> > Multi-Arch: foreign. That tag is an assertion on the interface of the
> > provided package to be architecture-agnostic. Certainly, loading a
> > shared library depends on the architecture of the library and the
> > program it is loaded into to be equal. We've unsuccessfully tried
> > getting this into Debian policy thus far. My own attempt can be found at
> > https://wiki.debian.org/DependencyHell#Multi-Arch:_foreign. Does that
> > help get a better understanding?
>
> I read that page. But... The interface is in Python. I can write Python
> code with 'import numpy', which is architecture-agnostic. The linked
> page says
>
> Binary executables generally differ with architectures, but if the
> interface that they expose to be used by other software (for example the
> command line interface, the content on standard input/output, the way
> they process files) does not allow distinguishing the architecture, they
> are still considered architecture-independent as far as this area is
> concerned.
>
> Does this not apply here?
When you run 'import numpy' in Python, then that statement by itself is indeed
architecture agnostic. It's the same statement independent on the architecture
I want to run it on which is why we would package a script containing it as
Architecture:all. But whether a package is Arch:all or not is independent on
whether or not its interface is architecture dependent or not. If I package a
shell script that runs "uname -a", then that package would be Architecture:all
but it could not be Multi-Arch:foreign because its output when run would depend
on the native architecture. The reason why python3-numpy cannot be M-A:foreign
is a bit different but I wanted to give an example why it doesn't matter that
the code itself is architecture independent.
Why can python3-numpy not be Multi-Arch:foreign? If it were (and if we ignore
that python3-numpy itself depends on python3 and thus python3 would conflict
with itself) then it would mean that it is possible to load up python3 from
architecture A and in that interpreter run "import numpy" loading up
python3-numpy from architecture B. Why is this not possible? Because numpy
provides its functionality as libraries loaded into the interpreter:
dpkg -L python3-numpy | grep '\.so$'
If you try to load a *.so from architecture A into an interpreter of
architecture B you will get an error because you cannot load a foreign
architecture library unless you do trickery as software like box64 does.
I'll leave your remaining questions for somebody else to answer. :)
Thanks!
cheers, josch
Reply to: