Re: Multiarching perl
(Disclaimer: this is all from my understanding of how perl/python
works internally, which might still be entirely wrong. And at this
point I just hope the mail actually makes sense, as the words are
sarting to lose meaning every time I review it again :)
[Not CCing mulitarch-devel, because alioth rejects my mails...]
On Fri, 2012-11-23 at 11:35:10 +0200, Niko Tyni wrote:
> On Wed, Nov 21, 2012 at 12:06:10AM +0100, Guillem Jover wrote:
>
> > OOC what would be the point of making perl co-installable with itself
> > when perl-base will not be? The .so files in perl will not be usable
> > for example w/o the matching perl-base:arch.
>
> [...]
>
> > and the XS module is arch:bar, and it simply Depends (w/o being
> > loosened with an :any arch-qualifier) on the /usr/bin/perl package,
>
> OK, this seems to be where we aren't on the same page.
Ah, thanks for the details, now I see what you mean! But unfortunately,
after some pondering on and off these past days, having slept over this,
and if I've not missed something obvious, I think my conclusion now
is that M-A:allowed is useless (or outright wrong) for the specific
packages it was envisaged (that is for at least both the perl and python
interpreters), if we want full multiarch support for them. :(
It should still be useful for the case it was designed for, though. An
“interpreter” (no part of it being co-installable) that can handle/load
arch-indep and arch-dep plugins/modules/scripts.
> I think the .so files in the perl package and the XS module packages
> can be usable without matching the architecture of /usr/bin/perl, as
> long as they match an installed libperl. Therefore XS module packages
> shouldn't Depend on the /usr/bin/perl package at all, only on the
> libperl one (probably via a virtual package like perlapi-5.14-amd64
> or something like that).
>
> Applications generally link against libperl because they embed a Perl
> interpreter and let the user run arbitrary programs on that. Offering a
> Perl interpreter without the standard library is not acceptable at
> all IMO. So if we make libperl5.xx co-installable with itself (which
> I understand to be the whole point of "multiarching perl"), we need to
> make the standard library co-installable with itself too.
>
> I also think we should strive to make the whole XS module stack packaged
> in Debian available to non-native arch embedded Perl interpreters,
> which means making them co-installable with themselves too.
So, the problem as I see it, is that there are two conflicting
requirements that were not (AFAIR) taken into account when we added
the M-A:allowed value. Those are:
* Having an interpreter program (perl or any external program linking
against the “embedded interpreter”) that at the same time can be
called, can load (possibly arbitrary) arch-indep scripts and load
(possibly arbitrary) arch-dep plugins.
- This case, by itself should be covered by the M-A:allowed value
(as designed).
- Dependencies are guaranteed to be coherent, due to lack of multiple
instances of the arch-dependent packages (XS modules, etc), as
only one instance matching the installed embedded interpreter will
be ever installed.
* A libperl (or libpython) shared library, arch-dep plugins (XS modules)
that link against such library, and the arch-dep interpreter's
standard library objects, all of which should be co-installable.
- This case, by itself is partially covered by the M-A:same value,
but it does not allow for foreign arch packages linking against
libperl if those packages depend on an arch:all package that ends
up depending on an arch:any package (assuming mapping of arch:all
to arch:native, as is currently the case). Example:
arch-native = amd64
,--> libperl:i386
prog-link:i386 +--> perl-xs-module-a:i386
`--> perl-pm-module-a:all -/-> perl-xs-module-b:i386
- Dependencies are guaranteed to be coherent, due to all of these
being arch-dependent packages, including the ones linking against
the libperl package. Assuming we keep arch:all packages mapping to
arch:native (which implies some programs linking against libperl
are not cross-gradable), otherwise the following broken situation
could occur:
arch-native = amd64
,--> libperl:amd64
prog-link:amd64 +--> perl-xs-module-a:amd64
`--> perl-pm-module-a:all --> perl-xs-module-b:i386
The second requirement is already problematic, but when trying to cover
both scenarios the situation just gets worse. The root cause of this
conflict seems to come exclusively from the XS modules, if there were
no such modules, then AFAICS both requirements could be fulfilled just
fine. I don't currently see a way to guarantee non-brokeness through the
dependency system. I'll expand on this below.
If we have the co-installable stack (libperl, stdlib, XS modules), and
a single perl executable, how can arch:all perl scripts guarantee that
when they get called their XS:modules will match the architecture
of the calling perl executable? Let's say:
,--> perl-xs-module-a:i386 (M-A:same)
perl-script:all +--> perl-xs-module-b:amd64 (M-A:same)
`--> perl:amd64 (M-A:allowed)
(In this example I'm assuming arch:all does not map to arch:native
any longer, otherwise the point of the co-installable stack
disappears as we need at some point of the chain a dependency
from an arch:all package to an M-A:same package. See the following
example for an alternative which just shifts the problem elsewhere.)
,--> xs-pm-wrapper-a:all --> xs-so-module-b:i386 (M-A:same)
perl-script:all +--> perl-xs-module-b:amd64 (M-A:same)
`--> perl:amd64 (M-A:allowed)
The dependencies in such case would be satisfied, but the script would
fail to run. The same situation could happen, even if perl itself was
cross-graded (or it might work if perl-xs-module-a:i386 is installed).
What makes this combination problematic are the XS modules, because
they need to be both co-installable, and might need to be depended by
arch:all script packages which just make use of them through the perl
interpreter (be it embedded or the perl binary itself).
On a non-coinstallable stack, a problem of XS-modules and M-A:allowed
marking is that, if they are switched from M-A:foreign (which can
happen in case a normal module is turned into an XS one), then this
will cause uninstallability as the module is already installed and the
reverse-dependencies might not yet declare their depenencies as pkg:any.
This could be alleviated by splitting the .so into a arch:any package,
and leaving the .pm wrapper around the .so in an arch:all package, but
that implies several new packages. One doubt I have is, is it common
(or even allowed) to provide XS modules w/o the .pm wrapper?
A possible solution that comes to mind, which would require waiting
for another release cycle, could be to add a new arch-qualifier so
that one could use dependencies like:
Package: perl-script
Architecture: all
Depends: perl, arch-indep-module-perl, arch-dep-module-perl:arch=perl
Or something similar, which would guarantee that the architecture for
arch-dep-module-perl should be the same as the currently installed
perl package. But then this looks like all kinds of ugly and explictly
verbose, but it's the cleanest I've come up these past days. For dpkg
this feels like it should be “easy”, I'm not sure how easy it would be
for frontends though. In any case wider discussion would be needed.
> My current experimental implementation of a new perl package split that
> tries to solve this puzzle is:
>
> - perl-base contains only /usr/bin/perl, Pre-Depends on libperl5.xx,
> M-A: allowed based on the earlier discussion (but M-A: foreign also
> seems like an option?)
> - libperl5.xx (M-A:same) contains libperl.so.5.14 and the part of the
> standard library currently in perl-base
> - perl (M-A:same or M-A:allowed?) contains the rest of the standard
> library (possibly keeping the split into arch:all perl-modules, but
> they could be merged if necessary.)
>
> I was thinking libperl5.14/amd64 would Provide: perlapi-5.14-amd64
> or something like that, and XS module packages would Depend on that.
Don't all XS modules already depend on libperl? In any case if they'd
need to depend on the virtual package, the architecture is going to be
taken into account as the one used when calculating satisfiability is
the one from the providing package (so there's no need for explicit
arch specific provides).
> This guarantees there is a libperl on the system that can use the XS
> module, but not necessarily that /usr/bin/perl can use it.
>
> Many Perl scripts and most XS modules would need to Depend on perl because
> they use those parts of the standard library. This seems to indicate that
> it's the perl package that would have the dual nature and be M-A:allowed?
>
> Does this make sense, or am I overengineering all this?
Unfortunately, as stated above, I don't think a full multiarchification
of perl (or python) is possible currently w/o making the dependencies
very flaky (i.e. allowing for breakage).
Thanks,
Guillem
Reply to: