[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: [Multiarch-devel] arch-specific dependencies and M-A: foreign

On Sun, Apr 19, 2015 at 10:56:48AM +0200, Johannes Schauer wrote:
> Quoting Guillem Jover (2015-04-19 01:38:35)
> > On Fri, 2015-04-17 at 15:39:14 +0200, Johannes Schauer wrote:
> > > Yes, if bar:amd64 would not also really provide bar:i386, then it should not be
> > > M-A:foreign.
> >
> > Ah, I guess this might be where the different interpretations come
> > from. The way I see it, a M-A:foreign has a defined architecture,
> > which is relevant, because you might need to run it, and it does not
> > really provide :any, it is just able to satisfy unqualified
> > dependencies. When you explicitly provide another arch then *that*
> > counts as actually providing that arch.
> Okay, I see now where our understanding differs. It seems to be a subtle
> difference of how to think about this.
> Suppose there is package foo which is m-a:foreign and arch:amd64.
> Now pkg1 is arch:armel and depends on "foo" and pkg2 is arch:mipsel and depends
> on "foo" as well. Since pkg1 is arch:armel it will require foo to be arch:armel
> too, except that foo is m-a:foreign so foo in arch:amd64 works as well. Same
> goes for pkg2 but for mipsel instead of armel. So while pkg1 would normally
> require foo to be armel and pkg2 would normally require foo to be mipsel, since
> foo is m-a:foreign it is able to satisfy both as if it would provide foo:armel
> as well as foo:mipsel.
> This is how dose3 is able to treat m-a:foreign packages. Internally it converts
> the problem into a cudf representation and since cudf does not understand
> architectures, it lets a m-a:foreign package provide itself for all foreign
> architectures.
> This is where my way of thinking about this matter comes from.

That is what APT is thinking, too, for pretty much the same reasons.
I agree also with the explanations and have the same questions further
down in that mail, so I skip all those here.

There is one thing confusing me which I have to highlight through:
The documentation of dpkg mentioned above says that for e.g. Depends
"an optional architecture qualifier can also be appended to the package
name […], but the default is […] the binary package architecture".
(the text presents the derivation from the default for the other forms
of relations, so I hope I haven't created a reductio ad absurdum)

That sounds for me like
Depends: foo
Depends: foo:amd64
describe the exact same dependency relation as the first one is just
omitting the default value (which is how apt is seeing it). Yet, Guillem
says in the quote and also in the reply to my initial mail that if foo
is M-A: foreign (and we try to install it in i386 flavor), it is
satisfying the first, but not the second as this foo:i386 "does not
really provide :any, it is just able to satisfy unqualified

> What possible situation could there be in which you'd like a m-a:foreign
> package of a specific architecture? If it is a data package containing
> different data per architecture, then it should probably not be m-a:foreign. If
> it's an executable with an arch-independent interface, then why would you want
> to depend on it with an explicit arch? In that case all you care about is that
> you can somehow run the tool (which is a different problem).

As I briefly mentioned earlier, this topic came up in the context of
debug symbols and those traditionally depend on the package they provide
debug symbols for.

This is a different type of dependency relation. We usually have a
"A needs B" meaning A expects B to provide a certain interface, where
B can decide if it is able to provide this interface irrespective of the
architecture it itself was build for or not (and hence declare itself
M-A:foreign or not and if not can decide if it can at least be
co-installed with its architecture-different siblings with M-A:same).

But debug symbols A do not care for the interface B provides to the
outside. They care about the inside. A isn't usable to debug a provider
of B, only the real package B can be debugged with it and it can't be of
any architecture, but must be the exact same one (even assuming partial
architectures debug symbols for i486 aren't gone work for i686 in
general even through a i686 system can use a library build for either
i486 or i686 interchangeable)

A slightly weaker version of this might be applications with plugins,
where the application provides an architecture independent interface to
the outside, but a plugin requires an inside connection. Note that an
application might provide this inside connection architecture
independent as well. Maybe it doesn't. And maybe, that changes between
versions of the application.

Such applications we mostly dubbed language interpreters so far and
invented M-A: allowed and its companion :any for them to have a way
of saying this could be either way, with a default on M-A:no.

I wonder if it makes sense to add the inverse companion ':same'¹ to
M-A:foreign, so that you can write "A depends B:same" to ignoring the
fact that B is M-A:foreign much like you would write "A depends B:any"
to highlight the fact that B is "allowed" to be M-A:foreign.

That would (in by book) nicely solve the case of debug symbols and other
"inner world" enhancing packages for a M-A:foreign package without using
specific architectures for it (which is harder to write down in metadata
as that is then a dynamic field, rather than statically expressible)
freeing them from the :same meaning they try to emulate in some eyes.

(¹ I have the weird feeling of having that seen already somewhere.)

> I want to also take this opportunity to talk about the second point. Consider
> the following situation:
> Package: pkga
> Depends: pkgb:any
> Architecture: amd64
> Version: 1
> Package: pkgb
> Architecture: amd64
> Version: 1
> If the native architecture is amd64, then currently apt and dpkg will refuse to
> install pkga because the pkgb:any dependency is not satisfied. But why? As the
> dpkg comment above says, pkg:any dependencies are satisfied by a package of a
> *different* arch if the target package is m-a:allowed. The target package is
> not m-a:allowed here but it is also not of a different architecture than pkga.
> So why can neither apt nor dpkg install pkga?

The spec says: "[…] be satisfied by a package of any architecture,
so long as the python package declares itself as Multi-Arch: allowed".
That is easily read as "if it isn't, then no architecture satisfies it"
even through that isn't explicitly written here. I at least have read it
as such in the past, but now…

I agree with you here. I would even go a bit further and declare :any
also satisfiable by an M-A:foreign package of any architecture. This
would allow moving an M-A:allowed package to an M-A:foreign package (or
back) without flagdays or similar.

I realize through that this obsoletes M-A:allowed (which isn't
necessarily bad) and instead moves us to a world where a package
declares if it should be considered M-A:foreign by default or not and
a package depending on it can override this default with :any, the
potential :same or stick with the default.

> > > Now, imagine instead of 'bar' we had three other packages which provide
> > > 'bar'. Some of those providers are M-A:foreign, some not. What now?  And to
> > > turn this up to eleven: Lets change 'Depends' to 'Conflicts' …
> > 
> > I guess I don't see the problem, either the metadata is wrong or the
> > packages will or will not have satisfiable dependencies.
> I guess the problem here is what a specific piece of metadata is supposed to
> *mean*. Only then can we answer the question whether the metadata is wrong.
> Currently, the documentation on this is sparse, resulting in the many
> disagreements between dpkg, apt and dose3 when it comes to multiarch (see my
> other email).

Indeed. Especially as the documentation which exists tends to contradict
other parts of the puzzle, like MultiArchSpec specifically denying the
existence of foo:<arch>.

Wookey already suggested that we finally change that… I think the main
problem with it is that noone knows all parts. While I tend to have an
opinion on how package managers should handle MultiArch on a resolver
level the parts actually happening on the disk, being it dpkg refcount
or simply the <multiarch-triplet> is beyond me (Beside that I am
a total noob in terms of spec/policy writing…). So nobody does the first
step and waits for a non-existing know-it-all master to show up, right?

Maybe we, the resolver triumvirate, should just declare ourselves the
ultimate authority in dependency evaluation, take the current
MultiArchSpec as a base and go through the results of Johannes test
until we arrive at a spec containing all the things to which we
hopefully can all agree to… How to get that one translated into policy
would remain the topic of another day. That is potentially easier
through if all the major players agree and strictly better than the
status quo.

I will be at DebConf and very likely also Camp, but I am not sure its
the right setting. Ideally we would have at least a rough sketch by then
so that the last remaining potentially controversial things can be dealt
with in person (assuming I am not the only one at DebConf/Camp).

Discussing M-A seems to involve a lot of (made-up) examples juggling
qualifications left and right which I don't think is very well suited
for talk. Especially if we talk about ~7700 examples in one session…
(oh boy, M-A has grown fast…)

> Let me also point out that I don't have a very strong opinion about this topic
> and just want to weight the arguments I can come up with against others. I find

Yeah, mine isn't set in stone either, which is why I wanted to discuss
this particular issue (and now the potentially 7700 others) first before
crying "bug in dpkg/apt/dose" because it doesn't work as I expect it.
I see it as "could work either way", I would just prefer to have the
option which is easier to understand – the question is just which one
that might be and hence I made a semi-poll.

> it way more important that all tools agree on the same thing than to discuss
> about corner cases without real-life examples (which I do not see existing).

I hope the example(s) above help a bit. Some situations are surely a bit
obscurer than others, but stuff like the interpretation of :<arch> and
M-A:foreign in general seem pretty important to have a common ground of
operation to then look at the obscure corner cases united.

> As far as dose3 is concerned, me (and as I understand it also Pietro) have no
> problem with just following what dpkg does. I think it's more important that we
> pick one way to interpret these things and implement the same behaviour in all
> tools resolving dependencies now. If these decisions turn out to become
> problems, then they can still be changed in the future.

I am not sure we have the liberty to change in the future. At least in
very radical ways as systems which were okay in release are suddenly
with the upgrade to release+1 horribly broken and dpkg/apt have to deal
with that somehow – which means the release-versions have to deal with
that somehow, at least for a moment. That could be a huge pain I would
like to avoid happening. I think with the three tools we cover enough
ground to get a solid foundation. At least we should try.

Best regards

David Kalnischkies

Attachment: signature.asc
Description: Digital signature

Reply to: