Re: Seeking feedback on a meta package builder
Hi Raphaël,
On Tue, Jun 08, 2021 at 12:05:44AM +0200, Raphael Hertzog wrote:
> The point is that if you have to look behind the abstraction to do some
> other related tasks like selecting the worker, then using the abstraction
> to actually execute the build doesn't bring you much, just supplementary
> issues if the assumptions that you make no longer match the assumptions
> implemented in the abstraction.
You do have a point here. My focus definitely was on making it easy for
the one issuing a build. debusine seems to assume both roles here.
I felt that worker selection (let alone worker addressing) would be so
inhomogeneous that covering it in a general abstraction does not seem
feasible to me. That's why I left it out.
With "look behind the abstraction", I think that you mean that debusine
would have to implement the mdbp api to perform worker selection. While
that would be possible indeed, I don't see this as required though.
> And it seems to me that aiming for "it works out of the box without having
> to configure mdbp (if you already have sbuild schroot or pbuilder
> tarball)" is a nice thing to aim for. So somehow it would be nice if your
> various mdbp-* implementations could report whether they are able to
> perform a given build.
This is mostly true. One of the ideas is that when you encounter an
application or service that uses mdbp, it should be easy to direct it at
your existing builder. Some backends will need configuration though, so
a magic autoconfiguration seems infeasible to me. For instance, the ssh
one will need a remote host and for mmdebstrap you most likely want to
specify a local mirror.
I do wonder though, in what kind of situation would you want to merely
know whether a backend can perform a build as opposed to just attempting
it and being able to tell backend issues from actual build failures
apart.
> It would not be enough for debusine yet (for debusine I need to be able to
> export data from the worker and then make that decision on the server and
> not on the build machine itself) but it would be nice step forward for the
> local use case where you want "mdbp" to figure out alone which backend to
> use based on what the user did setup earlier.
Yes, that makes sense. I note that the decision is meant to be made on
the build-issuing side for mdbp as well. If you use the ssh backend, the
relevant command might look like this:
mdbp-ssh someserver.somesite mdbp-mmdebstrap --mirror http://mirror.somesite/debian
A user would append their build object to the command line and run it.
This is not meant as a recommendation for debusine to use mdbp-ssh. We
already agreed that it is not a good match. It is meant to illustrate
that coming up with an automatic backend choice is not obvious though.
> That said if you are willing to complexify your mdbp abstraction to
> cover this use case, then there might be room to actually use mdbp in
There is a trade-off involved here. The reason we're talking now is that
I do want to consider and enable other uses even if that adds
complexity.
> debusine. I would still be annoyed by having to interface with CLI calls
> to make basic requests like those and I would hope that we could define a
> Python API instead.
Let us compare this to gettext. gettext also is an interface, one that
exists in multiple languages though. There is a CLI interface, but for
many languages there are individual bindings. The same can be done here
without loosing the gist.
In fact I already had to do this. The CLI uses arguments, stdout and
files. The latter tend to work badly over ssh, so I already added an
alternative API called mdbp-streamapi that only uses file descriptors to
interface with. The mdbp-streamapi command just converts this
alternative representation back to the "normal" CLI.
Sure, we can add more APIs if needed. The one thing that I'd like to
keep though is for them being compatible or convertible. Being able to
plug them into one another with ease.
> This abstraction definitely makes sense to me. Before looking closely
> at your build_schema.json, but after having looked at your mail here,
> I wrote this as a try to go in your direction:
> https://freexian-team.pages.debian.net/debusine/devel/ontology.html#task-packagebuild
Great. Maybe that's the level where we can make best progress?
> Feel free to submit merge requests to update my document to more
> closely match yours (many entries are missing).
I've taken the liberty to rather open a discussion issue at
https://salsa.debian.org/freexian-team/debusine/-/issues/10. Hope this
works for you. Bringing debusine's PackageTask and mdbp's build_schema is
the thing I'm really after. If these match, converting between the
relevant APIs should become a simple matter.
> Yes, the "command line interface" does not really help for debusine,
> unless you extend it to cover the requirements I explained above. And even
> then I don't really like to have to run an external executable to be able
> to answer a question like "can this worker handle this build request".
Agreed.
> On the opposite, the fact that debusine will expose a "PackageBuild" task
> matching your abstraction means that it will be trivial to write
> mdbp-debusine.
Also agreed. Being convertible is the thing that counts.
> PS: I already hate the "mdbp" name after having it typed so many times.
I'm not attached to either. Any suggestions?
Helmut
Reply to: