Re: Upstream packaging (was Re: Derivatives, MongoDB and freezes)
On 2013-04-24 12:43, Guillem Jover wrote:
Hi!
On Sat, 2013-04-20 at 11:05:29 -0700, Clint Byrum wrote:
[...]. IMO this is why upstream packaging should be
embraced and enhanced rather than focusing on dpkg.
I'm not sure if you refer to the tool here, or to the packaging work,
doesn't change much anyway.
I'm referring to delivering software in general, most of which falls
into a few categories which are not "a programming language",
"plumbing", and "system development tools".
I once worked on the 'pkgme' project for Ubuntu and there have been
others, but never followed through. The idea was just to build
debian source packages from upstream sources. Upstreams should be
able to release a package which serves their needs, and Debian
should be able to consume these almost directly.
Respectfully, I think you've entirely missed the point of a
distribution
(and sadly I'm seeing this trend more often than before now, with all
the hype around app stores and similar...). Packaging and maintaining
a
consistent and unified distribution cannot be delegated to upstreams
(and I'm talking as an upstream developer here too), because that
entails
a bit more than slapping some files somewhere, tarring the thing up
and
calling it a day.
Indeed, there is a great deal of hard work in putting together an
operating system. But for the bulk of software, things like MongoDB
included, I see very little point in distributions spending a lot of
time tweaking and poking and prodding the software to work into a grand
policy.
For the most part, developers ship and test a good tarball that builds
with some pretty standard and easy to detect options. The bits where
that doesn't work ought be fixed upstream, and I know sometimes they
are, but in the mean time, the distribution maintainers (myself
included) spend their precious time conforming to the broken upstream,
instead of the other way around.
Building a nice distribution requires a high-level view and QA of the
entire system; requires curating sane namespaces, be them on the
package/project names, on the version strings, on the filesystem (by
avoiding file collisions, using alternatives or diversions), on
exposed
programming interfaces, etc; requires making sure a diverse set of
programs interact correctly with each other; performing security
updates; ideally keeping single runtime versions; doing global
transitions to use other or newer runtimes, programs, libraries or
packages; an unified way to build from sources to cope with the
endless
and interesting different upstream build systems; porting and building
for different binary architectures, not just what upstream might have
around; documentation; translation; tagging stuff with metadata to
allow for easy searches; excision of the embedded code copies cancer;
even stuff like how the Delete and Backspace keys should behave;
sets a qualify bar for upstream projects, stuff of low quality will
not be accepted most of the time; license checks; etc, etc...
All of the things you mention are huge accomplishments, but the scope
is what I am suggesting has gotten out of hand. Do we really need "a
high level view" and "QA of the entire system" for MongoDB? It is a
daemon and some client tools. If I install it from tarball, I know this
is shocking, but it actually works fine and doesn't interfere with
anything else. If it does, I will complain to upstream. My suggestion is
not to stop packaging, but to shift focus from "make an awesome package"
to "make an awesome upstream" that results in an automatically generated
awesome package. Debhelper and many of the other tools definitely help
with this, but we can do more.
And all this, upstream will never be able to provide (at least not as
defaults), because each distribution has its own policies, some are
better, some are worse, and some are just different. In general I'd
never trust the packaging produced by an upstream for a distribution
they are not involved in. But most of the time there will be no such
packages for the desired distribution anyway.
Debian wants to provide end-to-end generalized tight integration. Thus
packages lagging behind upstream. I'm suggesting that this tight
integration ideal actually *limits* the scope of Debian.
Although there's been work on creating distribution-neutral specs that
some upstreams have picked up, there's always going to be something
new,
the specs will just not cover all needed things, the specs might not
be liked by some/many people, or might not have been fully adopted by
all upstreams.
Right, and Debian maintainers can, and will keep bending over backward
to get all the upstreams into Debian. That doesn't mean its a good use
of someone's time.
A distribution (any, most) is the gel that binds and gives an unified
and coherent shape to the software ecosystem. An app store is just
like
a scrapyard, you might find magnificent and isolated gems there, but
most
of it will probably be junk, or not combine with the other scrap
parts.
App store is your term btw. That is not at all what I suggested.
Also if people thought that distributions are unneeded, then the
amount
of them would reflect that, or start decreasing, which I'm not seeing.
Distributions will exist as long as there's FLOSS, because by its
decentralized nature, there's no single coordination point; people
who are distraught by the amount of distributions or by their mere
existence might probably only be able to find peace in closed-systems
instead, with their centralized control points.
Nobody is distraught. The problem statement was something like
"distributions lag upstreams, how can we solve this?" and I'm suggesting
that its kind of already solved with upstreams who use their native
packaging format (i.e., autotools, pypi for python, or maven for java).
Its just that Debian has such strong policies that we are unable to
consume said packaging, because the upstream packaging is less
expressive. So, we have to lag while we QA all of our specialness.
Where Debian's efforts should be focused is on things like license
verification and helping bug reports and fixes get to upstream.
So basically, getting rid of most of the fun stuff and turning it into
a lawyerish play-ground and support center... I'd venture to say, not
the most attractive work for most people here if it was the only
thing to be done, which we do because we think it's important non the
less.
I am suggesting that there are a lot of things that are much more fun
than conforming software to Debian policy. I'm also suggesting that
these things don't need to happen in the context of Debian. Its not so
much that Debian should only care about "lawyerish" things. However,
going through those steps for people is something that I feel Debian
uniquely cares about.
Reply to: