[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Thoughts about tdebs

On Tue, 02 Dec 2008 17:51:30 +0100
Thomas Viehmann <tv@beamnet.de> wrote:

(I'm subscribed to the dpkg list, Thomas, thanks for the prompt though.)

> Raphael Hertzog wrote:
> > I just read the specs drafted at the meeting this WE and I'm not
> > really excited about the prospective implementation of TDebs.
> Thanks for your appreciation and taking the time to comment.

I've been wondering for a while about just when this should get more
people involved. So far, it has been very gradual and I'm very
cautious about taking this to a high volume/high membership list like
debian-devel until Lenny is released. It's fair enough to discuss it
here (and on debian-embedded).

> > IMO we're heading in the wrong direction by focussing only on
> > translation and by special casing everything to meet this expected
> > usage.
> It is a very sizable special case, though, and growing thanks to the
> tireless translation efforts within and beyond Debian. This is why the
> consensus at the meeting seemed to be to get this right even if it
> means having to implement some special casing.

Raphael - I think you've missed some of the purpose behind TDebs and if
that is a fault of the current draft TDeb spec, then I would value
your comments on how to fix it.

TDebs need to remain focused on l10n support to remain useful to the
i18n teams and to the embedded developers.

If the idea becomes too generalised, I would not be able to use it as
the basis for Emdebian TDebs which are a level on from Debian TDebs due
to scalability issues.


If the idea becomes separated from translations, automated creation of
TDebs by translation review teams would be compromised, difficulties in
re-assembling the source to ensure that subsequent uploads retain the
intervening translation updates become even more complex and the drive
to deliver TDebs starts to wane. (Personally, I am also not at all
convinced that ftp-master would consider partial debs as a viable

A key point of agreement with ftp-master is that the TDeb can be
updated entirely separately from the rest of the package - this is
absolutely essential to all concerned. If the idea gets diluted to
include security updates (which almost inevitably change the compiled
binaries of any compiled package involved in a security bug) then this
update method becomes impossible and we start talking about binary
patches or mass-replacement of binaries by partial debs. I find that
idea more than a little insane. ;-) It would be a sizeable regression
even from where we are now with binNMUs and source NMUs. This is about
removing the code churn from i18n updates pre-release, it is about
preventing translation updates during a release freeze from
ever rebuilding any binaries.

However, updates to existing packages is only half the story - the
drive for TDebs has come from Emdebian. Emdebian must have TDebs in
order to meet any of the objectives for Emdebian itself. Partial debs
are not a solution, embedded systems need intelligently handled
translations and lots of smaller packages. The Emdebian solution
for TDebs cannot work within Debian unmodified, everyone accepts this,
but the Emdebian requirements do need to be achievable directly from the
Debian implementation.

The current specification supports this - partially via the premise
that TDeb updates only contain changes to translations and partially
because the individual localisations can be easily split out to create
the extra packages for Emdebian (as described on the Emdebian website
at the link above).

> > I guess you understand better the idea by now. This needs more
> > thoughts but I wanted to share it so that you can think about it
> > too.

It certainly does need lots of thought - but this isn't about splitting
the archive or just making uploads easier for certain kinds of
packages. This is about making i18n and l10n genuinely functional
within Debian. It is also about bringing the (stable) technology of
translation packages out of the embedded world (OpenEmbedded and, from
that, Emdebian) into mainstream Debian and all the scalability issues
that result.

Emdebian can afford to have over 40 TDebs per source package (we can
afford to have over 40 TDebs per source package per architecture too
and already do so).

> > I'd like to suggest something simpler: partial .deb. The idea is
> > that we would provide debian packages (.pdeb?) that contain only
> > some parts of the original package (the part that we want to
> > update), in our case the translation. Each partial package would
> > state on which original version he can be applied and would have an
> > identifier that define the part that it touches (with a version
> > number so that we can have several updates of the same part). We
> > could apply updates coming from several partial packages (security
> > update, translation update, misc data update, …).

There are use-cases for such things but I am not the one to take on
that task. My motivation is translation, building from the embedded
implementation. As I see it, partial debs are not useful within Debian
but for deployments of Debian - e.g. I've been asked before about kiosk
type implementations where there is a need to only update those parts
of the package that have actually changed in the latest release - many
Debian revisions change only a single file and many of those have
absolutely no effect on the compiled binaries or actual functionality
of the package. This has little benefit for Emdebian - what we need are
smaller packages in smaller repositories containing everything
necessary and no bloat.

Actually, from my perspective, your idea is far from simple and seems
to be a lot more complex than anything so far agreed with ftp-master.

I think this would be better handled within what we have by deployment
of dpkg classes to allow arbitrary selection of package components.
That is not the same as the locale root components within a TDeb, your
suggestion is far more general and needs a different solution. TDebs
can use dpkg class support once available but the tdeb is a special
case (just as a udeb is a special case).

In essence, it is the old problem of enabling a query interface (which
file do you want from this package?) and always getting a different
answer depending on which user you ask. Maybe partial debs would be
useful to other users as a layer on top of dpkg classes but partial debs
would not be suitable for what we need from TDebs.

> >From what I understand of your idea, you seem to think about
> translations mainly as updates. While it is one of the goals to enable
> translation-only updates, it is not quite obvious to me what your
> proposal has to offer in terms of splitting translations out of the
> debs, which is an explicit goal. Also, the proposal specifically aims
> at limiting the amount of data that the archive and apt have to
> handle.
> I'm not quite sure how useful the notes about design considerations
> are, but you could always ask Neil whether they are in a shape to
> publish them for further discussion. It seems that at Casar de
> Cáceres we found quite a few corner-cases that had not been
> previously considered.

I've tried to include all the notes about design into the specification
but there has been little work on comparing this with the original TDeb
proposals put forward by Eric Petrisor. (AFAIAM, Eric is happy with the
new model).

If there is anything in the specification that is unclear, please let
me know. In particular, if there is a need for more detail relating to
the background, purpose and motivation behind TDebs.

You might also be interested in the slides for the presentation I made
at FOSDEM last year on TDebs:


> With some distance, it might well be that we could generalize (at
> least parts of) the tdeb-implementation to class support (with
> different members of the ar archive per class[1]). We would still
> want to separate them out, but that might be a good way to arrive at
> a better mechanism.
> In summary, yes, there might be room for generalization, but taking
> your suggestions as an indication, maybe part of your dissatisfaction
> with the direction that the tdeb proposal takes stems from a
> disparity in the goals that each is concerned with.

Within Emdebian, we want to use dpkg class alongside DEB_VENDOR to
provide more flexibility in how Debian packages are assembled. 

I certainly want to have class support within tdebs (to make it easier
to remove translated manpages that are useless in Emdebian).

Right now, I'm concentrating on what changes are necessary within
debhelper and dpkg to implement TDebs based on the agreement with
ftp-master for what is actually achievable and with i18n for what is
actually usable - whilst still retaining the functionality that is
necessary for TDebs in Emdebian.


Neil Williams

Attachment: pgpQW6ap5rXPM.pgp
Description: PGP signature

Reply to: