Re: multistrap fixes
+++ Neil Williams [2010-08-20 23:53 +0100]:
> On Fri, 20 Aug 2010 14:52:16 +0100
> Wookey <wookey@wookware.org> wrote:
>
> > +++ Neil Williams [2010-08-20 09:17 +0100]:
> > > Please put these into a new package - this is how the other
> > > configuration file groups are expected to be handled, e.g.
> > > debian-live or custom hardware / derivative distribution groups.
> >
> > Are you sure it's worth it for a few hundred bytes of configs?
>
> Absolutely. I do not want multistrap tied to Ubuntu release cycles. (I
> don't want any Debian package I maintain to be tied to Ubuntu release
> cycles.) It's a native Debian package and I don't have time to support
> it in Ubuntu. The changes are too invasive.
This doesn't make the package tied to ubuntu release cycles. It just
means it has some extra useful configs in it. Nothing makes us release
more often than we would anyway. And nobody is asking you to do
_anything_. I can look after a couple of extra config files if
necessary. How can two extra config files be 'too invasive'?
> > Debian's Debootstrap has ubuntu configs in it. I think multistrap
> > should too for the same reasons.
>
> That is down to d-i, it wouldn't be my choice.
Well, if I can find somewhere else to put them then I'll put them
there, but currently I don't see another sensible place.
> > > > 2) Multistrap does not deal correctly with 'flat' source URLs, as
> > > > produced by apt-ftparchive.
> > >
> > > Has this been tested with the current version in SVN and with
> > > explicitsuite=false ?
> >
> > No, but the point is that it should still be possible to use flat URLs
> > with explicitsuite=true because you may have other stanzas where you
> > need to be explicit about the suite.
>
> Depends if apt itself supports that combination.
It does. I've tried it (quite a lot now).
> > explicitsuite=false applies to _all_ the stanzas and you might not
> > want that. The flat URLs are 'special' with regard to suite/component
> > and should be dealt with acordingly.
>
> ... and Ubuntu specific.
No they aren't. This has nothing to do with Ubuntu. I just happened to
notice the problem whilst doing some testing there.
aptftparchive-style URLs are the same wherever they are used, and
multistrap can't use them without my (small) patch.
> I don't have a problem with specifying that
> repositories for multistrap need to be in a particular form, at least
> until after Squeeze.
But apt supports these and they are quite common, and the idea is that
multistrap doesn't get in the way of the stuff apt supports, and I've
done the work to support them so why not just put in the support? I'm
talkoing about putting it into SVN, not necessarily doing another
release for squeeze - that would require talking to the release
managers.
> > Exactly how we decide whether we are doing a chroot or an image is a
> > good question. (yet another) explicit config option is probably the
> > right answer. However the (temporary) nobbling part should probably be
> > automatic according to this rule: "if doing the configuring in a
> > chroot, either natively or via qemu, then things must be temporarily
> > nobbled. If the image is intended to be a real image then this
> > nobbling must be reverted before running live.". Anyone disagree?
>
> I don't think this should be done in multistrap itself. I am also not
> convinced that the rule is that black-and-white. Packages can look
> after these things themselves, it doesn't have to be something done
> from outside (which, IMHO, is usually the wrong way to do this).
Maybe. I thought that to start with, but after thinking about it for a
while I changed my mind. In principle every package could contain the
same support that udev does for detecting chroots and do different
things accordingly. But that's a lot of packages which start services,
and it is both more reliable and a lot less code to deal with
sandboxing the environment in the chroot-creation software. This
approach is already proven in debootstrap.
We could try and find a good way for each package to do the chroot
detection and for them all to the DTRT, but that's a long-term job. We
might well have to implement something in multistrap in the meantime
anyway. I guess if init could tell whether it was being called from a
chroot then it could deal with this itself - that might be worth
investigating.
The current multistrap support is nothing more than a chroot.sh script
that a config can choose to run, so it's entirely non-invasive.
> > Exactly how we achieve that probably requires some more discussion.
> > But if we can agree the objective, that would be a good start.
>
> If this is upstart-specific, then I cannot allocate time to that until
> after Squeeze is released.
>
> > Underlying this is the idea that multistrap can be used to make
> > chroots as well as real syustem images. The inital focus was real
> > system images, but I see no reason not to support making chroots too
> > unless it causes fundamental breakage.
>
> For Debian chroots, I haven't seen such problems.
It's just that we haven't been doing anything that actually breaks
things in ways we've noticed. But in fact every chroot is probably
setting the host sysctl configs to the defaults set in the chrooted
package (which isn't desirable) and anything that runs daemons (ntp,
apache, cron, at, nfs) is quite likely to do bad things to your
system. I think it's just that we've mostly been making very small
images that means we haven't realised the implications of this.
> > > > The problem with this is that nothing now ensures that the
> > > > specified packages actually come from the specified repository,
> > > > even if explicitsuites is set.
> > >
> > > I cannot accept such a patch.
> >
> > Why not? I think you misunderstand. I don't mean that explicitsuite
> > stops working because this functionality exists. I mean that _for this
> > deb source_ you can't use the $package/$suite syntax because it is
> > meaninglyess and apt will fail to find the packages. My patch allows
> > explicitsuite to still work for all stanzas with 'normal' deb-source
> > lines - indeed that's the point: if you just turn explicitsuite off
> > then you don't need the patch.
>
> The patch itself still needs improvement for the wildcard issue and
> there are other unresolved issues resulting from the loss of the
> "suite" identifier in the rest of the code.
What issues? Suite being null already happens when suite isn't set to
anything explicit, doesn't it? Do you mean the way simulute whinges
about unset variables? That does need fixing, I agree.
> I'm suggesting that, at the moment, flat URL's might just be
> unsupported - mainly because I don't have time to implement, test and
> support such a large change.
You don't need to implement it. I already have. Or test it, I've done
plenty of that too. And it's hardly a large change. I wasn't planning
to start a long thread about this stuff. I could just have checked it
in, but thought it polite to check for gross incorrectness. I really
don't understand why you are resiting this simple and useful fix. I'm
not asking you to _do_ anything.
> This is completely the wrong timing for me. Multistrap in Squeeze is
> fine
So long as you don't need to use it with an aptftparchive repo, or
install any daemons, (or create a maverick image), or use two
stanzas one of which is a subset of the name of the other, and only
the longer one is selected. But like I said - I'm not asking you to
make a new release (although I think that last bug probably deserves
one), or indeed do anything except check my patches aren't egregiously
wrong. Which I think we've done.
> Ubuntu support is a significantly lower priority than simply getting
> Grip updated and in a decent shape for the upcoming Debian release.
Of course. But that's only the config files - the rest of it applies
exactly the same to Debian.
> Release time was always going to be the worst possible time to consider
> major structural changes to packages.
And I haven't proposed any 'major strucutural changes' - trying to
present these patches as such is silly.
> > > > + # don't set suite or component if URL is of
> > > > apt-ftparchive trailing-slash form
> > > > + if (($mirror =~ /.* .*\/$/)) {
> > >
> > > That '.' must be escaped otherwise it is a wildcard.
Right, but they are wildcards, so that's correct.
> I'm not sure why you have two wildcard matches in the first place.
Because the format of an aptftparchive URL is
deb <URL> <path/>
So I'm making sure I catch paths ending in '/' but not just URLs
ending in '/' which are acceptable as normal format. (This could happen
if a simple-minded regexp for /.*\/$/ was used).
I don't match the start of the line to allow for things like the
arch/vendor qualifier in lines like:
deb [arch=armel] <URL> <Path>
(which I am pleased to see works just fine in the exisating codebase
without any changes)
> I'm
> also not convinced that the reliance on a trailing slash is itself
> definitive for this situation.
So far as I can tell that is correct. In practice I have only ever seen
the path './' used, but in principle other paths could be used.
This regexp could be made more explicit - is there a perl shorthand
for 'things valid in URLs' or 'things valid in posix paths'?
> Can you put all your changes into a branch so that I can test it
> properly? (Having said that, I won't have time to do that testing for
> at least a couple of weeks.)
OK. I'll do that. If there is anything you can think of that might
break I'm happy to do some testing.
> I am concerned about how the changes impact on existing support and
> what happens internally when the suite is undefined. Also, I just have
> to concentrate on other stuff during the release process - changes to
> multistrap (or other packages) for support outside Debian/Emdebian just
> won't get much of my time for a while.
As I said - all of this patch except the new configs applies just the
same to Debian/Emdebian as any other distro.
Wookey
--
Principal hats: Linaro, Emdebian, Wookware, Balloonboard, ARM
http://wookware.org/
Reply to: