[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Bits from the CD team: plans for debian-cd v3.0

On Tue, Jul 12, 2005 at 05:23:39PM +0300, Steve McIntyre wrote:
>At Debconf we've had a couple of very good discussion sessions about
>changes that are wanted/needed in debian-cd. Firstly we had several
>members of the debian-cd team thrash out what we wanted to do for the
>next version, then a second chat with some more of the debian-cd users
>to see what they would like us to do for them. I came to Debconf with
>some ideas of my own for discussion, and several of these other people
>have thrown extra things into the pot. Here's a summary of what we
>came up with; I'll follow up to debian-cd with more details. Please
>follow up there if you have comments or would like to get involved.

And now, the more detailed version:

 1. Re-structuring

    We need to refactor the code; the top-level Makefile is a mess,
    and there are far too many hacks in the codebase that make it
    impossible to maintain

    Some top-level wrapper scripts could make some of the common jobs
    much easier. At the moment, making a custom netinst CD (for
    example) is _hard_, even for the people using debian-cd every day.

 2. Tasks

    The current code to maintain the tasks that go on the CDs is
    messy. We must generate the task list each time, rather than use
    the pre-gen lists from CVS. An easy way to check that tasks have
    been fulfilled in a CD build would be very useful, like we
    currently try to do with debootstrap. Includes/excludes should be
    much clearer.

    Hooks to add tasks at the right place would be considered useful
    by CDD developers, to allow lots more flexibility to control the
    layout of CD sets.

 3. Dependencies

    I'm going to merge in the JTE patches, and add a dependency on the
    mkisofs-jte package. Also, I'm going to turn more of the current
    recommends/suggests into dependencies. It's not funny to build CDs
    and make a set that don't have boot graphics, or won't boot on
    mips, etc. On the Recommends/Suggests front, we need to check that
    the requisite packages are installed before starting a build.

    Also, we want to make the _packaged_ debian-cd more useful. At the
    moment, most serious users use the CVS version and their own
    patches, which is not ideal.

 4. Code management

    CVS is too limiting. After some discussion we think that svn is
    probably the best choice, with svk for patch management.

 5. More intelligence in debian-cd

    A debian-cd run should have more of an idea of what its target
    is. At the moment, we simply have to rely on the size of the
    target image in a few places. There are several places where
    knowing exactly if we're working on a DVD or CD would be very

 6. Actual layout of the CDs could be better

    At the moment, we guess how much space the docs/boot/tools
    etc. will take at the start of each CD run, then ask apt to fill
    up the available space. Then we run apt-ftparchive to generate the
    Packages and Sources files. This is not very accurate - a lot of
    the time during past releases has been taken up by manually
    juggling CD sizes to make them as large as possible without going
    over 650MB.

    A much better way would be to check sizes as each file is copied
    into the temporary tree, and Packages and Sources files generated
    by copying stanzas in to match. By keeping careful track of how
    much space the tree takes, once the tree gets to within a small
    margin of the target, we can start running mkisofs -print-size to
    check exactly how large the resulting image would be. It will even
    be possible to maximise the image size by simply running until a
    package/source file would take the image over the configured
    limit, then rolling back.

    This would also allow easier customisation of CDs - with some
    effort, custom CDs with extra packages should be feasible by
    adding the new packages to the list, and rolling back at the end
    of the build process to remove any trailing packages that no
    longer fit.

    An extra mode for image sizing (but _NOT_ image creation) may
    allow us to generate CD/DVD jigdo files directly from the archive
    metadata without actually reading in the .deb files at all. This
    will not be too hard to write support for, BUT will create jigdo
    files that do NOT contain MD5 sums.

 7. Multi-arch (and optionally source) discs

    The new layout method should also make mixed discs easy,
    e.g. single DVDs for i386, amd64 _and_ the source for all the
    packages. Simply add the appropriate arch binaries and sources as
    part of the process of filling the disc tree. It's also possible
    to make CDs/DVDs that boot on multiple arches, and that might be
    very useful.

    On the binary & source front, this would make it possible to
    create standalone CDs that meet all source requirements. A little
    extra effort may be needed to make sure that things like syslinux
    source make it onto the CD.

 8. Running without a local mirror

    A common complaint is that debian-cd basically requires a local
    mirror to be useful. This is not ideal for many end users, who may
    be trying to create small custom CDs and don't have the disk space
    for all the Debian packages needed. It should be feasible to make
    debian-cd work with remote mirrors, grabbing just the pieces it
    needs. This _may_ need custom work, or it may be possible to use
    one of several packages suggested to create a partial mirror. I'm
    actually leaning more towards custom code, perhaps even some
    tweaks to pipe the data into mkisofs as it is needed to save local

 9. Are business card ISOs actually useful?

    Compared to the netinst and other CDs, they depend much more on
    the software in the mirror exactly matching the code on the CD,
    which make them _much_ more fragile. Sizing them to fit on the BC
    CD media is also much more difficult. Do people still think the
    effort to create these is useful?

10. Signed release files

    The new functionality in apt for checking signatures on Release
    files is a good step on the road to better security. However, the
    CD builds by necessity have to generate new Packages and Sources
    files on each disc. This means that we have to also generate new
    Release files, which currently are not signed. Signing these would
    be good, possibly with a new key held on the machine(s) building
    official images.

    Similarly to the current apt code, if the Release file on the CD
    is not signed with a known-good key then the user should be
    presented with a message checking that they want to trust the
    CD. Customised CDs could contain new keys, then apt would ask the
    user to verify the fingerprint before adding it to the trusted

11. CD errata package

    After the recent problems with the first attempt at the sarge
    CD/DVD release, lots of discussion happened about ways to make
    CD/DVD mistakes less likely to happen. More testing always helps,
    but even then it's possible for bugs to slip through.

    Phil suggested that there could/should be an errata package
    created on our website somewhere (suggested:
    release.d.o/CD/$dist/errata.deb). As one of the last steps as a
    machine installs for the first time, it should ask the user, then
    check if there is an errata package available. If so, this package
    will download and (where possible) apply fixes to the new

    Another simpler option would be to simply have an HTML page
    listing the errata, and instructions for the user to follow to fix
    their installation.

    Alternatively, an errate package could be pushed into the archive
    so that all installations might get it, not just those from CD.

    My own personal favourite would be the simple HTML file, but I
    think some discussion of the options here would be useful.

Steve McIntyre, Cambridge, UK.                                steve@einval.com
Is there anybody out there?

Attachment: signature.asc
Description: Digital signature

Reply to: