Installing from scratch with debs
I recently got a new hard drive and wanted to reinstall Debian on it.
The machine tracks unstable, so I figured it would be easier just to
download all the relevant .debs and dpkg -i them into the target file
system, than make boot floppies for Potato and then re-upgrade.
Installing debs onto a bare system proved to be harder than I'd
thought. Obviously you start with the Essential packages. But the
moment you try 'dpkg --root=/mnt --install libc6_2.2.1-1_i386.deb' you
discover five or so serious problems:
1. dpkg won't do anything at all if /var/lib/dpkg doesn't exist and
doesn't contain two files (status, available) and two directories
(info, updates). They can all be empty, but you have to create
them by hand. I don't see why dpkg doesn't create these if they
don't exist.
2. Preinst scripts won't run chrooted into a bare file system; there's
nothing to run them with. I tried dpkg-deb -x'ing enough packages
that a minimal preinst could run - bash, libc6, fileutils - only to
hit the next problem:
3. All the Essential packages have incomplete dependency lists. This
isn't surprising; their dep lists are intended to make upgrades of
complete systems work. Their true dep lists are much longer, and
contain lots of cycles. You rapidly get into a tangle where A
won't unpack because it pre-depends on B, but B won't configure
because its postinst expects A to be present. They also have a
fair number of dependencies on packages which are not Essential,
and you have to get those too.
4. dpkg-deb -x doesn't update the status file or extract the control
files, which means dpkg has no idea that certain packages should be
hit by -a --configure.
5. makedev is not Essential, and not much depends on it, but it's the
only package which will create /dev - if you leave it out, you get
no /dev, and all kinds of things mysteriously break. This is
easily worked around by doing (cd /mnt/dev && MAKEDEV generic) by
hand, first, if you remember.
I solved all these problems by writing a shell script that would
extract a .deb into a specified directory, ignoring all dependency
fields, and not running preinst scripts. (It appears that none of the
Essential preinsts need to do anything when installing from scratch
with a modern dpkg.) Then it would extract the control files,
generate a .list file, and fake a status file entry, leaving the
package in the 'unpacked' state. (To find out what Status: line
corresponded to 'iU' in dpkg -l, I had to dig through the apt
sources. Why is the status file format undocumented?)
I ran this script for all the Essential packages and the transitive
closure of their dependencies. In case you're curious, these are all
the packages which are not Essential but included in the transitive
closure:
libc6
libcap1
libdb2
libncurses5
libpam-modules
libpam-runtime
libpam0g
libreadline4
libstdc++2.10-glibc2.2
mawk
I'm not sure why base-files wants awk.
Once this had been done, 'dpkg -a --configure' succeeded, leaving me
with a minimal but functional installation. Then I downloaded all the
other packages I want, and did 'dpkg -iGE *.deb' to install them. I
believe this would have worked in principle. dpkg presently has a
bug, which causes it to segfault after unpacking five to ten packages.
I think this is the same bug that is being discussed right now on
debian-dpkg. I also think it has something to do with the length of
the command line. Note that -iGE will skip over hundreds of packages
without a problem; the crash only happens after it starts unpacking
things.
You can't just repeat the command, because the packages that it
unpacked have not yet been configured, and for some reason dpkg will
try to unpack them again instead of just configuring them and moving
on. It will therefore crash again, in the same place.
dpkg -a --configure doesn't necessarily work, either, because it may
not have gotten to all the dependencies of the unpacked packages. You
have to find those and install them individually, then configure
pending packages, then try to install *.deb again. Repeat ad
nauseam. After about an hour of this I got everything installed.
I couldn't use apt to do this, because for some mysterious reason,
even if you provide apt with a complete set of Packages and Release
files, pkgcache.bin, and all the debs in /var/cache/apt/archives, and
say --no-download, it still tries to do DNS lookups. These fail
because there's no /etc/resolv.conf yet (in my configuration, that
gets created by the DHCP client - which is running outside the chroot,
but apt inside doesn't know that.)
Packages providing daemons tend to start them in their postinst.
That's fine in most cases, but not when you're installing into a
chroot tree and the same daemon is already running outside. They try
to bind their TCP socket(s), fail, and die. Fortunately most
postinsts don't consider this a fatal error.
base-config has serious bugs at present. Passwords are echoed to the
screen, and it does some ridiculous grep loop over the password file
which takes about five minutes to complete. With no indication of
what's happening. I thought it had wedged. Finally, it gets
seriously confused when it hits the bit about running tasksel or
dselect, and errors out. I wound up setting up shadow MD5 passwords
and accounts by hand - this is not hard.
I understand that installer generation has to jump through some of the
same hoops, so I hope my experience provides some useful insights.
I've attached the unpacking script.
zw
#! /bin/sh
# dpkg-extract: extract a .deb archive into a virgin system.
# This is the same as dpkg -i except that:
# - preinst and postinst scripts are not run
# - the available file is not updated
# - the package is left in the "unconfigured" state
# - dependencies are ignored
# Usage: dpkg-extract package.deb /path/to/target/filesystem
deb=$1
targ=$2
emsg () {
echo "$@" >&2
exit 1
}
[ -n "$deb" ] && [ -n "$targ" ] ||
emsg "usage: $0 package.deb /path/to/target/filesystem"
[ -f "$deb" ] || emsg "$deb: file not found"
[ -d "$targ" ] || emsg "$targ: no such directory"
pkg=${deb%%_*}
dpkgdir=$targ/var/lib/dpkg
scratch=tmp.$$
# Create the skeletal ROOT/var/lib/dpkg if it doesn't already
# exist (it may well not).
set -e
umask 022
[ -d $dpkgdir ] || mkdir -p $targ/var/lib/dpkg
[ -d $dpkgdir/updates ] || mkdir $targ/var/lib/dpkg/updates
[ -d $dpkgdir/info ] || mkdir $targ/var/lib/dpkg/info
[ -f $dpkgdir/status ] || touch $dpkgdir/status
[ -f $dpkgdir/available ] || touch $dpkgdir/available
# Likewise ROOT/dev, or lots of stuff will break.
# ??? check for devfs...
[ -d $targ/dev ] || mkdir $targ/dev
[ -c $targ/dev/null ] || (cd $targ/dev; /sbin/MAKEDEV generic)
# Extract the control files and generate the list file.
mkdir $scratch
dpkg-deb -e $deb $scratch
dpkg-deb -c $deb | awk '{print $6}' > $scratch/list.T
# Make sure no file in the list exists in the target directory.
# (Directories may exist already.) We do this before editing the list
# file, so we still know what's a directory and what isn't.
abort=
while read file; do
case $file in
*/) ;;
*) if [ -e $targ/$file ]; then
echo "$file exists below $targ already" >&2
abort=yes
fi
;;
esac
done < $scratch/list.T
# Now edit list and control to match what dpkg wants to see in
# info/pkg.list and status, respectively.
# "Status: install ok unpacked" is believed to be the appropriate
# string to make dpkg -a --configure DTRT later. (The format of the
# status file, and the exact semantics of the Status: line, are
# documented nowhere. Down, not across.)
cd $scratch
mv control control.T
sed 's|^\./$|/.|; s|/$||; s|^\.||' list.T >list
sed '/^Architecture: /d
/^Package: /a\
Status: install ok unpacked
' control.T >control
rm control.T list.T
# Make sure the package info files don't already exist in $dpkgdir/info.
for file in *; do
mv $file $pkg.$file
if [ -e $dpkgdir/info/$pkg.$file ]; then
echo "$pkg.$file exists in $dpkgdir/info already" >&2
abort=yes
fi
done
# Bail out now if any of the above checks failed.
[ -z "$abort" ] || exit 1
# Finally, update the package info database, and unpack the package.
cd ..
cat $scratch/$pkg.control >> $dpkgdir/status
echo >> $dpkgdir/status
rm $scratch/$pkg.control
cp $scratch/* $dpkgdir/info
rm -rf $scratch
dpkg-deb -x $deb $targ
Reply to: