core recovery tools, apt-get, and dpkg should be static
(This discussion was originally in the devel list, I am restating my
views here because it is actually a policy issue.)
HOW TO MAKE DEBIAN LESS FRAGILE:
All the core system tools, including the package manager (apt_get
and dpkg) should be available as static binaries rather than dynamic
executables.
Currently Debian provides these as dynamic binaries, which leads to
a fundamental fragility and inability to recover from system failure.
There are three reasons why the core system packages should be static:
#1-- Allows me to recover from a system failure without having to
insert any floppies; and without having to have any special
non-main, non-standard package installed.
As a long-time Unix administrator, I expect single user mode to
be usable even under conditions of extreme failure, where /usr
is not mounted, dynamic linking is failing.
#2-- Limits the possibility that the package manager will mess itself
up due to a bug. Statically linked tools can be copied into
the system through a simple bootstrapping process, and do not
involve complicated library dependencies.
The recent problem in the unstable release with bash/readline
will be fixed by the time potato is stable, but it does
illustrate the concept rather nicely--a circular dependency
involving bash caused its re-installation to fail, resulting
in the removal and inability to re-install the basic system
shell and the C library.
#3-- Allows the package management suite to operate effectively even
when the system is in a half-installed state, or some other
state of disrepair
This is standard practice on most other Unix systems. RedHat provides
static binaries for system repair and a statically linked rpm. Solaris
has static tools in /sbin. All the *BSD free unixes have static tools
in /bin, including the package manager. It's a widespread practice
based on 30 years of hard-won Unix experience.
Therefore I think the Debian policy also refers to this, when it states
that anything an experienced Unix administrator would expect from a Unix
installation should be provided by Debian.
I previously raised this issue in the devel list, and met with various
responses. Here are refutations of many of the common counter-arguments:
"We need dynamic binaries for performance reasons"
There is no performance issue with fsck, restore, mke2fs, shutdown,
mount, umount, apt-get, dpkg, fdisk, dump, mknod, ifconfig, route,
ping, init, halt, insmod, ldconfig, lilo, lsmod, mkswap, rarp, arp,
telinit, swapon, swapoff, etc.; These are commands that are only
used to upgrade or repair a system, and are not normally run
simultaneously by hundreds of different users.
There is similarly no performance issue with most of the basic shell
tools, such as ls, chmod, chown, test, kill, ln, dd, cat, df, rm,
mkdir, more, ed, echo; etc.; those commands that are commonly used
are already builtins in modern shells like bash, and the rest are
so small that it's hardly an issue. Only if you were running Debian
as a shell server with thousands of users logged in would it become
an issue, and in that case you could make dynamic versions of these
tools available in /usr/bin.
Finally, if there were an issue, static versions could be installed
into a different directory, such as /stand, and not used by ordinary
users. During a system failure the system administrator could work
from these standalone binaries; and the package manager could be
written to work from them.
"dpkg and apt are so effective that they eliminate all the problems"
dpkg and apt can only eliminate problems which they themselves cause.
They cannot protect a system against an error on the part of its
administrator, against hardware failure, against malicious hacking,
or anything else outside their own control. All of these things can
lead to a failed system without dynamic linking available.;
Further dpkg and apt have been known to have bugs, even in the stable
version, and there can never really be any assurance that any software
is absolutely bug free.
The ability to recover a system, and install it under adverse conditions,
is so important that relying on dpkg and apt is an unacceptable risk
which makes Debian far too fragile.
"We do adequate testing in unstable; there are no problems in stable"
Once again, it is impossible to test in advance for errors caused
by the system administrator. Just because a flaw isn't the fault of
any Debian installed tool doesn't mean the system administrator
should be hung out to dry, without any help from the OS.
And once again, no software can ever be completely bug free.
"We provide resecure boot floppies for this kind of situation"
Boot floppies are an unacceptable response for three reasons:
#1. The system administrator may be working for remote when
the failure occurs; gaining physical access to the machine
at 4am in the morning may take hours, or be impossible.
#2. The downtime involved in rebooting from a boot floppy is
likely to be substantially longer than if the problem can
be resolved without rebooting
#3. Critical applications may be running on the system, and
there may be a very high cost involved in a reboot
While certainly static binaries cannot avoid the need for boot
floppies (consider a hosed boot loader), they are not acceptable
or viable under many circumstances.
"You can install the sash package if that's what you want"
The Debian policy states that anything which an experienced
Unix administrator would expect to have available should be already
installed by default on an ordinary system. There is common sense
in that policy: most people won't think about this issue until they
get burned; and sash only solves half the problem (it doesn't help
the package manager survive under adverse conditions).
"OK, we need recovery tools--but apt-get and dpkg are different"
System installation is so fundamentally similar to system recovery,
and the two processes are so frequently the cause of one another,
that I fail to see the value of making this distinction. I might
need to install software in order to recover from a failure; and I
might cause a failure by installing software.
The installation tools are more likely than any others to have to
run under adverse conditions--either because they themselves remove
key system packages as part of their ordinary operation, or because
they are likely to be used in response to disk failures or other
problems unrelated to installation itself.
I have always thought that Debian was intended to be more stable and
more reliable than the other Linux distributions. Stability and reliability
are not a primary goal of all distributions, but if they are to be a
primary goal for Debian, then issues like this must be dealt with.
The current reliance on dynamic linking is not acceptable for a production
server that is administered remotely, or where downtime is to be avoided
at all costs. Although many steps have been taken to ensure that the
installation process goes smoothly, it may not always; and even if it
does, problems may arise that are no fault of Debians but which still
must be recovered from.
The package manager and all of the core system tools should be static
binaries. The package manager should take advantage of the reduced
runtime dependencies to engineer a more robust way to bootstrap itself
under adverse conditions.
Justin
Reply to: