On Tue, Jan 04, 2011 at 07:32:06PM +0100, Olaf van der Spek wrote:
> On Tue, Jan 4, 2011 at 7:29 PM, Steve Langasek <vorlon@debian.org> wrote:
> > On Tue, Jan 04, 2011 at 06:51:13PM +0100, Olaf van der Spek wrote:
> >> On Tue, Jan 4, 2011 at 6:49 PM, Sven Joachim <svenjoac@gmx.de> wrote:
> >> > Maybe we're talking at cross-purposes here; I was speaking about the
> >> > case of turning a directory into a symlink on upgrades, which cannot
> >> > safely be done while there are still files under it.
> >
> >> > Thinking more about it, this cannot be done even if all packages move
> >> > their files away from /usr, since an unknown amount of stuff exists
> >> > under /usr/local, so /usr can ever only be made a symlink to / for new
> >> > installs (or by the sysadmin).
> >
> >> I think it's about /usr/lib being a symlink to /lib.
> >
> > No, it isn't.
>
> Then what is it about?
> I thought the problem was having both /lib and /usr/lib.
Well, that's the issue at hand. The reason I mentioned this is
because I believe that the / and /usr separation is a case where we
should stop to consider the "bigger picture" rather than just the
immediate problem. Solving that would solve the /usr/lib and /lib
issue as a natural consequence.
> Linking /usr to / would also solve that but is more complex than
> linking /usr/lib to /lib.
Only minimally; the same issues apply to unifying both.
A bit of background can be found in this thread from 2009:
http://lists.debian.org/debian-devel/2009/05/msg00075.html
This is a long thread, but there's a lot of useful detail in it, the
most important bits I've summarised below. Note that this is
concerning not allowing /usr as a separate filesystem, but eliminating
/usr entirely is a logical consequence of it and so is mostly
relevant here.
Historically, we have always had a separate / and /usr, with the
possibility of /usr being a separately-mounted filesystem. So, the
points to consider are
• what are the benefits of the separation?
• what are the costs of the separation?
• do the benefits and costs of the separation as originally conceived
apply today in 2011?
I was previously a proponent of keeping /usr separate, but after
spending some time considering it I've revised my opinion of the
matter. I previously thought it had a useful purpose in some cases;
I'm now of the opinion that it's essentially pointless. Most of the
historical reasons for keeping it separate are today no longer
relevant; they are either superseded by better methods, or are not
compatible with modern practices.
To provide some context, I'll run through a few of the use cases and
try to provide some insight into the rationale behind them, and some
opinion on whether it's actually useful. Most of these were discussed
in the above thread.
1) /usr is shared by many machines over NFS
This is a "traditional" use of a separate /usr. It may have even
been common back when SVR3 was popular and disc space was at a
premium, and modern package managers were absent. But, it's
completely insane to do this today.
Pros:
• Saves disc space by sharing one filesystem amongst many hosts
• Allows centralised management of software updates; updates only
need doing once for all hosts
Cons:
• Debian uses dpkg for package management; you can't separate / and
/usr and expect it to work since packages need to update both
locations. If you have a / per host and a shared /usr, a
disaster is inevitable.
• /bin, /sbin, /lib are not shared, so you need an alternative
mechanism to propagate changes to the hosts; and /etc is even
more complex. Also, how do you run dpkg maintainer scripts on
each host?
• Debian does not support a *shared* /usr over NFS. It supports
mounting a *non-shared* /usr over NFS, but that's not the same
thing.
If you're doing this, you're doing it wrong. You should be NFS
mounting the root filesystem in its entirety or else dpkg will
break. I use debian-live to manage such images, and NFS mount the
shared root (containing /usr) with a writable unionfs overlay to
allow host-specific configuration and customisation. It's robust
and supported.
2) /usr is mounted read-only for security and safety
Mounting /usr read-only is common practice; I even do this myself
with apt-get configured to remount read-write when changes are made
and then remount read-only afterwards.
Pros:
• You have read-write / and read-only /usr which prevents changes
to /usr during normal system operation.
• Allows mounting with nodev
Cons:
• Does not protect /bin, /sbin, /lib etc.
• Can be done on read-write / with read-only bind mounts
• Can be done with read-only /
• With udev, nodev can be used on /
If /usr is eliminated, then we can just mount / read-only instead;
it's arguably even safer. Quite a lot of work has gone into having
a read-only / such as /lib/init/rw. It works today, though a few
small issues remain (/etc/mtab, LVM backup/cache/archive).
3) A minimal root filesystem makes recovery easier
Having / without /usr and other filesystems mounted makes it
possible to troubleshoot and recover from problems such as
filesystem corruption.
Pros:
• /etc will be recoverable if /usr gets corrupted
Cons:
• The system is unusable without /usr mounted; do we really care
about the small chance that one filesystem will be corrupted over
the other. i.e. does having them separate really reduce the
probability that we will lose /etc in any meaningful way?
• You also need /var (for dpkg status etc.) for recovery
• Given that / is typically read-write and /usr might be read-only,
the chances are that / will be corrupted rather than /usr unless
you have hardware failure, in which case both will likely be
screwed
I've occasionally used single user mode when LVM problems or
hardware changes prevented filesystems being mounted etc. But it's
been more common to use the initramfs shell or a rescue CD.
Overall, I can see that this mitigates recovery, but I'm not
convinced that it's giving that much extra robustness. I'll be
keeping backups for recovery/reinstallation rather than playing the
probability game.
3) Different backup policies for / and /usr
Pros:
• Alleged performance gains
Cons:
• You don't need them on separate filesystems for separate backup
policies
I don't buy this one. While you might have a different backup
policy for /etc and /var, there's no reason to treat any dpkg-managed
location such as /bin, /lib and /usr/* differently from each other
since they exist as a managed whole.
4) / is encrypted and /usr is unencrypted
Pros:
• Performance
Cons:
• Do you really need the entire / encrypted, or just specific bits?
This looks like it's for encrypting things in /etc. I'm not sure
that this solution is necessarily the best one. The / vs /usr
distinction is one of convenience rather than actually encrypting
only the bits actually required.
5) I have /usr on LVM and / on a partition
This is a fairly common setup; I used it myself in the past.
Pros:
• It is possible to mount the root filesystem with no LVM in case
there are LVM problems
Cons:
• / is of fixed size; you can't online resize it
Nowadays this is a non-issue. I've had my root filesystems on LVM
for years now; with initramfs and separate /boot; there's no need
for these workarounds and recovery on LVM vs a normal partition is
just as easy. A rescue CD can deal with both just as effectively.
If you're using LVM, you should have your / on LVM, which would
allow it to be dynamically resized as for all other partitions.
6) Libraries are separated in /lib and /usr/lib
Programs in /bin and /sbin use /lib; programs in /usr/bin and
/usr/sbin use /lib and /usr/lib. Programs on / must work
prior to /usr being mounted. Libraries on / must only depend
upon other libraries in /.
Cons:
• It's difficult to draw the line of what should be in / and what
should be in /usr
• Special care needs to be taken before /usr is mounted since
important pieces such as locales are not available
This is the problem initially brought up here. As we increase the
complexity of what is required prior to mounting /usr such as
having NSS modules with dependencies on other libraries, those
other libraries need moving to / as well. The same applies to
crypto support as mentioned initially. If / and /usr are unified,
the problems introduced by having them separate are immediately
resolved: everything becomes available from / being mounted except
/var.
7) A separate /usr is required to boot the system
Many years ago, a minimal root filesystem was required to bring
the system up to a certain level of operational readiness in order
to mount /usr and other filesystems. Examples included:
• NFS mounts which required networking and portmap/rpc
• RAID requiring array reconstruction
• LVM which required discovery of available volumes
• Loading additional kernel modules to access hardware
• And other things
For quite some time now, Debian uses an initramfs to boot. This
does all the setup for you. This means you can mount / from NFS
or from LVM or MD (or both stacked) etc. There's no requirement
for a separate /usr for booting.
8) Filling up /usr won't fill up /
Installing software won't use up all the space on the root filesystem
Nice idea in priciple… but you can still fill up your root
filesystem with kernel images and initrds if /boot isn't separate.
This is a preventative measure, but not a particularly effective
one. And if filling up / occurs, then it's not exactly hard to
reverse the procedure.
9) / and /usr are using different filesystem types
This is a rather hard one to take seriously. While there are times
when you would choose one filesystem type over another for various
reasons, using different types for / and /usr makes little sense
given their similar requirements. It provides zero benefit.
I think many instances of confusion and misunderstanding over the / and
/usr separation, particularly with regard to NFS mounts, but also
covering read-only mounts and recovery are due to not fully considering
the implications of a modern package manager on the traditional UNIX
filesystem hierarchy. We can consider that there are just two
different types of directory in the file system:
• those managed by dpkg
• those containing user data which are not under dpkg control
All locations managed by dpkg must be considered a unified whole; it
does not make any sense to share one part and not another. They must
be updated together or else the system will be left in a broken and
inconsistent state. A separate /usr is no longer required to boot the
system now we have initramfs. In consequence, there's no reason to
have /bin separate from /usr/bin, /lib from /usr/lib etc. It makes
sense to have /var separate (writable). It makes no sense to have /usr
separate.
Regards,
Roger
--
.''`. Roger Leigh
: :' : Debian GNU/Linux http://people.debian.org/~rleigh/
`. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/
`- GPG Public Key: 0x25BFB848 Please GPG sign your mail.
Attachment:
signature.asc
Description: Digital signature