[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: how to make Debian less fragile (long and philosophical)




I would be satisfied with a backup root disk as a substitute for 
durable recovery tools if:

  1. I could swap root in a running system without a reboot

  2. Debian installs the backup root disk by default

Debian could install the backup root by default, but I think swapping 
root on a live system is something that is currently beyond Debian's 
ability.

I do often have backup roots on the system--so that I can reboot 
without being physically present at the machine. The minimal requirement
when a backup root is available is that:

  1. rebooting is OK (often it is not)

  2. i have a shell and enough static tools to alter what gets booted
     by default (the most common situation is I don't have a serial
     console on the remote machine) -or- I have a serial console on
     the remote machine--which is much desired, but rarer.

Otherwise I'm in the same boat as with the boot floppy, where I have to
buy a plane ticket and fly to wherever my server is (yes) or else walk
some tape monkey through the procedure over the phone, assuming I 
can get the tape monkey in at 4am (failures always happen when it's 
least convenient).

A more common scenario is I am doing a bunch of maintenance from 
remote that MIGHT bring the machine down--and if it does, and I had to,
I COULD go in to the office or hosting location and fix it--but it's 2am 
and I am at home and would like to go to sleep soon. If I have to go in
it will cost me a couple of hours of sleep (but no critical business).
If I can fix it from home, I get to go to sleep much sooner, and I don't
have to go out into the cold freezing Toronto winter at night.

And yes, admins do make OS/purchasing decisions based on how much
sleep they're likely to get, and whether they'll have to go out
into the cold freezing Toronto winter at night.

Justin


On Tue, Aug 17, 1999 at 12:26:44PM -0400, Raul Miller wrote:
> On Mon, Aug 16, 1999 at 10:45:17PM -0400, Michael Stone wrote:
> > Dynamic linking provides these benefits:
> > 1) Saves space.
> 
> optimization issue
> 
> > 2) Removes the need for seperate rescue and run-time binaries.
> 
> optimization issue
> 
> > 3) Easier to update critical binaries in the event that a library flaw
> > is discovered. (E.g., security problem or correctness issue.)
> 
> optimization issue
> 
> > and has this flaw:
> > 4) Library damage can prevent binaries from working.
> 
> recovery issue
> 
> > You can poo-poo point 1. For most people this isn't a real big issue.
> > Another 20-30M isn't a huge chunk on a modern hard disk. We do have some
> > users with older systems, but they can cope.
> 
> I suppose we need real numbers here.
> 
> > Point 2 is a little harder. This is a volunteer effort. It's hard
> > enough sometimes for people to maintain their packages without
> > maintaining an extra set of them. You could put the static versions
> > in /sbin instead of breaking them off in a seperate dir, but then
> > you waste RAM instead of hd space. (Dynamic libs are loaded once,
> > static libs are loaded for each binary.) It's a tradeoff, just like
> > everything else in a distribution.
> 
> I'm confused by this line of argument, but I'll try to break it down:
> 
> (a) consider the possibility that the static versions will be
> maintained by different maintainers than the dynamic versions (presumably
> not updated as often either).  [Perhaps I should do it, since I'm
> so interested in the subject.]
> 
> (b) the directory where something is stored has nothing to do with how
> much ram the executable consumes.  [Anyways, ram consumption has little
> to do with point 2.]
> 
> > Point 3 is more thought-provoking. If you statically link everything
> > then any libc update means updating 30 packages by 30 different
> > maintainers rather than a updating a single libc package.
> 
> Not really.
> 
> libc update usually doesn't matter for the key administrative programs.
> 
> Where it does matter, it may be advantageous to update the administrative
> programs well before libc is ready for general consumption.  [Of course,
> if we have both dynamic and static versions of these programs there's
> the added complexity of deciding to switch which program you're using
> under non-emergency circumstances -- which is a plausible argument in
> favor of not having two distinct instances/versions of the same program.]
> 
> > Against the pros of dynamic linking, you have a single con: a damaged
> > library can prevent programs from loading. But as I said earlier:
> > what are the odds that a massive failure will affect only libraries?
> 
> (1) This con is in a different class from the pros.  More generally,
> premature optimization is bad.
> 
> (2) there are other potential failure modes besides massive disk failure
> where these precautions would be vaild.
> 
> I don't think all users will want to take these precautions, just as 
> I'm sure that many of our users don't perform regular backups.  Like
> you say: it is a tradeoff.
> 
> > If even one of your static binaries is destroyed, you're in the same
> > place that you were with a broken library.
> 
> Again, not really.
> 
> More likely you're forced to operate under degraded (inconvenient)
> circumstances.  But you've got more chances at working around the
> problem if you have some working executables.
> 
> > (E.g., a disk problem or a bad case of the dumbs wiped out /dev and
> > /lib. You've got static bins, but mknod also got wiped out. Bummer.)
> 
> Well, if sash is working you could use its built-in -mknod.
> 
> If it's not maybe you've got uudecode [or some other way of quoting
> binary characters] (which might mean you can paste a program into a
> running session).
> 
> > For static bins to be useful you need a particular combination of
> > disaster and luck.
> 
> Knowing your way around the system also helps.
> 
> > Optimizing for that combination is like writing an algorithm for
> > best-case performance: I can't say that it never helps, but it buys
> > you nothing when you really need it.
> 
> In general, if you have a shell or can obtain a shell you're a lot
> better off than if you cannot.
> 
> > If a particular machine has to be highly available, it needs more than
> > static binaries.
> 
> Very true.
> 
> > What do I think a machine needs to be reliable? Here's a couple for
> > starters: 1) A backup root disk. It's easy, it's cheap in today's
> > market, and it buys me protection from both hardware and human error.
> > (Even static binaries quail in the face of rm.) 2) A serial console.
> > If some disaster is pulling the rug out from under my server, I don't
> > want to rely on its network interface. There's too much that can go
> > wrong, and I can't boot over it. 3) A failover machine. Sometimes
> > things really do break.
> 
> In many circumstances these are good practices.
> 
> However, this is far from complete.  You wouldn't expect that a backup
> root disk would replace a regular backups -- why expect it to replace
> static binaries?
> 
> > That's exactly right. That's why you don't screw around with a
> > production system unless you have a way out. And that's why you don't
> > run a production system unless you have a way to compensate for
> > catastrophic failure. And that has nothing to do with static binaries.
> 
> I disagree -- static binaries are just one of many ways to compensate
> for catastrophic failure.
> 
> I agree that there are many classes of catastrophic failures which
> static binaries can do nothing for.  However, I've run into several
> real life cases where static binaries have helped (or: would have
> helped).
> 
> -- 
> Raul
> 
> 
> -- 
> To UNSUBSCRIBE, email to debian-devel-request@lists.debian.org
> with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
> 


Reply to: