Re: Alternative to 64/32 hybrid - 32 bit UML
On Wed, Sep 03, 2003 at 10:03:29AM -0500, Michael Shuey wrote:
> On Wednesday 03 September 2003 08:30 am, Dan Shearer wrote:
> > On Wed, Sep 03, 2003 at 10:41:25PM +0930, Dan Shearer wrote:
> > > UML should be hardly noticeable as an emulation layer if packaged
> > > correctly.
>
> No. That's a *very* bad approach, for several reasons:
>
> UML is NOT a program emulation layer, it is a SEPARATE OS INSTANCE. That
> means you'll need essentially the guts of a Linux distribution for each UML
> you start up.
... which is the whole advantage, given that allegedly library
management problems make 32-bit support on 64-bit Opteron is a
showstopper. I never said it was going to be as good as not having
UML at all :-)
> That's a significant amount of extra management for someone
> that just wants to run one or two 32-bit apps.
No, not at all necessarily. It is possible to provide that
infrastructure for someone already packaged up. It comes at a cost...
but the overwhelming cost according to this list is the time required to
develop a working 32/64 solution wrt library management. That goes to
zero if you can get what you want done with UML.
> UML is NOT a lightweight beast. Pretty much all IO though UML is quite slow
> - a horrible tradeoff for people with performance needs for their 32-bit
> apps
I'm proposing this to illustrate that a 64-bit solution can deliver
32-bit capabilities while using the existing 32-bit Sarge library
layout (because it is a distro-within-a-distro.)
I quite agree that if someone want high performance then this isn't
going to work for them.
> (and there are a lot of those people - think of the vast numbers of
> cluster users out there buying Opterons now).
"vast numbers"? Are you sure about that? Industry numbers are
completely rubbery things but I hadn't seen evidence of this.
> UML is NOT flexible - you have to give it a chunk of RAM when it starts.
> What if your 32-bit app suddenly needs more memory than you initially gave
> it? Oops, gotta reboot the emulator. If you guess too high and give the
> emulator too much memory you end up starving the host environment.
That is quite true. Dynamic memory pools for UMLs will require some
changes in how tmpfs works on the host.
> > Just to qualify a little: I am thinking of servers (where Opterons are
> > most likely to be used at the time Sarge is released) and of
> > longer-running processes so that the startup time, which can already be
> > trimmed right down, is negligible. Nobody normally cares if Apache takes
> > another 0.25 seconds to start up an instance. If someone is after
> > performance they'll use a 64-bit binary anyway. If someone wants a
> > 32-bit /bin/ls then they're a corner case.
>
> I want performance on 32-bit binaries. Why? Because I use 3rd party
> software that's not built for Opterons. Sometimes my 32-bit stuff needs a
> small amount of memory, sometimes a lot. I'm not going to want to
> sacrifice flexibility on this; I'd rather just bite the bullet and go with
> Suse.
Yes, so you should. But why would you buy either an Opteron or an
Itanium anyway? Perhaps a hyperthreading P4 would do a better job for
you?
> DEC didn't use a full virtualized OS like UML to support their 32-bit (IA32)
> emulation layer. They used a wonderful little binary emulator, but it
> didn't emulate an OS. It merely executed IA32 instructions and provided
> syscall translation to the underlying Unix core. Such functionality is
> already provided in an Opteron processor and kernel (instruction execution
> is native to the proc, syscall emulation and the like in the kernel).
> That's a good approach; it lets 32-bit apps run alongside 64-bit apps with
> no hassles.
>
> If you throw UML into the mix you add a second kernel, trying to do its own
> memory management on top of the underlying kernel. You can't directly
> interoperate with the 32-bit process, you have to go through (essentially)
> another machine first. That's insane, especially when faster, more
> convienient support already exists.
Well, the insanity as I saw it was that Opteron support for Debian might
be held up for 18 months because the feature you described cannot be
exposed neatly due to library management issues.
What I describe is more than adequete for running occasional or
performance-insensitive 32-bit applications, and could go out the door
with Sarge, without impacting a hybrid solution for Sid and without
costing a lot of effort.
> I run a good-sized Linux cluster - essentially a few hundred servers. We're
> beginning to branch out from i686 procs to Itanium and Opterons. Opterons
> are quite nice because of the 32-bit compatibility on the servers. Some of
> our clients have no problems rebuilding their code; others rely on 3rd
> party 32-bit only binaries. With a proper OS on the Opterons we can
> support both.
I see, I hadn't considered this, of course, you have a good case. And in
your shoes I'd go for something other than Debian, probably avoiding
SuSE because of their business model. You might like to look at
SCI (SuperComputer Inc) who are working with Gentoo specifically for
Opteron 64-bit in cluster applications.
> > When a server administrator gets a chance to wring a few percentage
> > points of extra performance out of his hardware raid or network card, he
> > usually does by installing a better driver if one is available. Even if
> > the performance isn't really required or noticeable! So people are
> > suggesting that owners of expensive brand-new Operterons are going to
> > want to do dist-upgrade from old system partitions rather than install
> > from scratch? I don't think so. 64-bit Opteron works faster and has the
> > memory model. Too good to miss.
>
> True - few sysadmins are going to bother dist-upgrading their Opterons,
> rather than reinstalling. However, if you allow for full compatibility
> between 32-bit and 64-bit userlands you should be able to get this minor
> feature essentially for free.
Yes, I can see this. And it would be a good feature.
--
Dan Shearer
dan@shearer.org
Reply to: