Re: swap

[Douglas Allan Tutty <dtutty@porchlight.ca>]

On Thu, May 10, 2007 at 05:12:23PM -0400, Greg Folkert wrote:
> On Mon, 2007-05-07 at 15:39 -0400, Douglas Allan Tutty wrote:
> > On Mon, May 07, 2007 at 01:07:23PM -0400, Greg Folkert wrote:
> > > On Mon, 2007-05-07 at 12:15 -0400, Douglas Allan Tutty wrote:
> > > > On Mon, May 07, 2007 at 11:55:56AM -0400, Greg Folkert wrote:
 > 
> > I suppose the holy-grail would be something that does for CPUs in boxes
> > what LVM does for disks:  Allow a single-threaded process to utilize
> > multiple CPUs for more speed, those CPUs able to be both within one box,
> > and spread: a CPU pool and a memory pool.
> 
> That is pretty much what IBMs LPAR of AS/400 and AIX (and other
> hypervisor setups) do. 

You mean that with LPARs you can designate 10 processors to a job and
the job will think its running on a single super-dooper processor?

> Unfortunately, this company was "sold" a
> sooper-dooper machine in a deal for 2 of them. They would have spent
> more on smaller machines. But, IBM has sales quotas and deal deadlines
> for sales people. Forklift upgrades, full cabinet deals are pretty much
> the norm when it comes to the sales department. IOW, push more hardware,
> period, they'll eventually be suckers for upgrades.
> 
> > The focus for a while seems to have been how to divide up a big computer
> > in to several smaller virtual servers (ala xen or IBM's LPARs).  I
> > haven't kept up on efforts to solve a massivly sequential problem.
> > However, my interest is aroused.
> 
> Massively sequential problems are very, very, very difficult to
> parallize. Even vectored processor systems balk and fail badly at
> massively sequential problems.
 
> And finally idea number three. Using memory as swap... It is a good
> idea... but then, the whole purpose of swap was that memory was not
> sufficient enough to provide enough working room. Going above 64GB of
> memory on ANY machine is not cheap. Not cheap at all. If I were going to
> use it as swap, I just assume use it as REAL working memory. AIX really
> doesn't have a set-in-stone maximum amount of memory it can support, so
> to use REALLY expensive memory as swap, I'd fire myself for doing that.
> Nice idea on paper, but in reality... not viable.
> 

I guess I wasn't clear.  I wasn't suggesting to use memory as swap.  I
was wondering about ways to get swap that was faster than a disk array
and wondering if it could be farmed out to other boxes and present
something which could be swapped to faster (ideally approaching memory
speed).  For example, if your box _was_ maxxed out on memory but the
program still wanted to pull in _all_ the data to memory at once and it
wouldn't fit, so it swaps; then what if you had something like a SAN
that could present a block device that functioned as fast as memory,
this block device actually being run by (an)other processor(s) on the
same machine?

BTW, the load the whole file into memory but it doesn't fit problem is
_exactly_ the problem I have on both my 486 (32 MB ram) and PI (64 MB
ram).

For a long time I've been intreguied by the problems of solving
sequential problems that can not be parralized.  Paralized solutions
seem to be relativly old hat with things like beowolf clusters and for
that matter Cray clusters.  But what does a computer designed from the
outset to solve a sequential problem look like?  What does the fastest
single processor look like and what does it need to operate at full
speed.  Someone once said (I forgot who or where I hear/read it) that
the job of a supercomputer was to turn compute-bound jobs into IO-bound
jobs.  If you can get the IO (disk, memory, network, whatever) to keep
up with the processor the job becomes compute-bound again.  

So can you make a single virtual processor (emulator?), that may run on many
real physical processors, that runs faster (compute-wise, e.g. FLOPS not
MHz) than any existing single physical processor?  Even for a sequential
program, the internal workings of a CPU are paralelized; could that be
used to advantage?

> IBM has a lot of goodies on publib
> 
> http://publib.boulder.ibm.com/eserver/
> 
> Have fun reading *FOREVER*,
> I mean that literally. And if you have enough time you can also
> checkout:
> 
> http://www.elink.ibmlink.ibm.com/publications/servlet/pbi.wss
> 
> For even more IBM publications.

I fell in love with redbooks when I ran OS/2 (my version was not Y2K+1
capable and I couldn't afford to replace it, which brought me to Linux
via a $20 old stock "teach yourself redhat in 24 hrs book".  Redhat's
next upgrade wouldn't install on my 486, the same 486 that Etch now
won't install on.  On to BSD for that box.

Since I got internet, I've been a regular mooch at ibm's site.  Great
stuff.

I didn't have the chance to play with computers in school, other than
OS/2 and REXX on my home box, and certainly never an AIX box or anything
bigger.  Thanks for humouring me.  

Doug.