Re: swap

[Douglas Allan Tutty <dtutty@porchlight.ca>]

On Mon, May 07, 2007 at 01:07:23PM -0400, Greg Folkert wrote:
> On Mon, 2007-05-07 at 12:15 -0400, Douglas Allan Tutty wrote:
> > On Mon, May 07, 2007 at 11:55:56AM -0400, Greg Folkert wrote:
 
> Cluster? HA! Bigger Single computer? HA!
> 
> They have 8 processor machines with 64GB of memory already. The batch
> process can only utilize 1 processor. The other 7 processors, are
> basically idle. I've trended the entire machine for them. If they could
> LPAR the machine(s) out, they'd be marvelously happy. But they would
> need to get the memory upto 512MB or better and then multi-path IO for
> the swap... sheesh. It would be cheaper to just buy another machine and
> add it, but then they already have 3 hours at worst, 4 hours at best, of
> growth left.
> 
> In any case, a "pre-batch" program assigns jobs to each machine, it
> takes nearly an hour to estimate loads. Again single processor usage. 
> 
> This whole package was never meant to scale. But it has been forced to.
> It also was meant to be a temporary fix until a new system was to be
> spec'd and written. Nothing ever came of the effort in the 70's and was
> dropped when this was "good enough".

I suppose the holy-grail would be something that does for CPUs in boxes
what LVM does for disks:  Allow a single-threaded process to utilize
multiple CPUs for more speed, those CPUs able to be both within one box,
and spread: a CPU pool and a memory pool.

The focus for a while seems to have been how to divide up a big computer
in to several smaller virtual servers (ala xen or IBM's LPARs).  I
haven't kept up on efforts to solve a massivly sequential problem.
However, my interest is aroused.

If you have a box with 8 processors and your process can only use one,
can you use something like Xen, designate one whole processor and its
memory to your main process and use the other processors as helpers?
(maybe you don't need Xen for that, I don't know).  

For example, if the process needs more memory and therefore uses swap,
and the MB is maxed out for memory, could another processor be used by
the OS to manage a multi-disk swap farm?  Put another way, if a linux
box can serve data to saturate a gigabit ethernet, and it is possible to
create a block device that looks like a disk that really gets its data
over ethernet from another computer, can an 8-way MB take that input
and present a virtual swap device to one processor so that swap
functions at the same speed as memory?

I guess that's called a mainframe :)

Greg, I'm just babling on this.  If you have links for reading I could
do, I'd appreciate it.  Then I may at least know what I'm babbling
about.

Thanks,

Doug.