Re: More than 2, but less than 3 GiB per process memory?
On Fri, Sep 23, 2005 at 10:43:40AM -0500, Ron Johnson wrote:
> On Fri, 2005-09-23 at 14:45 +0200, Malte Cornils wrote:
> > Hello,
> > we've been trying to make a program (ITK/VTK image processing for a university
> > project) work. Unfortunately, the process needs slightly above 2 GiB of
> > virtual memory.
> > Judging from the documentation I've seen, on 32bit systems I should be able to
> > allocate up to 3 GiB of virtual memory (1 GiB of 32bit address space is
> > reserved for the kernel).
> > However, our test case for this terminates when trying to allocate more than 2
> > GiB of memory, even though we have a really big swap file.
> > Does anyone have a clue why 2 GiB is the limit?
> 0GB-1GB User space - Used for executable and brk/sbrk allocations
> (malloc uses brk for small chunks).
> 1GB-2GB User space - Used for mmaps (shared memory), shared libraries
> and malloc uses mmap (malloc uses mmap for large
> 2GB-3GB User space - Used for stack.
> 3GB-4GB Kernel Space - Used for the kernel itself.
> > Would using a swap partition
> > instead of a swap file help? The 64G HIGHMEM kernel config option does not
> > seem to make a difference. On a real AMD64 system, the program works fine
> > with >2 GiB RAM, as was expected. Is there any way to make this work, besides
> > changing the algorithm to use less than 2 GiB of memory?
Actually, some kludgery might help here.
It seems malloc restricts itself to the user space.
But if you were te replace malloc (or provide your own allocation
and freeing methods, which *miht* be possible in C++) you could do
Near the start of your program, allocate a *huge* array on the stack, like
char * hugepointer;
hugepointer = huge;
Then have your private malloc (or allocation methods)
allocate space within huge, and when it's full, use
calls to the regular malloc for the rest.