Re: More than 2, but less than 3 GiB per process memory?

[Hendrik Boom <hendrik@pooq.com>]

On Mon, Sep 26, 2005 at 11:34:15AM -0600, Bob Proulx wrote:
> Hendrik Boom wrote:
> > > Malte Cornils wrote:
> > > > Does anyone have a clue why 2 GiB is the limit?
> 
> It is not for me.  I can malloc() up to 2.9G.
> 
> > It seems malloc restricts itself to the user space.
> > But if you were te replace malloc (or provide your own allocation
> > and freeing methods, which *miht* be possible in C++) you could do
> > the following.
> 
> Although I am sure the details of malloc() are involved I don't think
> that is squarely the problem here.  I have a simple test program which
> mallocs and reports and I can definitely get to 2.9 on a 32-bit system.

I've heard that the kernel allocates memory "optimistically".
That is, when it allocates it it only allocates address space.
It commits real storage (whether RAM or swap space) to the allocated
storage only when the storage is actually used.

Could it be that just mallocing the storage is not enough for it to count
as being "used?"  Would you need actually to write something in each page?

In any case, your test would suggest that the gigabyte boundaries are not
as rigid as an earlier post would suggest.

Or has all this changed with each kernel upgrade?

-- hendrik

> 
>   Out of memory at 2934 M
> 
> Bob
> 
> Here is the very simple test program.
> 
> #include <stdio.h>
> #include <stdlib.h>
> #include <malloc.h>
> 
> const int amountK = 1024;
> 
> int main()
> {
>   unsigned long count = 0;
> 
>   for (;;)                      /* loop forever */
>     {
>       char *p = malloc(amountK*1024);
>       printf("%08lu 0x%lx\n",count++,(unsigned long)p);
> 
>       if (p == 0)
>         {
>           printf("Out of memory at %ld M\n",count / (1024 / amountK));
>           exit(0);
>         }
>     }
>   return 0;
> }