Re: OT: free cmd is lying to me

[Richard Cobbe <cobbe@airmail.net>]

Lo, on , December 2, Michael Heldebrant did write:

> IIRC from the Understanding the Linux Kernel book by O'Reilly, linux
> doesn't actually worry about memory until you actually use it.  I'm not
> sure if a malloc counts as using it for storing data since I'm no C
> programmer but unless you actually write to the memory linux doesn't
> bother setting up the actual pages since it's a waste for the system to
> make and tear down pages that are never accessed.

Well, I *am* a C programmer; perhaps I can clarify this.

/Understanding the Linux Kernel/ is correct; Linux (in fact, most modern
OSes) don't allocate pages to a process until that process requests the
memory.  However, it's not entirely clear what constitutes such a
request.

I think you're thinking of demand-based loading.  When you start a large
process, like Netscape or Emacs, the kernel does *NOT* load the entire
text segment (containing the actual executable code and constant data)
into memory.  Instead, it sets up the process's page table such that the
pages in the text segment are, in effect, swapped out to the program's
executable file on disk.  On the first reference, the page is
automatically brought in by the kernel's VM system.

The non-constasnt but initialized data pages probably use a similar
setup, although they'd need a copy-on-write scheme.  Uninitialized but
pre-allocated data is also likely allocated on demand, but it's
initialized to zero and swapped to your swap device as normal.

However, the heap doesn't work like this.  If malloc(3) can't find
enough memory to satisfy the program's request, it calls sbrk(2) to get
one or more additional pages.  I'm pretty sure that the kernel will have
to adjust the process's page table immediately, although there's a good
chance that the pages won't be mapped into memory until they're needed.
However, malloc() has to keep bookkeeping info along with the user
program data, so the first reference is likely to follow the allocation
pretty rapidly.

The stack is allocated on demand; I don't know if the kernel will shrink
it or not.

Richard