on Fri, Oct 22, 2004 at 07:21:20AM +0000, Alexis Huxley (ahuxley@gmx.net) wrote:
> On 2004-10-21, Gilbert, Joseph <jgilbert@insuresuite.com> wrote:
>
> > taken for rote. I was told back when I first started working with Unix that
> > the swap space needed to be at least twice the size of physical memory in
> > order to ensure a stable system.
>
> I believe this was just a rule of thumb when memory was so expensive.
>
> Think about it, if you want to dramatically improve the performance
> of a system then you add so much memory to it that the system stops
> swapping. And if it stops swapping, then what purpose does having
> "two times 'so much'" swap space have? Well, none, 'cos you've
> deliberately ensured that it won't be used.
Close, but not quite.
"Swapping" and "swapped" are two different issues.
- "Swapping" is the state of paging memory to disk and disk to memory.
The read/write takes time. This is to be avoided.
- "Swapped" is the state of memory being stored on disk.
Once you've written memory to disk, it's really not hurting you. Modulo
edge cases, disks don't "get slower" as they fill up, nor do swap
partitions.
> But some programs are so inactive that they'll get pushed in to swap
> (by some user's 'qsort() routine or other short-lived memory eater)
> and may never come back because they're waiting for something that may
> never happen (e.g. 'at' daemon waiting for next 'at' job when nobody
> actually uses 'at'), so adding memory to stop processes like these
> being swapped (out, once) in the first place is a waste of money;
> it brings no performance increase.
Depends. If the process is called or invoked infrequently, it may be
swapped in or out several times over a long period. This is relatively
sane.
> Of course there should always be swap space, because you *cannot*
> guarantee that someone won't run perl in slurp mode, or qsort on a
> massive data set, etc, and, generally, "I don't want to ever encounter
> the error message 'out of memory'" is a hard requirement, but the
> comparatively (very) slow access speeds of swap space should give you
> some idea of how often you should actually be regularly (by which I
> mean swapping in and out, not just out) using it, and when disk space
> is so cheap then, hell, you can make the swap ten times the memory and
> probably not notice less disk space. It's a small price to pay to meet
> that hard requirement.
More to the point:
Swap allows you to retain state over long periods of time, for large
processes.
If you've got no swap, you've got your existing memory nearly filled,
and you decide to run another large process, you've got two options:
1. Don't run the additional process.
2. Shut down one (or more) existing processes until you've got
sufficient free memory to run the new process.
If you've got swap, you have option number three:
3. Run the new process. The OS will work out what other processes it
can "swap out" to disk. If it needs to access a process on disk,
it will pull it back off ("swap in").
In the long run, if you've got a reasonable amount of RAM for your
computing needs, you'll end up with a set of memory-resident processes
which are used relatively frequently. You'll have a set of swapped
processes which are used relatively infrequently. And you are spared
the headache of deciding which is which: the OS handles this for you
transparently.
And how do you know if you've got a reasonable amount of RAM? Simple:
you're not swapping heavily. That is: as you (or your users) do
your/their routine tasks, unreasonable amounts of time aren't being
spent waiting for processes to be swapped in or out. Your system isn't
"swapping" heavily (or worse: thrashing). Periodically, yes, you'll
wait a few seconds as something's read or written. But not frequently
enough that it's a hassle.
My own experience: the 2x-3x rule works out pretty well for new
systems. What I figure is that by the time I build a system out, I can
probably drop to about 1x physical memory for swap.
My current system has 1 GiB RAM (its max) built out from 512 MiB
(original configuration). I'd initially configured 2x RAM, and "grew"
out to 1x, by increasing RAM.
The system functions largely as a workstation and sometime development
box. Websurfing, mail, homebrew spam reporting tool, data analysis,
some graphics, and a few other odds and ends. Typically, 200 - 250
processes running, though occasionally far more.
Current usage:
total used free shared buffers cached
Mem: 972336 950548 21788 0 81420 220648
-/+ buffers/cache: 648480 323856
Swap: 1060208 523580 536628
Filename Type Size Used Priority
/dev/hda6 partition 530104 504940 -1
/dev/hda7 partition 530104 18420 -2
Note that my swap is half used.
Oh, and why 2-3x RAM to start? Because you can add memory to a system
pretty easily. Repartitioning to add swap is considerably more
difficult (though you can also add swapfiles rather than partitions).
So overcommit to swap when you build your system. Plan to grow into
it. And you'll find that applications and user expectations *will*
expand to fill available memory expansion slots.
Where's the memory go? Web browser, X, newsreader, OO.o, mutt. And a
lot of other stuff. Some of it's used frequently, some not. But I
don't have to worry about limits, and if I run something very memory
intensive, the system bogs a bit for a while as swapping occurs...but I
don't have to run around shutting down processes and performing triage,
hoping I don't kill something important.
My last 512 MiB memory stick ran about $120. The 1 GiB in swap costs me
a couple of bucks worth of disk. That's a lot of system flexibility for
a pretty low expenditure.
Think of swap as something like /tmp: you don't plan to fill it all the
time, or even most of the time. But when you need it, it's a bit of
reserve capacity, available to you.
> Harder is working out the memory requirements.
>
> I would say one needs some experience of how many users you get, what
> they sort of thing they run, how often, for how long, in order to get
> a 'feel' for how much memory and swap are required.
> Maybe formulating some statements about your requirements is helpful.
> E.g. the aforementioned "I never want to run out of memory", (which
> really means swap) but also something more real-worldly soft
> requirements like "If 95% of user processes, do not get swapped 95% of
> the time then that is acceptable."
In an enterprise setting, you may be able to quantify costs if you can
assess how much time is spent waiting for swap, how much production
waits for swap, etc. Balance the lost productivity with the cost of
increasing available RAM.
> Maybe run 'sar' and archive the logs monthly (otherwise they'll be
> overwritten)
Configurable via logrotate.
> ....and then you can try to work out the memory and swap required to
> meet your requirements.
>
> And talk to your users, then you'll know if the requirements you've
> set are actually acceptable.
What! Users? Who're thay?
;-)
Peace.
--
Karsten M. Self <kmself@ix.netcom.com> http://kmself.home.netcom.com/
What Part of "Gestalt" don't you understand?
I could give you my word as a Spaniard?
No good. I've known too many Spaniards.
- Princess Bride
Attachment:
signature.asc
Description: Digital signature