Re: Strange hard lock
On Tuesday 06 December 2005 11:42, Hugo Vanwoerkom wrote:
> Graham Smith wrote:
> > On Tuesday 06 December 2005 10:39, Hugo Vanwoerkom wrote:
> > * snip *
> >>What kernel version? I had something similar and oom-killer was no help
> >>in identifying the culprit.
> >>This is no help. :-(
> > It's running
> > Linux compost 2.6.12-1-386 #1 Tue Sep 27 12:41:08 JST 2005 i586 GNU/Linux
> > the machine had been up for 61 days (I checked a few hours before it
> > failed) with no problems and is running Debian testing.
> > Having a closer look at the logs I notice that the first time it killed
> > postmaster but after that it just seemed to repeatedly kill the apache2
> > process.
> > I see what oom-killer is trying to do but I can't believe for one minute
> > that I was running low on memory (the machine is running quite a few
> > processes but never thrashes the disk). A typical top looks like this:
> > top - 11:14:29 up 2:19, 4 users, load average: 0.04, 0.11, 0.12
> > Tasks: 143 total, 1 running, 142 sleeping, 0 stopped, 0 zombie
> > Cpu(s): 4.6% us, 1.3% sy, 0.0% ni, 93.7% id, 0.0% wa, 0.0% hi, 0.3%
> > si Mem: 321792k total, 308852k used, 12940k free, 15320k
> > buffers Swap: 393584k total, 8340k used, 385244k free, 55168k
> > cached
> > Which is a little odd because I thought that machine had 768 MB of ram
> > not the 300 odd that is being reported. Maybe I removed some :o) it's
> > been so long since I opened that box I can't remember.
> Well, again this is no help, but anyway:
> What does 'cat /proc/meminfo'show?
> In response to my oom-killer I started my sensor apps to show Free
> Memory. That doesn't get you any closer because that value floats from
> the high end to about 10-15MB on later kernels and 3MB on earlier kernels.
> Do you remember what BIOS shows on memory installed?
I have a sneaky suspicion that I know what the problem is. I think there might
be a memory leak in the Java JAI (Java Advanced Imaging) library. Yesterday
was the first time I extensively used them on the server. I've got top
monitoring the memory usage (M sorted) and so far no problems. What I did
notice was that a few hours before going down apache seemed to be having
problems serving up content. Randomly bits wouldn't appear like it had just
Result of 'cat /proc/meminfo'
MemTotal: 321792 kB
MemFree: 20804 kB
Buffers: 23988 kB
Cached: 62792 kB
SwapCached: 1904 kB
Active: 239588 kB
Inactive: 39356 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 321792 kB
LowFree: 20804 kB
SwapTotal: 393584 kB
SwapFree: 375960 kB
Dirty: 472 kB
Writeback: 0 kB
Mapped: 229540 kB
Slab: 15760 kB
CommitLimit: 554480 kB
Committed_AS: 539200 kB
PageTables: 2272 kB
VmallocTotal: 712624 kB
VmallocUsed: 2384 kB
VmallocChunk: 709832 kB