Re: du and df conflict; disk space missing

[Heikki Vatiainen <hessu@cs.tut.fi>]

Paul Reavis <preavis@partnersoft.com>
> I'm very confused. I have a samba server running print services, and the
> only real problem I have with it is that my users keep getting "out of
> disk space" errors when printing. My /var partition is totally full:
> 
> Filesystem         1024-blocks  Used Available Capacity Mounted on
> /dev/sda6             253807  240711        0    100%   /var
> 
> But I can't see why with `du`:
> 
> padmount:/# du -s /var
> 32567   /var
> 
> What could cause this?

There are probably processes that have open files in /var. Even if 
you remove a file, the disk space will not be freed before the last 
process keeping the file open exits. One possible process that has 
open files in /var is syslogd. Others might include daemons taking 
care of print services.

The tool which can help you to find these mysterious prcesses is
lsof (list open files). Below is an example where I use tail to
keep a file open to demonstrate how du and df see things
differently. I also use lsof to show how the xemacs binary is still
using the disk space even if it has been removed earlier. Notice
that disk space will not be really freed until I kill tail with
Control-C.

If you find something like syslogd keeping a file open, something 
like /etc/init.d/sysklogd restart should release the file.

% du -k .
315     .
% df -k .
Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/hdc4              1169519   1083042     26048  98% /
% cp /usr/bin/xemacs .
% tail -f xemacs > /dev/null&
[1] 9862
% du -k .
3690    .
% df -k .
Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/hdc4              1169519   1086417     22673  98% /
% rm xemacs
% du -k .
315     .
% df -k .
Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/hdc4              1169519   1086417     22673  98% /
% lsof +D .                                                                                       19:19:rae
COMMAND  PID  USER   FD   TYPE DEVICE    SIZE  NODE NAME
zsh      530 hessu  cwd    DIR   22,4    1024 22571 .
tail    9870 hessu  cwd    DIR   22,4    1024 22571 .
tail    9870 hessu    3r   REG   22,4 3440528 22654 ./xemacs
% fg
[1]  + running    tail -f xemacs > /dev/null
^C
zsh: exit 130
% df -k .
Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/hdc4              1169519   1083042     26048  98% /


// Heikki
-- 
Heikki Vatiainen                  * hessu@cs.tut.fi
Tampere University of Technology  * Tampere, Finland