Re: Qmail - huge performance increase

[Craig Sanders <cas@taz.net.au>]

On Thu, Jun 21, 2001 at 01:45:23AM +0800, Jason Lim wrote:
> SO... by increasing conf-split to 97 (from the default of 20
> something afaik), each directory ends up only having a hundred or so
> files. Doing "ls" now is far speedier.
>
> I couldn't find any documentation anywhere stating this, so I'll share
> it with you all :-)

this is actually a well-known limitation of ext2fs and similar
file-systems - as soon as you get more than a thousand or so files in a
directory, performance takes a nosedive.

you can see this effect on news servers such as INN and also mail
servers which use Maildir when the users keep thousands of messages in
the one "folder", and in mail servers with many thousands of messages in
the queue...in fact, in any application which has lots of files in one
directory.

i'd recommend using reiserfs or perhaps xfs for these applications.

this is one of the major advantages that a modern filesystem like
reiserfs has over ext2 - you can have as many files as you want in a
directory and it doesn't suffer from this slowdown.

reiserfs is included in kernel 2.4.x now, and SGI's linux port of their
xfs is available as a patch. i've been using reiserfs on production
servers for ages with no problems (even using it on mission critical
mail servers), and have been trialling xfs since the 1.0 release...with
no problems so far, so i'm starting to use it on non-mission-critical
servers (e.g. my anonymous ftp server / debian mirror). if it lives up
to expectations, i'll start replacing reiserfs with xfs over the next
6-12 months.


BTW, even sendmail lets you set the hash queue depth (to use postfix
terminology, which i am more comfortable with :) these days. sendmail
used to have just one directory for all queue files....which made for
lousy performance.


craig

-- 
craig sanders <cas@taz.net.au>

Fabricati Diem, PVNC.
 -- motto of the Ankh-Morpork City Watch