Re: Password file with over 3000 users.

[Craig Sanders <cas@taz.net.au>]

On Tue, Sep 18, 2007 at 05:19:15PM +0200, Ian wrote:
> I have a server which runs with normal passwd/shadow/group/gshadow
> files. There are now over 3000 users and I am beginning to notice a
> performance slowdown and it is about time I looked for something more
> efficient - I like the idea of db or cdb database files.
> 
> My requirements:
> 
> - Must be pam compatible.
> 
> - Most users have MD5 passwords, but some are still crypt passwords. I
> do not have ready access to the original plain text passwords.
> 
> - I don't want to mess around with too much, so I must have real "Unix
> accounts" (UID's and home directories) for each user.
> 
> - Vanilla Debian "deb" packages.
> 
> What alternatives should I consider?

if you have the libnss-db package (part of nsswitch) installed, you have
everything you need already.

nsswitch.conf allows you to specify "db" as one of the sources for
passwd, shadow, group, and other files. they are read from a Berkeley
db file in /var/lib/misc (i.e. a quick indexed lookup rather than a
sequential search).

there is also a Makefile in /var/lib/misc for generating the .db versions.

to set up:

1. edit /etc/nsswitch.conf and insert the word "db" into the lines that you
want to use the db module.

e.g. change this:

passwd:         compat
group:          compat
shadow:         compat

to this:

passwd:         db compat
group:          db compat
shadow:         db compat


2. edit /etc/default/libnss-db to tell it which dbs to generate.

3. run "cd /var/lib/misc ; make"

4. set up a cron job (to run, say, every 5 minutes) to run the commands
in step 3 (i.e. "cd /var/lib/misc ; make"). you probably want to
redirect stdout and stderr to /dev/null so that root's mail doesn't get
flooded with make's output (mostly "make: Nothing to be done for `all'."
unless passwd,group, or shadow have changed)


see /usr/share/doc/libnss-db and other related documentation for more info.



NOTE: LDAP is also a good alternative, but a *LOT* more work to set up.
libnss-db is a simple way to speed up what already works by putting the
passwd etc files into hashed database files.


craig

PS: this works.  i did this several years ago on one server when the number of
accounts grew to about 5000.  there is one small catch - with the cron job
running every 5 minutes, there is a small window of time when the source files
in /etc have been updated but the .db versions haven't been regenerated yet.

the nsswitch.conf file will check both the db and the original source files in
order, so it does not prevent new accounts from logging in.  for account
deletions, however, the deleted account will still work until the .db files
are regenerated.  similarly, password changes will not take effect
immediately.

actually, it's been years - i can't remember if only the old password
(in /var/lib/misc/shadow.db) works, or if both the old (shadow.db) and
new (/etc/shadow) password will work. either way, that's only until the
cron job runs make again (i.e. at most, up to 5 minutes. or less if you
have cron run make more frequently).

if you have written scripts to assist with account
creation/deletion/changing, you could easily modify them to run "cd
/var/lib/misc ; make" after any change, thus eliminating the delay.

you still want the cron job, though, in case there are other ways for a
password to be changed - shell login by users or poppassd or samba, for
instance.



-- 
craig sanders <cas@taz.net.au>