Re: Serious performance bug in Perl

[Daniel Martin at cush <dtm12@jhunix.hcf.jhu.edu>]

Chris Fearnley <cjf@i21.com> writes:

> On Thu, Jun 18, 1998 at 12:38:45PM -0500, Richard Kaszeta wrote:
> > Christopher J. Fearnley writes ("Re: Serious performance bug in Perl"):
> > >to call it (instead of the default perl - 5.004.04-6).  Performance
> > >improved several hundred-fold.  So I believe the problem is either in
> > >perl or libc6.
> > >
> > >Any suggestions on how to resolve this?  As I said before the slowdown
> > >seems to occur in the get_current_uids subroutine (and possible
> > >get_current_gids).  Which has a loop on getpwent (and getgrent).
> > >
> > >Can anyone else duplicate this behavior?
> > 
> > I can duplicate this behavior.  Performance gets exponentially better
> > if I move my NIS password records into the local password file.  So in
> > my case I am tempted to blame libc6's NIS performance (which in other
> > circumstances I have found to be rather slow anyways)
> > 
> > Are you running NIS?
> > 
> > -- 
> > Richard W Kaszeta 			Graduate Student/Sysadmin
> > bofh@me.umn.edu				University of MN, ME Dept
> > http://www.menet.umn.edu/~kaszeta

It's a perl thing.  I can almost guarantee it.  The problem is perl's
shadow password support and it's getpw* functions.  Whenever these
functions are called (and assigned to a variable; if you just call
them and discard the value immediately this doesn't happen), perl
attempts to look up the shadow password entry associated with the
given struct passwd *, and this takes a while - especially since the
/etc/shadow file is closed immediately after each fetch.  (Which could 
be because perl is doing something like a getspnam call each time - of 
course, since it's doing these getspnam calls in order, one might think
that perhaps libc6 should be smart enough to not close and re-open
/etc/shadow each time, but maybe perl surrounds the getspnam calls
with setspent() and endspent() calls or somesuch for security
reasons).

This clearly seems to be a case that calls for a bit of magic (as in
magic variables that invoke arbitrary code each time they're
referenced) - the second element of the list returned by getpw* should
be a magic scalar, which is looked up only when asked for.  Could be
an idea that gets forwarded upstream.

In any case, at the moment you should be able to speed adduser up
considerably by modifying the get_current_uids routine as follows:

sub get_current_uids {
    my(@uids, $uid);
    my($olduid);
    $olduid = $<;
    $< = $>;
    $> = ((getpwnam "nobody")[2]);
    setpwent;
    push @uids, $uid while defined($uid = (getpwent)[2]);
    endpwent;
    $> = $<;
    $< = $olduid;
    @uids;
}

(This should work even if adduser is made an suid script executable by
only a certain group)

This still causes (you can see with an strace) many attempts to open
/etc/shadow, but they're all denied so the whole process is much faster.

For what it's worth, I made libc5 and libc6 C programs that just
called getpwent or getspent repeatedly - both blazed through my 1000
user passwd and shadow files in well under 0.05 sec each, though the
libc5 one was about twice as fast on getpwent and the libc6 one was
about twice fast on getspent.  Go figure.


--  
To UNSUBSCRIBE, email to debian-devel-request@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org