On Sun, Sep 09, 2007 at 04:23:42PM -0400, Marty wrote: > The following script seems to run abnormally slow on a 400Mhz Sarge system, > getting only about one iteration per second in the while loop. It extracts > md5sums from a 180k Packages file and makes an indices file. I've narrowed > down the slowdown to the lines in the while loop starting with > "search=..." how have you determined this? just FTR, I'd put echo "pre-search" or "post-search" around each search= statement jsut to double check that. I played with this a bit, and it seems to spend a lot of time generating useless $search's although that's likely another issue. What does a sampling of $inputline look like? I wonder if you're getting much more than one line of the package data for each $inputline and that results in the grep taking a lot more time than you'd think. > The system is otherwise unloaded and not swapping when I see the slowdown. > Thanks in advance for any ideas or clues about the slowdown, or about how > to speed up the loop. > > > mkdir /mnt/install/debian-multimedia/indices > find /mnt/install/debian-multimedia/dists/etch/main/ |grep Packages |grep > -v z |xargs cat| \ > while read inputline > do > search=`echo $inputline | grep Filename |cut -f 2 -d ' '` > if [ "$search" != "" ] > then > filename=$search > fi > search=`echo $inputline | grep MD5 |cut -f 2 -d ' '` > if [ "$search" != "" ] > then > nextline="$search $filename" > echo $nextline > fi > done |gzip >/mnt/install/debian-multimedia/indices/md5sums.gz > FTR, you may do much better using something like awk to do this, though I'm no script master, just an observation. A
Attachment:
signature.asc
Description: Digital signature