Re: Slow bash script
Andrew Sackville-West wrote:
On Sun, Sep 09, 2007 at 04:23:42PM -0400, Marty wrote:
The following script seems to run abnormally slow on a 400Mhz Sarge system,
getting only about one iteration per second in the while loop. It extracts
md5sums from a 180k Packages file and makes an indices file. I've narrowed
down the slowdown to the lines in the while loop starting with
"search=..."
how have you determined this?
I checked the output rate by outputing to stdout (instead of piping to gzip
after the "done" statement). I also timed it with the "time" command.
just FTR, I'd put echo "pre-search" or
"post-search" around each search= statement jsut to double check
that. I played with this a bit, and it seems to spend a lot of time
generating useless $search's although that's likely another
issue. What does a sampling of $inputline look like? I wonder if
you're getting much more than one line of the package data for each
$inputline and that results in the grep taking a lot more time than
you'd think.
If I echo each input line before the grep statement, it looks normal. I've used
similar combinations of pipes and commands in several other scripts, but I don't
recall such a slowdown.
FTR, you may do much better using something like awk to do this,
though I'm no script master, just an observation.
I tried awk instead of cut, with no dramatic change.
Reply to: