Parallel unpack (was: Couple of patches)
Raphael Geissert <atomo64+debian@gmail.com> writes:
> Implemented, see attached mbox.
This all looks good to me except this one:
> diff --git a/collection/strings b/collection/strings
> index b649035..94bac2d 100755
> --- a/collection/strings
> +++ b/collection/strings
> @@ -34,7 +34,9 @@ for bin in $(grep ELF file-info | cut -d: -f1); do
> ;;
> *)
> mkdir -p "strings/$(dirname "$bin")"
> - strings "unpacked/$bin" > "strings/$bin"
> + strings "unpacked/$bin" > "strings/$bin" &
> ;;
> esac
> done
> +
> +wait
This could potentially spawn thousands of parallel strings commands for
packages with tons of binaries, which can be rather unfriendly for small
systems. The rest of the patch never spawns more than three or four
background processes at a time, and I think going up to maybe five or six
would be fine, but I don't want to have any place where we could launch a
potentially unbounded number of jobs.
For cases like this, if it's really worth parallelizing, we should instead
add the necessary machinery to maintain a queue of background jobs and
stall until one of them finishes if we have too many already started.
Applying the rest of it now.
--
Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>
Reply to: