[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Patch to tidy up wget use, and add return=4 when 404's are seen



I'm still not sure whether this qualifies as a simplification or not :-/

Cheers,
FJP

On Saturday 01 March 2008, Philip Hands wrote:
> +++ packages/debian-installer-utils/debian/rules	2008-02-29
> 21:48:43.000000000 +0000 @@ -40,13 +40,14 @@
[...]
> +	dh_link -p di-utils /usr/lib/fetch-url/http /usr/lib/fetch-url/ftp

Looks like without leading / is preferred.

> +++ packages/debian-installer-utils/fetch-url-methods/http	2008-03-01
> 13:42:40.000000000 +0000 @@ -0,0 +1,38 @@
> +protocol_fetch() {
> +	local url="$1"
> +	local file="$2"
> +
> +	wget404() {
> +	# wrapper for wget that returns 4 for 404 errors while preserving
> STDERR & STDOUT
> +	# see README.wget404 in the debian-installer-utils udeb source for more
> info about this

Second comment can be shortened to:
   see also README.wget404 in debian-installer-utils source package

> +		local RETVAL=$( {  
> +			echo 1
> +			wget "$@" 2>&1 >&3 && echo %OK%
> +			echo %EOF%
> +			} | ( sed -ne '1{h;d};/server returned error
> 404/{p;s/.*/4/;h;d};/^%OK%$/{s/.*/0/;h;d};$!p;$x;$w /dev/fd/4' >&2 ) 4>&1
> +		) 3>&1
> +		echo "wget404 returns $RETVAL" >> /tmp/fetch-url.log
> +		return $RETVAL
> +	}

/me somewhat wonders how robust this is and whether it is safe on all 
architectures and also for other kernels...
Especially the writing to /dev/fd/4 scares me.

> +	# use the proxy for wgets (should speed things up)
> +	if db_get mirror/$proto/proxy; then
> +		export ${proto}_proxy="$RET"
> +	fi
> +
> +	for i in 1 2 3; do
> +		if [ -e "$file" ]; then
> +                        # busybox wget can sometimes become confused
> +                        # while resuming, so if it fails, restart
> +                        if wget404 -q -c "$url" -O "$file"; then
> +                                return 0
> +                        fi
> +		fi
> +
> +		wget404 -q "$url" -O "$file"
> +		local RET=$?

So basically you now first try a restart and if that fails a new full 
download and that three times, while in the old code we only did the full 
retry.
I'm not sure if that is wise, given the many caveats listed in the wget 
manpage for the -c option...

> +++ packages/net-retriever/net-retriever	2008-02-29 
> @@ -16,36 +16,11 @@
>  hostname="$RET"
>  db_get mirror/$protocol/directory
>  directory="$RET"
> -db_get mirror/$protocol/proxy
> -proxy="$RET"
> -if [ -n "$proxy" ]; then
> -	if [ "$protocol" = http ]; then
> -		export http_proxy="$proxy"
> -	elif [ "$protocol" = ftp ]; then
> -		export ftp_proxy="$proxy"
> -	fi
> -fi

Huh? Aren't we losing all support of the proxy here?

> +++ packages/preseed/preseed_fetch	2008-03-01 09:21:03.000000000 +0000
> +# Am suspicious that there are assumptions that we can overwrite 
> +# in preseeding, so lets discard old files for now
> +[ -e $dst ] && rm $dst

This can just be
rm -f $dst

Attachment: signature.asc
Description: This is a digitally signed message part.


Reply to: