Re: Cannot connect to snapshot.debian.org
Hi Peter,
the server refuses our connections.
wget
--delete-afterhttp://snapshot.debian.org/archive/debian/20100801T035459Z/
--2010-08-05 10:55:30--
http://snapshot.debian.org/archive/debian/20100801T035459Z/
Resolving snapshot.debian.org... 206.12.19.13
Connecting to snapshot.debian.org|206.12.19.13|:80... failed: Connection
refused.
Is our ip blocked again?
regards
Steffen
Peter Palfrader schrieb:
> Steffen Eichler schrieb am Montag, dem 26. Juli 2010:
>
>> thanks for your fast reply. Sorry for the trouble we have caused.
>>
>> The request came from our debian package caching system. All our servers
>> (over 400) running over this cache.
>> We solved this with apt-cacher-ng. Every sunday we change the package
>> source urls timestamp to the actual date.
>>
>> For example:
>> http://snapshot.debian.org/archive/debian/20100726T135349Z/
>> http://snapshot.debian.org/archive/debian-security/20100726T135349Z/
>> http://snapshot.debian.org/archive/backports.org/20100726T135349Z/
>> http://snapshot.debian.org/archive/debian-volatile/20100726T135349Z/
>>
>> Unfortunately it seems so that apt-cacher-ng passes all request for
>> /dists/ directly to your system and this caused the 550000 hits.
>> At the moment I switched back to the "normal" package mirrors
>> (packages.debian.org, backports.org, ...).
>>
>> I hope to solve the problem with apt-cacher-ng and to reduce the hits on
>> snapshots.debian.org. I would be very pleased if you could disable the
>> block of our ip. So I can test an possible solution.
>
> That's an interesting usecase. I have removed the block for now, tho if
> it causes problems again we'll have to start rate limiting requests.
>
>
> Looking at the status codes returned a good chunk is for files that don't even
> exist:
> | # http code
> | 1026 200
> | 352016 206
> | 199136 404
> | 117 500
>
> It'd be nice if apt-cacher could be taught to not be so stupid about
> requests and caching tho. Snapshot sends a
> 'Cache-Control: public, max-age=864000' header for most static content,
> so in theory a client should not even come back for requests that got
> answered successfully for a day.
>
> Yet we had almost 20k requests for
> archive/backports.org/20100719T095641Z/dists/lenny-backports/contrib/binary-amd64/Packages.bz2
> alone.
>
>
> Maybe we should just set up squid as a reverse proxy on snapshot.d.o to
> get rid of the more stupid clients out there.
>
>
> Cheers,
> Peter
Reply to: