Re: Expires: headers for Packages.gz, Sources.gz
On Sat, Feb 10, 2001 at 12:49:08AM -0700, Jason Gunthorpe wrote:
> On Sat, 10 Feb 2001, Matt Zimmerman wrote:
> > I don't see what effect timezone skew would have...Expires: contains the
> > time zone information. If the object is expired after 24 hours, but the
> > file hasn't
>
> It does in HTTP, most of our mirrors are FTP based and have a nasty habit of
> adding hours to the times, this totally wacks the whole idea of using
> absolute Expires headers.
Hmm. Well, even if the mirrors are botching the timestamps, they should at
least be doing it in a consistent way (off by a fixed amount), so estimating
expiration time relative to the modification time should still work.
At any rate, as you mentioned later, there are a lot of mirrors, and nobody
seems very interested in implementing expiry.
> > been updated, I'm not sure how that's handled. What happens when an
> > object's expiration time (as specified by the origin) is in the past? At
> > any rate, an
>
> Then the cache has to IMS every query.
Blecch. In that case, the package files would have to be touched daily
regardless of whether or not they changed. A waste of bandwidth for ftp-based
mirroring.
> > I don't see how using Expires: could possibly make things worse than they
> > are now, and it seems like it would do a lot to help caches do the right
> > thing more often.
>
> Expires make it so things like stable (which has static package files) will
> always result in a IMS query, instead of 1 every day, as it is now.
It would only make sense to implement this for package files which change on a
daily basis.
> > Come to think of it, .deb's should get a very long expiry time, as they
> > should never change. RFC2068 recommends using an expiry time one year from
> > the response time.
>
> The default expiry mechanisms in a cache should handle this case well enough.
> DEBs do expire BTW, not their content, but the value in keeping them (we
> churn DEB files at the rate of ~150 meg/day in the archive)
I don't think that Expires: is intended to have any bearing on how long a cache
will hold onto the object, only how long it can be considered to be fresh.
> > import priority standard and higher packages into a CVS repository. Why
> > not fix the problem once and for all on the server side?
>
> Er, we have about 170 mirrors. Having even a noticable fraction of them
> insert expires headers that are worthwhile just isn't going to happen.
>
> For us the best solution is to let the clients do it because that is the side
> we control.
Quite right. I wish I could do this with Squid's refresh_pattern, but it has
no way to express mtime + a constant factor (only a percentage of the age).
> BTW, you can use apt-get to fetch those source files for you..
I am using apt-get to fetch the source archives. If it can also determine
which packages to fetch based on priority, do let me know how.
--
- mdz
Reply to: