Re: [PATCH 1/1] snapshot::web Fix internal redirects to farm
On Tue, 2024-11-19 at 10:02 +0100, Philipp Kern wrote:
> On 11/17/24 2:52 PM, MOESSBAUER, Felix wrote:
> > On Sat, 2024-11-16 at 23:48 +0100, Philipp Kern wrote:
> > > Hi,
> > >
> > > On 11/15/24 2:27 PM, Felix Moessbauer wrote:
> > > > We already have a rule to let Varnish follow the 302 redirects
> > > > from
> > > > the
> > > > /archive endpoint to the farm (/file) endpoint. This rule was
> > > > not
> > > > effective, though, as it assumed that the 302 redirects are
> > > > absolute
> > > > URIs (including the protocol and host part). We now change this
> > > > to
> > > > follow the realtive (internal) redirects, restoring this
> > > > behavior.
> > > >
> > > > A nice side-effect is that the artifacts are now delivered with
> > > > the
> > > > correct file-name (instead of the farm hash). Further it
> > > > effectively
> > > > cuts the number of http requests to s.d.o in half.
> > > I tried, but this caused Bad Requests to be emitted by Apache.
> >
> > Did you see any further indication what Apache did not like about
> > the
> > request? Could you try commenting out the "set bereq.http.host =
> > beresp.http.host;" line? Maybe the beresp does not contain this
> > header
> > (as it is technically a request header), hence overwriting the
> > correct
> > value in the request.
> >
> > I tested the patch locally with Varnish in front of the flask app
> > (without Apache). Maybe flask simply does not check this.
> >
> > Nonetheless, the current (unpatched) code is not effective, as the
> > 302
> > redirect is forwarded to the client (can easily be seen in curl).
>
> I tested the change on lw07, and the location header we see from the
> webapp is fully qualified, so the new proposed patch does not match
> either:
>
> > - RespHeader Location:
> > http://snapshot-lw07.debian.org/file/4c086f4fd3df97030a4e300a43ab3f897d5b15f8
That's interesting. The internal redirect seems to work properly for
that host. Then we just need my redirect in addition (e.g. for mlm-01).
I'll update my patch and resend it. Sample output:
curl -vv -s
http://snapshot-lw07.debian.org/archive/debian/20241119T082930Z/pool/main/0/0ad/0ad_0.0.23.1-2_amd64.deb
> /dev/null
* Trying 185.17.185.185:80...
* Connected to snapshot-lw07.debian.org (185.17.185.185) port 80 (#0)
> GET /archive/debian/20241119T082930Z/pool/main/0/0ad/0ad_0.0.23.1-
2_amd64.deb HTTP/1.1
> Host: snapshot-lw07.debian.org
> User-Agent: curl/7.88.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Tue, 19 Nov 2024 09:12:07 GMT
< Server: Apache
< X-Content-Type-Options: nosniff
< X-Frame-Options: sameorigin
< Referrer-Policy: no-referrer
< X-Xss-Protection: 1
< Permissions-Policy: interest-cohort=()
< Last-Modified: Wed, 09 Jan 2019 16:32:20 GMT
< ETag: "537d00-57f0901818c52"
< Content-Length: 5471488
< X-Clacks-Overhead: GNU Terry Pratchett
< Cache-Control: max-age=31536000, public
< X-Varnish: 98315 32778
< Age: 118
< Via: 1.1 varnish (Varnish/6.5)
< Accept-Ranges: bytes
< Connection: keep-alive
>
> On lw07 Apache did not fail to process the request. At this point I
> am
> also confused if the machines are set up identically - AIUI at least
> sallinen is configured differently/has different data.
They are definitely not - as we know by now. Unfortunately not even the
ETags are idendical:
Package from above:
- snapshot-lw07: 537d00-57f0901818c52
- snapshot.d.o (cache-fra-etou8220130-FRA): 537d00-57f09017dad00
Do we have any machine readable endpoint to get the version of the
snapshot software (e.g. the git commit).
Best regards,
Felix Moessbauer
>
> Kind regards
> Philipp Kern
--
Siemens AG, Technology
Linux Expert Center
Reply to: