Re: build of installation-guide on web server broken [Was: Re: debian-doc files on the Debian web server]
- To: Osamu Aoki <osamu@debian.org>
- Cc: debian-www <debian-www@lists.debian.org>
- Subject: Re: build of installation-guide on web server broken [Was: Re: debian-doc files on the Debian web server]
- From: Holger Wansing <hwansing@mailbox.org>
- Date: Tue, 1 Feb 2022 19:56:19 +0100
- Message-id: <[🔎] 20220201195619.039f230e42a884fa77b497b4@mailbox.org>
- In-reply-to: <65355ad51ecc27bfac041920f3bdcfe7b092e4e8.camel@debian.org>
- References: <3426ae4eaf46e3419c822c237a3c46812b446389.camel@debian.org> <69C9FA73-3189-442F-A9FB-3656C18F041E@mailbox.org> <a50726e91dd750f046c83a5bdd640880cd3c258f.camel@debian.org> <20220131190629.c61390fea4d9b6e81b3b06ba@mailbox.org> <65355ad51ecc27bfac041920f3bdcfe7b092e4e8.camel@debian.org>
Hi,
Osamu Aoki <osamu@debian.org> wrote (Wed, 02 Feb 2022 00:29:40 +0900):
> Hi,
>
> I am going to sleep :-)
>
> I will look into this on my week end.
>
> I suppose path needs some adjustment. I don't know where should it be.
>
> Since I have no direct access to the build environment, I need to run cron job to
> debug...
Ok.
I set up a test environment here locally and debugged the situation.
Found the error:
The log had:
starting the lessoften cron job at Mon Jan 31 14:49:01 UTC 2022
Already up to date.
run-parts: executing /srv/www.debian.org/cron/lessoften-parts/1ftpfiles
I: Download http://ftp.de.debian.org/debian/dists/sid/main/source/Sources.xz
I: Download: pool/main/i/installation-guide/installation-guide_20220129.dsc
I: Download: pool/main/i/installation-guide/installation-guide_20220129.tar.xz
I: Download http://ftp.de.debian.org/debian/dists/stable/main/source/Sources.xz
I: Download: pool/main/i/installation-guide/installation-guide_20210730.dsc
I: Download: pool/main/i/installation-guide/installation-guide_20210730.tar.xz
I: Download http://ftp.de.debian.org/debian/dists/oldstable/main/source/Sources.xz
I: Download: pool/main/i/installation-guide/installation-guide_20190622.dsc
I: Download: pool/main/i/installation-guide/installation-guide_20190622.tar.xz
So, it downloads the *.dsc and *.tar.xz files everytime.
That's because the check for the existence of the files used the wrong file path
(it searched for
"pool/main/i/installation-guide/installation-guide_20190622.dsc"
and not for
"installation-guide_20190622.dsc"
That caused the line
rm -f ${srcmask}_*
to be executed everytime, and wiped out the just downloaded *.dsc file, when
going through the for loop the second time (during processing the *.tar.xz file).
I fixed that, and now it says (here locally):
ned@t520:~/deb_holgerw/website/on-salsa/cron/cron$ ./lessoften-parts/1ftpfiles
I: Download http://ftp.de.debian.org/debian/dists/sid/main/source/Sources.xz
I: Skip: pool/main/i/installation-guide/installation-guide_20220129.dsc (already downloaded)
I: Skip: pool/main/i/installation-guide/installation-guide_20220129.tar.xz (already downloaded)
I: Download http://ftp.de.debian.org/debian/dists/stable/main/source/Sources.xz
I: Skip: pool/main/i/installation-guide/installation-guide_20210730.dsc (already downloaded)
I: Skip: pool/main/i/installation-guide/installation-guide_20210730.tar.xz (already downloaded)
I: Download http://ftp.de.debian.org/debian/dists/oldstable/main/source/Sources.xz
I: Skip: pool/main/i/installation-guide/installation-guide_20190622.dsc (already downloaded)
I: Skip: pool/main/i/installation-guide/installation-guide_20190622.tar.xz (already downloaded)
and now *.tar.xz and *.dsc files are both there in the target dir.
So I guess it will work tomorrow as it should.
Best regards
Holger
> Osamu
> -----Original Message-----
> From: Holger Wansing <hwansing@mailbox.org>
> To: Osamu Aoki <osamu@debian.org>
> Cc: debian-www@lists.debian.org
> Subject: build of installation-guide on web server broken [Was: Re: debian-doc files
> on the Debian web server]
> Date: Mon, 31 Jan 2022 19:06:29 +0100
>
> Hi Osamu,
>
>
> Osamu Aoki <osamu@debian.org> wrote (Thu, 04 Nov 2021 11:34:06 +0900):
> > Hi, (Sorry)
> >
> > On Wed, 2021-11-03 at 18:14 +0100, Holger Wansing wrote:
> > > Hi Osamu,
> > >
> > > Am 3. November 2021 15:45:44 MEZ schrieb Osamu Aoki <osamu@debian.org>:
> > > > Hi,
> > > >
> > > > I finally made update to the cron script so we don't rely on FTP protocol.
> > > > url = git@salsa.debian.org:webmaster-team/cron.git
> > > >
> > > > 526ebd0 ("Download deb via HTTP", 2021-11-01)
> > > >
> > > > No more use of FTP protocol
> > > >
> > > > Ensure to avoid unnecessary downloads of non-sid packages and already
> > > > downloaded packages.
> > > >
> > > > unpack once and do not reprocess
> > > >
> > > > Signed-off-by: Osamu Aoki <osamu@debian.org>
> > > >
> > > >
> > > > I stop making a copy of doc. I don't see the following data are used any more.
> > > > > $crondir/ftpfiles/doc
> > > >
> > > > Although it was tested on my machine, since this is rather big change, I am
> > > > afraid
> > > > something may break.
> > >
> > > As far as I can see, the installation-guide source
> > > packages are
> > > no longer downloaded by 1ftpfiles.
> > > This was handled by this line:
> > >
> > > wgetfiles installation-guide installation-guide_*
> >
> > I thought this was some kind of left-over. Sorry, you are getting *.dsc file and
> > tarball here. When I wrote wgetfiles, I didn't anticipate such uses.
> >
> > > Now, building of installation-guide can no longer work.
> > > (the lessoften-parts/1installation-guide script relies on the
> > > source files to be existing in $ftpdir/pool dir.)
> > >
> > > Or did I miss something?
> > >
> >
> > Give me sometime (2 days), I have some idea. Instead of Packages, we need to
> > download Sources there. Then it is trivial to download correct version. No
> > rmadison
> > complication needed.
>
> Two days ago, we had the first upload of installation-guide package since
> the above changings.
>
> Now I see, that the build is broken.
> At https://www-master.debian.org/build-logs/webwml/lessoften.log I see:
>
> ---------------------------------------------------------------------------------
> starting the lessoften cron job at Sun Jan 30 14:49:01 UTC 2022
> Already up to date.
> run-parts: executing /srv/www.debian.org/cron/lessoften-parts/1ftpfiles
> I: Download http://ftp.de.debian.org/debian/dists/sid/main/source/Sources.xz
> I: Download: pool/main/i/installation-guide/installation-guide_20220129.dsc
> I: Download: pool/main/i/installation-guide/installation-guide_20220129.tar.xz
> I: Download http://ftp.de.debian.org/debian/dists/stable/main/source/Sources.xz
> I: Download: pool/main/i/installation-guide/installation-guide_20210730.dsc
> I: Download: pool/main/i/installation-guide/installation-guide_20210730.tar.xz
> I: Download http://ftp.de.debian.org/debian/dists/oldstable/main/source/Sources.xz
> I: Download: pool/main/i/installation-guide/installation-guide_20190622.dsc
> I: Download: pool/main/i/installation-guide/installation-guide_20190622.tar.xz
>
> 1ftpfiles finished (at Sun Jan 30 14:49:08 UTC 2022)
>
> run-parts: executing /srv/www.debian.org/cron/lessoften-parts/1installation-guide
> no new installation-guide for stable (bullseye), skipping build
> extracting installation-guide source for testing (bookworm) ...
> dpkg-source: error: cannot read
> /srv/www.debian.org/cron/ftpfiles/sid/installation-guide_20220129.dsc: No such file
> or directory
> building installation-guide for testing (bookworm) ...
> /srv/www.debian.org/cron/lessoften-parts/1installation-guide: line 116: cd:
> installation-guide-20220129/build: No such file or directory
> FAILED to build the installmanual for testing (bookworm) ...
> no new installation-guide for oldstable (buster), skipping build
> ---------------------------------------------------------------------------------
>
> So, even if 1ftpfiles says, it downloads the *.dsc file, it is not there.
> The 1installation-guide script however needs it.
>
>
> Could you take a look?
>
>
> Thanks
> Holger
>
>
>
>
>
--
Holger Wansing <hwansing@mailbox.org>
PGP-Fingerprint: 496A C6E8 1442 4B34 8508 3529 59F1 87CA 156E B076
Reply to: