Bug#924172: www.debian.org: differences under english/ between builds in stretch and buster
Hi all,
On Mon, 8 Jun 2020 00:35:26 +0200
Cyril Brulebois <kibi@debian.org> wrote:
> (Looping in wml package maintainer + original author)
>
Some notes:
1. I am not the original author of wml, just the current upstream maintainer
(though I believe that I have invested quite a lot of work in it).
2. The newest release of wml is 2.28.0 (see
https://github.com/thewml/website-meta-language/releases ) while Debian sid is
stuck at 2.12.x (see https://packages.debian.org/sid/wml ). I'd rather not
support such an old release, so if the new version still exhibits some
regressions, please send a failing testcase to
https://github.com/thewml/website-meta-language/tree/master/src/wml_test and
I'll try to fix it.
For a joke about it, see:
https://www.shlomifish.org/humour/fortunes/tinic.html#hamakor-discs-mozilla-1.1-1
3. I recommend the Debian site to gradually transition away from using wml,
because while wml is fairly powerful and flexible, it is slow, quirky and hard
to learn. It is a hindrance that the debian site takes 3 hours to build on a
modern computer:
https://xkcd.com/303/
Converting some of my sites away from wml has shortened their build times
considerably.
Regards,
Shlomi
> Hi!
>
> Laura Arjona Reina <larjona@debian.org> (2020-06-07):
> > Thanks Cyril for your work here.
> >
> > We're trying to fix all the issues so the migration of www-master to
> > buster can be done as soon as possible.
>
> You're welcome, helping towards that goal was my original intent.
>
> > I'll comment on the issues you found:
> >
> > 1. Changes related to canonicalization(?) of the URL
> >
> > The files affected are:
> >
> > 1.1.- */News/news.*.rdf (all languages)
> >
> > All those files are built using /english/News/news.rdf.in which includes
> > this line:
> >
> > <?xml-stylesheet href="$(HOME)/security/dsa-rdf.css" type="text/css"?>
> >
> > it seems that the variable $(HOME) is interpreted differently by the
> > make or wml commands in buster.
> >
> > I have played with the "-D HOME~." line in /english/.wmlrc but it does
> > not solve the issue.
> >
> > The file is built (in english and all the other languages except
> > Chinese) by lines 36-37 of the /english/News/Makefile:
> >
> > $(WML) $(shell egrep '^-D (CUR_|CHAR)' ../.wmlrc) \
> > $(ENGLISHDIR)/News/news.rdf.in
> >
> > If I change that to
> >
> > $(WML) $(shell egrep '^-D (CUR_|CHAR)' ../.wmlrc) news.rdf.in
> >
> > then the English file is built with correct URL to the CSS, but the
> > other languages fail.
> >
> > I don't know how to solve this, except using an absolute reference to
> > the CSS file in the /english/News/news.rdf.in instead of the $(HOME)
> > variable.
>
> I'll skip this for now, to concentrate on the topic I've debugged this
> evening; this might be related though!
>
> > 1.2. */sitemap.*.html (all languages)
> >
> > I guess it's the same problem (that the $(HOME) variable is used and
> > interpreted wrongly with the new make or wml), but this file and how it
> > is built is more cryptic for me so I wouldn't know how to start.
> >
> > The sitemap would be completely broken until this issue is fixed, if
> > we migrate www-master to buster :/
> >
> > Help needed!
>
> For the record, we're talking about:
> https://salsa.debian.org/webmaster-team/webwml
>
> The problem can be triggered with:
> make -C english sitemap.en.html
>
> Another file in the same directory can be obtained with:
> make -C english index.en.html
>
>
> OK, so I've spent some time on this. Based on the HOME mention, I tried
> to switch it from `-D HOME~.` to `-D HOME=.` in english/.wmlrc but that
> wasn't sufficient. I had to turn *all* such defines in that file from
> `~` to `=` to get a reasonable sitemap.
>
> The wml_intro.7 manpage quickly explains the difference between both,
> but that seems to be happening in passes 2 and 3; english/sitemap.wml is
> one of the only files were pass protection is used, but that turned out
> to be a red herring…
>
>
> Adding the `-v2` flag to the `$(WML)` calls in english/Makefile made the
> issue apparent: as early as pass 1, we have different parameters being
> sent to wml_p1_ipp between stretch (2.0.12) and buster (2.12.2), among
> which:
>
> * stretch:
>
> "-DBUGS=Bugs"
> "-DINTRO=intro"
>
> * buster:
> "-DBUGS=../english/Bugs"
> "-DINTRO=../english/intro"
>
> for our english/sitemap.wml
>
> BUT
>
> that's not the case for index.en.html, built from english/index.wml!
>
> At this point, I suspected the origin of the change in behavior was the
> path to the input file:
>
> $(WML) … index.wml is OK
> $(WML) … ../english/sitemap.wml is not OK
>
> because the latter triggers the insertion of `../english/` in lots of
> places (defined with `~` instead of `=`) with 2.12.2, which was not the
> case with 2.0.12.
>
>
> Dropping the ../english/ prefix for sitemap.wml leads to a reasonable
> diff for the English sitemap (sitemap.en.html):
>
> --- stretch.en.html 2020-06-08 00:07:43.916768223 +0200
> +++ buster.en.html 2020-06-08 00:07:47.248781809 +0200
> @@ -4,8 +4,8 @@
> <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
> <title> Site map for Debian web pages </title>
> <link rel="author" href="mailto:webmaster@debian.org";>
> - <meta name="Generator" content="WML 2.0.12 (16-Apr-2008)">
> - <meta name="Modified" content="2020-06-07 22:07:43">
> + <meta name="Generator" content="WML 2.12.2">
> + <meta name="Modified" content="2020-06-08 00:07:46">
> <meta name="viewport" content="width=device-width">
> <meta name="mobileoptimized" content="300">
> <meta name="HandheldFriendly" content="true">
> @@ -377,7 +377,7 @@
> <p>
> Last Modified: Sun, Jun 7 21:49:47 UTC 2020
>  
> -Last Built: Sun, Jun 7 22:07:43 UTC 2020
> +Last Built: Sun, Jun 7 22:07:46 UTC 2020
> <br>
> Copyright © 1997-2020
> <a href="https://www.spi-inc.org/";>SPI</a> and others; See <a
> href="./license" rel="copyright">license terms</a><br>
>
> Therefore, I suspect we *might* be able to get away with this if we were
> to add a symlink from all language directories, from $LANG/sitemap.wml
> to ../english/sitemap.wml, as a companion change to dropping the
> ../english prefix mentioned above (only tested successfully on English).
> I'll be checking this hypothesis and possibly pushing a branch / opening
> an MR if that works fine.
>
> I'm sending this mail to reach out to upstream so they can comment on
> whether the change was intentional or an unwanted side effect.
>
> I might look at the News/RDF thing in the meanwhile.
>
>
> > 3. Changes in a log file
> >
> > Not important (and probably we shouldn't provide the card there, maybe
> > in the debian-flyers repo?)
> >
> > 4. Changes in ordering of coordinators
> > 5. Changes in ordering under wnpp
> > 6. Changes in order under l10n
> > 8. More ordering changes (architectures, DSAs)
> >
> > Thanks for reporting, and for the work towards reproducibility.
> > I think these are not blockers for the migration to buster of
> > www-master.debian.org
> >
> > Maybe we could open a specific bug about these "reproducibility issues"
> > and see if somebody is willing/able to work on it?
>
> I'm very fine with such a thing. I just wanted to classify differences
> between stuff that were possibly unimportant and stuff that might need
> fixing.
>
>
> Cheers,
--
Shlomi Fish https://www.shlomifish.org/
Understand what Open Source is - https://shlom.in/oss-fs
“God is dead.” — Nietzsche
“Nietzsche is dead.” — God (Unknown Source)
Please reply to list if it's a mailing list post - https://shlom.in/reply .
Reply to: