[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

www.debian.org: please reduce entropy / make build more deterministic



[ forwarding without the first patch which made this mail too big for
  the mailing list, and for which Laura has a real solution anyway;
  filed in the BTS as #926550 ]

Cyril Brulebois (2019-04-06):
| Package: www.debian.org
| Severity: normal
| User: www.debian.org@packages.debian.org
| Usertags: scripts
| 
| Hi,
| 
| As seen in other bug reports, I've been looking at builds within stretch
| and buster. I've started a branch called “fight-entropy” in which I've
| reworked Makefiles so that one can get a more deterministic order when
| building the website from scratch, with -j1 (to disable parallelism).
| 
| I've also applied workarounds for #924333 and #924365 (both attached),
| so that the build finishes successfully.
| 
| You'll find attached the diff of filtered build logs. Those are post
| processed to get rid of the path to the checkout (stretch vs. buster),
| and to suppress wget output.
| 
| There are a couple of different touch calls, some differences of
| versions for tools that are used to build documents (hello, LaTeX2e,
| Babel, dvips!), and english/international/l10n/scripts/gen-files.pl
| would likely benefit from an extra sort or two. But at least the build
| logs with -j1 are much more easily reviewable with the fight-entropy
| branch.
| 
| Slightly more worrisome: some encoding related issues (“Invalid UTF8” or
| “Malformed UTF-8 character”) that might need to be addressed.
| 
| I still have to investigate the differences between generated files
| (stretch vs. buster), but results will be reported in a separate bug
| report.
| 
| All in all, please consider merging:
|   https://salsa.debian.org/webmaster-team/webwml/commits/fight-entropy
| 
| and maybe consider the attached workaround patches as well.

I might have mentioned that on IRC only: build time gets a big bump:
270 minutes in stretch, 350 minutes in buster (keeping in mind that's
with a single core anyway; but the performance penalty is real…).


Cheers,
-- 
Cyril Brulebois (kibi@debian.org)            <https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant
>From 47a1c96f743d970aa1a52b35b3d0f7ebd9c88bc8 Mon Sep 17 00:00:00 2001
From: Cyril Brulebois <kibi@debian.org>
Date: Sat, 6 Apr 2019 12:21:54 +0000
Subject: [PATCH 2/2] Disable turkish which doesn't build on buster (#924365).

---
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index d368e08b785..800795450eb 100644
--- a/Makefile
+++ b/Makefile
@@ -4,7 +4,7 @@ LANGUAGES := english albanian arabic armenian bulgarian catalan chinese croatian
              danish dutch esperanto finnish french galician german greek hebrew \
              hungarian indonesian italian japanese korean lithuanian \
              norwegian persian polish portuguese romanian russian slovak slovene \
-             spanish swedish tamil turkish ukrainian vietnamese
+             spanish swedish tamil ukrainian vietnamese
 
 LANGUAGES-install := $(addsuffix -install,$(LANGUAGES))
 LANGUAGES-clean := $(addsuffix -clean,$(LANGUAGES))
-- 
2.11.0

--- stretch/stretch-j1.log.filt.wget	2019-04-06 20:19:48.198683352 +0200
+++ buster/buster-j1.log.filt.wget	2019-04-06 20:19:55.210799691 +0200
@@ -4,6 +4,8 @@
 make[2]: Entering directory '??????ROOT??????webwml/english/po'
 make[2]: Nothing to be done for 'install'.
 make[2]: Leaving directory '??????ROOT??????webwml/english/po'
+touch ../english/template/debian/basic.wml # because of navbar.wml language_names.wml footer.wml
+touch ../english/template/debian/template.wml # because of basic.wml
 (cd ./.. && ./build_vcs_cache.pl)
 Initialising VCS cache for performance
  ... done
@@ -13,6 +15,7 @@
 ./../touch_translations.pl ??????ROOT??????webwml/english/contact.wml en
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:donations.en.html@g+w   donations.wml
 ./../touch_translations.pl ??????ROOT??????webwml/english/donations.wml en
+touch ../english/template/debian/mainpage.wml # because of basic.wml
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:index.en.html@g+w   index.wml
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:license.en.html@g+w   license.wml
 ./../touch_translations.pl ??????ROOT??????webwml/english/license.wml en
@@ -32,6 +35,7 @@
 make[2]: Entering directory '??????ROOT??????webwml/english/Bugs'
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:Access.en.html@g+w   Access.wml
 ../../touch_translations.pl ??????ROOT??????webwml/english/Bugs/Access.wml en
+touch ../../english/Bugs/pkgreport-opts.inc # because of common_tags.wml templates.pot countries.pot langs.pot date.pot bugs.pot
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:Developer.en.html@g+w   Developer.wml
 ../../touch_translations.pl ??????ROOT??????webwml/english/Bugs/Developer.wml en
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:Reporting.en.html@g+w   Reporting.wml
@@ -52,6 +56,7 @@
 make[2]: Leaving directory '??????ROOT??????webwml/english/Bugs'
 make -C CD
 make[2]: Entering directory '??????ROOT??????webwml/english/CD'
+touch ../../english/template/debian/cdimage.wml # because of languages.wml links.tags.wml basic.wml
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:free-linux-cd.en.html@g+w   free-linux-cd.wml
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:index.en.html@g+w   index.wml -I ../../english/CD/
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:misc.en.html@g+w   misc.wml
@@ -159,6 +164,7 @@
 ../../touch_translations.pl ??????ROOT??????webwml/english/News/index.wml en
 make -C 1997
 make[3]: Entering directory '??????ROOT??????webwml/english/News/1997'
+touch ../../../english/template/debian/news.wml # because of basic.wml
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:1.3.en.html@g+w   1.3.wml
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:19970301.en.html@g+w   19970301.wml
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:19970307a.en.html@g+w   19970307a.wml
@@ -688,7 +694,9 @@
 make[3]: Leaving directory '??????ROOT??????webwml/english/News/press'
 make -C weekly
 make[3]: Entering directory '??????ROOT??????webwml/english/News/weekly'
+touch ../../../english/template/debian/projectnews/header.wml # because of template.wml
 cd . && wml -q -D CUR_YEAR=2019 -o UNDEFuEN:contributing.en.html@g+w   contributing.wml
+touch ../../../english/template/debian/weeklynews/header.wml # because of template.wml
 cd . && wml -q -D CUR_YEAR=2019 -o UNDEFuEN:index.en.html@g+w   index.wml
 cd . && wml -q -D CUR_YEAR=2019 -o UNDEFuEN:oldurl.en.html@g+w   oldurl.wml
 cd 1999/10 && wml -q -D CUR_YEAR=2019 -o UNDEFuEN:index.en.html@g+w   index.wml
@@ -1354,6 +1362,7 @@
 make[3]: Leaving directory '??????ROOT??????webwml/english/blends/Pics'
 make -C gis
 make[3]: Entering directory '??????ROOT??????webwml/english/blends/gis'
+touch ../../../english/template/debian/blend.wml # because of template.wml
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:about.en.html@g+w   about.wml
 ../../../touch_translations.pl ??????ROOT??????webwml/english/blends/gis/about.wml en
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:contact.en.html@g+w   contact.wml
@@ -1895,6 +1904,7 @@
 ../../../touch_translations.pl ??????ROOT??????webwml/english/devel/debian-installer/builds.wml en
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:errata.en.html@g+w   errata.wml
 ../../../touch_translations.pl ??????ROOT??????webwml/english/devel/debian-installer/errata.wml en
+touch ../../../english/devel/debian-installer/images.data
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:index.en.html@g+w   index.wml
 ../../../touch_translations.pl ??????ROOT??????webwml/english/devel/debian-installer/index.wml en
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:svn.en.html@g+w   svn.wml
@@ -2218,14 +2228,13 @@
 make -C misc
 make[3]: Entering directory '??????ROOT??????webwml/english/devel/misc'
 latex card.tex
-This is pdfTeX, Version 3.14159265-2.6-1.40.17 (TeX Live 2016/Debian) (preloaded format=latex)
+This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2019/dev/Debian) (preloaded format=latex)
  restricted \write18 enabled.
 entering extended mode
 (./card.tex
-LaTeX2e <2017/01/01> patch level 3
-Babel <3.9r> and hyphenation patterns for 83 language(s) loaded.
+LaTeX2e <2018-12-01>
 (/usr/share/texlive/texmf-dist/tex/latex/base/article.cls
-Document Class: article 2014/09/29 v1.4h Standard LaTeX document class
+Document Class: article 2018/09/03 v1.4i Standard LaTeX document class
 (/usr/share/texlive/texmf-dist/tex/latex/base/size11.clo))
 (/usr/share/texlive/texmf-dist/tex/latex/graphics/graphicx.sty
 (/usr/share/texlive/texmf-dist/tex/latex/graphics/keyval.sty)
@@ -2247,8 +2256,8 @@
 Output written on card.dvi (1 page, 5096 bytes).
 Transcript written on card.log.
 dvips card.dvi -o card.ps
-This is dvips(k) 5.996 Copyright 2016 Radical Eye Software (www.radicaleye.com)
-' TeX output 2019.04.06:1230' -> card.ps
+This is dvips(k) 5.998 Copyright 2018 Radical Eye Software (www.radicaleye.com)
+' TeX output 2019.04.06:1232' -> card.ps
 </usr/share/texlive/texmf-dist/dvips/base/tex.pro>
 </usr/share/texlive/texmf-dist/fonts/enc/dvips/base/8r.enc>
 </usr/share/texlive/texmf-dist/dvips/base/texps.pro>
@@ -2386,6 +2395,7 @@
 make[2]: Entering directory '??????ROOT??????webwml/english/doc'
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:books.en.html@g+w   books.wml
 ../../touch_translations.pl ??????ROOT??????webwml/english/doc/books.wml en
+touch ../../english/template/debian/ddp.wml # because of basic.wml
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:cvs.en.html@g+w   cvs.wml
 ../../touch_translations.pl ??????ROOT??????webwml/english/doc/cvs.wml en
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:ddp.en.html@g+w   ddp.wml
@@ -2431,6 +2441,7 @@
 ../../touch_translations.pl ??????ROOT??????webwml/english/events/talks.wml en
 make -C 1998
 make[3]: Entering directory '??????ROOT??????webwml/english/events/1998'
+touch ../../../english/template/debian/past_event.wml # because of basic.wml
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:1118-sane.en.html@g+w   1118-sane.wml
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:1205-vatech-installfest.en.html@g+w   1205-vatech-installfest.wml
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:1218-japanexpo.en.html@g+w   1218-japanexpo.wml
@@ -2472,6 +2483,7 @@
 make[3]: Leaving directory '??????ROOT??????webwml/english/events/2000'
 make -C 2001
 make[3]: Entering directory '??????ROOT??????webwml/english/events/2001'
+touch ../../../english/template/debian/event.wml # because of basic.wml
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:0126-linuxexpo-amsterdam.en.html@g+w   0126-linuxexpo-amsterdam.wml
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:0127-linuxconf.en.html@g+w   0127-linuxconf.wml
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:0307-linuxexpo-sydney.en.html@g+w   0307-linuxexpo-sydney.wml
@@ -3271,126 +3283,126 @@
 ../../../../english/international/l10n/scripts/isoquery.pl
 perl ../../../../english/international/l10n/scripts/gen-files.pl --sort=../../../../english/international/l10n/data/popcon --dist=unstable --l10ndir=../../../../english/international/l10n  --langs --po --podebconf --po4a
 Use of uninitialized value $date in concatenation (.) or string at ../../../../english/international/l10n/scripts/gen-files.pl line 666.
+Unable to delete contrib-nn.exc - No such file or directory
+Unable to delete non-free-nn.exc - No such file or directory
 Unable to delete contrib-bn.exc - No such file or directory
 Unable to delete non-free-bn.exc - No such file or directory
-Unable to delete contrib-fa.exc - No such file or directory
-Unable to delete non-free-fa.exc - No such file or directory
-Unable to delete contrib-he.exc - No such file or directory
-Unable to delete non-free-he.exc - No such file or directory
-Unable to delete contrib-lt_LT.exc - No such file or directory
-Unable to delete non-free-lt_LT.exc - No such file or directory
+Unable to delete contrib-brx.exc - No such file or directory
+Unable to delete non-free-brx.exc - No such file or directory
+Unable to delete contrib-gl.exc - No such file or directory
+Unable to delete non-free-gl.exc - No such file or directory
+Unable to delete contrib-is.exc - No such file or directory
+Unable to delete non-free-is.exc - No such file or directory
+Unable to delete contrib-lo.exc - No such file or directory
+Unable to delete non-free-lo.exc - No such file or directory
+Unable to delete contrib-ky.exc - No such file or directory
+Unable to delete non-free-ky.exc - No such file or directory
+Unable to delete contrib-th.exc - No such file or directory
+Unable to delete non-free-th.exc - No such file or directory
 Unable to delete contrib-am.exc - No such file or directory
 Unable to delete non-free-am.exc - No such file or directory
-Unable to delete contrib-mg.exc - No such file or directory
-Unable to delete non-free-mg.exc - No such file or directory
-Unable to delete contrib-ne.exc - No such file or directory
-Unable to delete non-free-ne.exc - No such file or directory
+Unable to delete contrib-wo.exc - No such file or directory
+Unable to delete non-free-wo.exc - No such file or directory
+Unable to delete contrib-la.exc - No such file or directory
+Unable to delete non-free-la.exc - No such file or directory
+Unable to delete contrib-lv_LV.exc - No such file or directory
+Unable to delete non-free-lv_LV.exc - No such file or directory
+Unable to delete contrib-ia.exc - No such file or directory
+Unable to delete non-free-ia.exc - No such file or directory
+Unable to delete contrib-hy.exc - No such file or directory
+Unable to delete non-free-hy.exc - No such file or directory
+Unable to delete contrib-vi.exc - No such file or directory
+Unable to delete non-free-vi.exc - No such file or directory
+Unable to delete contrib-ilo.exc - No such file or directory
+Unable to delete non-free-ilo.exc - No such file or directory
+Unable to delete contrib-nso.exc - No such file or directory
+Unable to delete non-free-nso.exc - No such file or directory
+Unable to delete contrib-es_MX.exc - No such file or directory
+Unable to delete non-free-es_MX.exc - No such file or directory
+Unable to delete contrib-ro.exc - No such file or directory
+Unable to delete non-free-ro.exc - No such file or directory
+Unable to delete contrib-et.exc - No such file or directory
+Unable to delete non-free-et.exc - No such file or directory
+Unable to delete contrib-lt_LT.exc - No such file or directory
+Unable to delete non-free-lt_LT.exc - No such file or directory
+Unable to delete contrib-ks.exc - No such file or directory
+Unable to delete non-free-ks.exc - No such file or directory
+Unable to delete contrib-be.exc - No such file or directory
+Unable to delete non-free-be.exc - No such file or directory
 Unable to delete contrib-br.exc - No such file or directory
 Unable to delete non-free-br.exc - No such file or directory
-Unable to delete contrib-ka.exc - No such file or directory
-Unable to delete non-free-ka.exc - No such file or directory
+Unable to delete contrib-ne.exc - No such file or directory
+Unable to delete non-free-ne.exc - No such file or directory
+Unable to delete contrib-ur.exc - No such file or directory
+Unable to delete non-free-ur.exc - No such file or directory
 Unable to delete contrib-zh_HK.exc - No such file or directory
 Unable to delete non-free-zh_HK.exc - No such file or directory
 Unable to delete contrib-dz.exc - No such file or directory
 Unable to delete non-free-dz.exc - No such file or directory
-Unable to delete contrib-nso.exc - No such file or directory
-Unable to delete non-free-nso.exc - No such file or directory
-Unable to delete contrib-ur.exc - No such file or directory
-Unable to delete non-free-ur.exc - No such file or directory
-Unable to delete contrib-si_LK.exc - No such file or directory
-Unable to delete non-free-si_LK.exc - No such file or directory
-Unable to delete contrib-nds.exc - No such file or directory
-Unable to delete non-free-nds.exc - No such file or directory
-Unable to delete contrib-lv_LV.exc - No such file or directory
-Unable to delete non-free-lv_LV.exc - No such file or directory
-Unable to delete contrib-bal.exc - No such file or directory
-Unable to delete non-free-bal.exc - No such file or directory
-Unable to delete contrib-wo.exc - No such file or directory
-Unable to delete non-free-wo.exc - No such file or directory
-Unable to delete contrib-lt.exc - No such file or directory
-Unable to delete non-free-lt.exc - No such file or directory
-Unable to delete contrib-km.exc - No such file or directory
-Unable to delete non-free-km.exc - No such file or directory
-Unable to delete contrib-mn.exc - No such file or directory
-Unable to delete non-free-mn.exc - No such file or directory
-Unable to delete contrib-my.exc - No such file or directory
-Unable to delete non-free-my.exc - No such file or directory
-Unable to delete contrib-eo.exc - No such file or directory
-Unable to delete non-free-eo.exc - No such file or directory
-Unable to delete contrib-ro.exc - No such file or directory
-Unable to delete non-free-ro.exc - No such file or directory
-Unable to delete contrib-tl.exc - No such file or directory
-Unable to delete non-free-tl.exc - No such file or directory
+Unable to delete contrib-aln.exc - No such file or directory
+Unable to delete non-free-aln.exc - No such file or directory
+Unable to delete contrib-fa.exc - No such file or directory
+Unable to delete non-free-fa.exc - No such file or directory
 Unable to delete contrib-ast.exc - No such file or directory
 Unable to delete non-free-ast.exc - No such file or directory
-Unable to delete contrib-hy.exc - No such file or directory
-Unable to delete non-free-hy.exc - No such file or directory
-Unable to delete contrib-si.exc - No such file or directory
-Unable to delete non-free-si.exc - No such file or directory
-Unable to delete contrib-et.exc - No such file or directory
-Unable to delete non-free-et.exc - No such file or directory
-Unable to delete contrib-brx.exc - No such file or directory
-Unable to delete non-free-brx.exc - No such file or directory
+Unable to delete contrib-ku.exc - No such file or directory
+Unable to delete non-free-ku.exc - No such file or directory
+Unable to delete contrib-vi_VN.exc - No such file or directory
+Unable to delete non-free-vi_VN.exc - No such file or directory
+Unable to delete contrib-sl.exc - No such file or directory
+Unable to delete non-free-sl.exc - No such file or directory
+Unable to delete contrib-lt.exc - No such file or directory
+Unable to delete non-free-lt.exc - No such file or directory
+Unable to delete contrib-kk.exc - No such file or directory
+Unable to delete non-free-kk.exc - No such file or directory
+Unable to delete contrib-ka.exc - No such file or directory
+Unable to delete non-free-ka.exc - No such file or directory
+Unable to delete contrib-mg.exc - No such file or directory
+Unable to delete non-free-mg.exc - No such file or directory
+Unable to delete contrib-nds.exc - No such file or directory
+Unable to delete non-free-nds.exc - No such file or directory
 Unable to delete contrib-xh.exc - No such file or directory
 Unable to delete non-free-xh.exc - No such file or directory
-Unable to delete contrib-tg.exc - No such file or directory
-Unable to delete non-free-tg.exc - No such file or directory
-Unable to delete contrib-la.exc - No such file or directory
-Unable to delete non-free-la.exc - No such file or directory
-Unable to delete contrib-aln.exc - No such file or directory
-Unable to delete non-free-aln.exc - No such file or directory
+Unable to delete contrib-af.exc - No such file or directory
+Unable to delete non-free-af.exc - No such file or directory
+Unable to delete contrib-he.exc - No such file or directory
+Unable to delete non-free-he.exc - No such file or directory
+Unable to delete contrib-si.exc - No such file or directory
+Unable to delete non-free-si.exc - No such file or directory
 Unable to delete contrib-bn_BD.exc - No such file or directory
 Unable to delete non-free-bn_BD.exc - No such file or directory
+Unable to delete contrib-tg.exc - No such file or directory
+Unable to delete non-free-tg.exc - No such file or directory
+Unable to delete contrib-tl.exc - No such file or directory
+Unable to delete non-free-tl.exc - No such file or directory
 Unable to delete contrib-bo.exc - No such file or directory
 Unable to delete non-free-bo.exc - No such file or directory
-Unable to delete contrib-ia.exc - No such file or directory
-Unable to delete non-free-ia.exc - No such file or directory
-Unable to delete contrib-az.exc - No such file or directory
-Unable to delete non-free-az.exc - No such file or directory
-Unable to delete contrib-vi_VN.exc - No such file or directory
-Unable to delete non-free-vi_VN.exc - No such file or directory
-Unable to delete contrib-es_MX.exc - No such file or directory
-Unable to delete non-free-es_MX.exc - No such file or directory
-Unable to delete contrib-gl.exc - No such file or directory
-Unable to delete non-free-gl.exc - No such file or directory
+Unable to delete contrib-si_LK.exc - No such file or directory
+Unable to delete non-free-si_LK.exc - No such file or directory
+Unable to delete contrib-eo.exc - No such file or directory
+Unable to delete non-free-eo.exc - No such file or directory
 Unable to delete contrib-cy.exc - No such file or directory
 Unable to delete non-free-cy.exc - No such file or directory
-Unable to delete contrib-ku.exc - No such file or directory
-Unable to delete non-free-ku.exc - No such file or directory
-Unable to delete contrib-be.exc - No such file or directory
-Unable to delete non-free-be.exc - No such file or directory
-Unable to delete contrib-lo.exc - No such file or directory
-Unable to delete non-free-lo.exc - No such file or directory
-Unable to delete contrib-kk.exc - No such file or directory
-Unable to delete non-free-kk.exc - No such file or directory
-Unable to delete contrib-sl.exc - No such file or directory
-Unable to delete non-free-sl.exc - No such file or directory
-Unable to delete contrib-ga.exc - No such file or directory
-Unable to delete non-free-ga.exc - No such file or directory
-Unable to delete contrib-zu.exc - No such file or directory
-Unable to delete non-free-zu.exc - No such file or directory
+Unable to delete contrib-az.exc - No such file or directory
+Unable to delete non-free-az.exc - No such file or directory
+Unable to delete contrib-my.exc - No such file or directory
+Unable to delete non-free-my.exc - No such file or directory
+Unable to delete contrib-km.exc - No such file or directory
+Unable to delete non-free-km.exc - No such file or directory
 Unable to delete contrib-id.exc - No such file or directory
 Unable to delete non-free-id.exc - No such file or directory
-Unable to delete contrib-ky.exc - No such file or directory
-Unable to delete non-free-ky.exc - No such file or directory
-Unable to delete contrib-nn.exc - No such file or directory
-Unable to delete non-free-nn.exc - No such file or directory
-Unable to delete contrib-af.exc - No such file or directory
-Unable to delete non-free-af.exc - No such file or directory
-Unable to delete contrib-ilo.exc - No such file or directory
-Unable to delete non-free-ilo.exc - No such file or directory
-Unable to delete contrib-ks.exc - No such file or directory
-Unable to delete non-free-ks.exc - No such file or directory
 Unable to delete contrib-sq.exc - No such file or directory
 Unable to delete non-free-sq.exc - No such file or directory
+Unable to delete contrib-zu.exc - No such file or directory
+Unable to delete non-free-zu.exc - No such file or directory
+Unable to delete contrib-bal.exc - No such file or directory
+Unable to delete non-free-bal.exc - No such file or directory
+Unable to delete contrib-mn.exc - No such file or directory
+Unable to delete non-free-mn.exc - No such file or directory
 Unable to delete contrib-lv.exc - No such file or directory
 Unable to delete non-free-lv.exc - No such file or directory
-Unable to delete contrib-is.exc - No such file or directory
-Unable to delete non-free-is.exc - No such file or directory
-Unable to delete contrib-th.exc - No such file or directory
-Unable to delete non-free-th.exc - No such file or directory
-Unable to delete contrib-vi.exc - No such file or directory
-Unable to delete non-free-vi.exc - No such file or directory
+Unable to delete contrib-ga.exc - No such file or directory
+Unable to delete non-free-ga.exc - No such file or directory
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:index.en.html@g+w   index.wml
 ../../../../touch_translations.pl ??????ROOT??????webwml/english/international/l10n/po/index.wml en
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:pot.en.html@g+w   pot.wml
@@ -3503,6 +3515,7 @@
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:index.en.html@g+w   index.wml
 make -C 2014
 make[4]: Entering directory '??????ROOT??????webwml/english/lts/security/2014'
+touch ../../../../english/template/debian/security.wml # because of basic.wml
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:dla-0016.en.html@g+w   dla-0016.wml
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:dla-0018.en.html@g+w   dla-0018.wml
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:dla-0019.en.html@g+w   dla-0019.wml
@@ -5630,6 +5643,7 @@
 ../../../touch_translations.pl ??????ROOT??????webwml/english/releases/etch/credits.wml en
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:errata.en.html@g+w   errata.wml
 ../../../touch_translations.pl ??????ROOT??????webwml/english/releases/etch/errata.wml en
+touch ../../../english/releases/etch/release.data
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:etchnhalf.en.html@g+w   etchnhalf.wml
 ../../../touch_translations.pl ??????ROOT??????webwml/english/releases/etch/etchnhalf.wml en
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:index.en.html@g+w   index.wml
@@ -5689,6 +5703,7 @@
 ../../../touch_translations.pl ??????ROOT??????webwml/english/releases/lenny/credits.wml en
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:errata.en.html@g+w   errata.wml
 ../../../touch_translations.pl ??????ROOT??????webwml/english/releases/lenny/errata.wml en
+touch ../../../english/releases/lenny/release.data
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:index.en.html@g+w   index.wml
 ../../../touch_translations.pl ??????ROOT??????webwml/english/releases/lenny/index.wml en
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:installmanual.en.html@g+w   installmanual.wml
@@ -5729,6 +5744,7 @@
 ../../../touch_translations.pl ??????ROOT??????webwml/english/releases/sarge/credits.wml en
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:errata.en.html@g+w   errata.wml
 ../../../touch_translations.pl ??????ROOT??????webwml/english/releases/sarge/errata.wml en
+touch ../../../english/releases/sarge/release.data
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:index.en.html@g+w   index.wml
 ../../../touch_translations.pl ??????ROOT??????webwml/english/releases/sarge/index.wml en
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:installmanual.en.html@g+w   installmanual.wml
@@ -5761,6 +5777,7 @@
 ../../../touch_translations.pl ??????ROOT??????webwml/english/releases/squeeze/credits.wml en
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:errata.en.html@g+w   errata.wml
 ../../../touch_translations.pl ??????ROOT??????webwml/english/releases/squeeze/errata.wml en
+touch ../../../english/releases/squeeze/release.data
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:index.en.html@g+w   index.wml
 ../../../touch_translations.pl ??????ROOT??????webwml/english/releases/squeeze/index.wml en
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:installmanual.en.html@g+w   installmanual.wml
@@ -5801,6 +5818,7 @@
 ../../../touch_translations.pl ??????ROOT??????webwml/english/releases/wheezy/credits.wml en
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:errata.en.html@g+w   errata.wml
 ../../../touch_translations.pl ??????ROOT??????webwml/english/releases/wheezy/errata.wml en
+touch ../../../english/releases/wheezy/release.data
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:index.en.html@g+w   index.wml
 ../../../touch_translations.pl ??????ROOT??????webwml/english/releases/wheezy/index.wml en
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:installmanual.en.html@g+w   installmanual.wml
@@ -10598,7 +10616,7 @@
 make[3]: Leaving directory '??????ROOT??????webwml/english/security/key-rollover'
 make -C oval
 make[3]: Entering directory '??????ROOT??????webwml/english/security/oval'
---2019-04-06 13:03:14--  https://security-tracker.debian.org/tracker/data/json
+--2019-04-06 13:14:38--  https://security-tracker.debian.org/tracker/data/json
 Resolving security-tracker.debian.org (security-tracker.debian.org)... 23.111.9.35
 Connecting to security-tracker.debian.org (security-tracker.debian.org)|23.111.9.35|:443... connected.
 HTTP request sent, awaiting response... 200 OK
@@ -10640,6 +10658,7 @@
 make[2]: Leaving directory '??????ROOT??????webwml/english/security'
 make -C users
 make[2]: Entering directory '??????ROOT??????webwml/english/users'
+touch ../../english/template/debian/users.wml # because of basic.wml
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:index.en.html@g+w   index.wml
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:who_template.en.html@g+w   who_template.wml
 ../../touch_translations.pl ??????ROOT??????webwml/english/users/who_template.wml en
@@ -12222,6 +12241,7 @@
 ../../touch_translations.pl ??????ROOT??????webwml/english/women/participate.wml en
 make -C profiles
 make[3]: Entering directory '??????ROOT??????webwml/english/women/profiles'
+touch ../../../english/template/debian/profiles.wml # because of basic.wml
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:akira.en.html@g+w   akira.wml
 ../../../touch_translations.pl ??????ROOT??????webwml/english/women/profiles/akira.wml en
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:alba.en.html@g+w   alba.wml
@@ -13825,9 +13845,13 @@
 'others.mo' -> '../../locale/zh/LC_MESSAGES/others.mo'
 make[2]: Leaving directory '??????ROOT??????webwml/chinese/po'
 ln -sf totw.pl bin/tohk.pl
+touch template/debian-zh/template.tmpl
 Processing code_of_conduct.wml: [zh_CN], [zh_TW], [zh_HK].
 Processing contact.wml: [zh_CN], [zh_TW], [zh_HK].
-Processing donations.wml: [zh_CN], [zh_TW], [zh_HK].
+Processing donations.wml: [zh_CN]Invalid UTF8: ????????????????????????????????????????????????????????????????
+, [zh_TW]Invalid UTF8: ????????????????????????????????????????????????????????????????
+, [zh_HK]Invalid UTF8: ????????????????????????????????????????????????????????????????
+.
 Processing index.wml: wml -q -D CUR_YEAR=2019 -o UNDEFuZH@uCNuCNHKuCNTW:index.zh-cn.html.tmp@g+w -o UNDEFuZH@uHKuCNHKuHKTWuTWHK:index.zh-hk.html.tmp@g+w -o UNDEFuZH@uTWuCNTWuHKTWuTWHK:index.zh-tw.html.tmp@g+w --prolog=./bin/format_zh.pl --epilog=./bin/strip_initial_blank_line.sh \
            index.wml
 [zh_CN], [zh_TW], [zh_HK].
@@ -13835,7 +13859,10 @@
 Processing support.wml: [zh_CN], [zh_TW], [zh_HK].
 make -C Bugs
 make[2]: Entering directory '??????ROOT??????webwml/chinese/Bugs'
-Processing Reporting.wml: [zh_CN], [zh_TW], [zh_HK].
+Processing Reporting.wml: [zh_CN]Invalid UTF8: ???????????????????????????????</li>
+, [zh_TW]Invalid UTF8: ???????????????????????????????</li>
+, [zh_HK]Invalid UTF8: ???????????????????????????????</li>
+.
 Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
 Processing pseudo-packages.wml: [zh_CN], [zh_TW], [zh_HK].
 make[2]: Leaving directory '??????ROOT??????webwml/chinese/Bugs'
@@ -13854,7 +13881,8 @@
 make[3]: Leaving directory '??????ROOT??????webwml/chinese/CD/faq'
 make -C live
 make[3]: Entering directory '??????ROOT??????webwml/chinese/CD/live'
-Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
+Processing index.wml: [zh_CN]Malformed UTF-8 character: \xe9\x98\x0a (unexpected non-continuation byte 0x0a, 2 bytes after start byte 0xe9; need 3 bytes, got 2) in substitution (s///) at ../../bin/tocn.pl line 108, <> line 113.
+, [zh_TW], [zh_HK].
 make[3]: Leaving directory '??????ROOT??????webwml/chinese/CD/live'
 make -C mirroring
 make[3]: Entering directory '??????ROOT??????webwml/chinese/CD/mirroring'
@@ -13862,7 +13890,8 @@
 make[3]: Leaving directory '??????ROOT??????webwml/chinese/CD/mirroring'
 make -C netinst
 make[3]: Entering directory '??????ROOT??????webwml/chinese/CD/netinst'
-Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
+Processing index.wml: [zh_CN]Malformed UTF-8 character: \xe9\x98\x0a (unexpected non-continuation byte 0x0a, 2 bytes after start byte 0xe9; need 3 bytes, got 2) in substitution (s///) at ../../bin/tocn.pl line 108, <> line 102.
+, [zh_TW], [zh_HK].
 make[3]: Leaving directory '??????ROOT??????webwml/chinese/CD/netinst'
 make -C releases
 make[3]: Entering directory '??????ROOT??????webwml/chinese/CD/releases'
@@ -13888,7 +13917,10 @@
 make[3]: Entering directory '??????ROOT??????webwml/chinese/News/2000'
 Processing 20000521.wml: [zh_CN], [zh_TW], [zh_HK].
 Processing 20000523.wml: [zh_CN], [zh_TW], [zh_HK].
-Processing 20000815.wml: [zh_CN], [zh_TW], [zh_HK].
+Processing 20000815.wml: [zh_CN]Invalid UTF8: ??????????????????????????????????????
+, [zh_TW]Invalid UTF8: ??????????????????????????????????????
+, [zh_HK]Invalid UTF8: ??????????????????????????????????????
+.
 Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
 make[3]: Leaving directory '??????ROOT??????webwml/chinese/News/2000'
 make -C 2007
@@ -13897,7 +13929,10 @@
 make[3]: Leaving directory '??????ROOT??????webwml/chinese/News/2007'
 make -C 2009
 make[3]: Entering directory '??????ROOT??????webwml/chinese/News/2009'
-Processing 20090214.wml: [zh_CN], [zh_TW], [zh_HK].
+Processing 20090214.wml: [zh_CN]Invalid UTF8: ?Sun SPARC (sparc)???
+, [zh_TW]Invalid UTF8: ?Sun SPARC (sparc)???
+, [zh_HK]Invalid UTF8: ?Sun SPARC (sparc)???
+.
 Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
 make[3]: Leaving directory '??????ROOT??????webwml/chinese/News/2009'
 make -C 2011
@@ -13948,6 +13983,12 @@
 Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
 Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
 Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
+Processing index.wml: [zh_CN]Malformed UTF-8 character (unexpected end of string) in substitution (s///) at ../../bin/tocn.pl line 13, <> line 146.
+Malformed UTF-8 character (unexpected end of string) in substitution (s///) at ../../bin/tocn.pl line 15, <> line 146.
+panic: do_trans_simple_utf8 line 362 at ../../bin/tocn.pl line 20, <> line 146.
+, [zh_TW]Invalid UTF8: ????
+, [zh_HK]Invalid UTF8: ????
+.
 Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
 Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
 Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
@@ -13956,6 +13997,10 @@
 Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
 Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
 Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
+Processing index.wml: [zh_CN]Invalid UTF8: ???????????????????????
+, [zh_TW]Invalid UTF8: ???????????????????????
+, [zh_HK]Invalid UTF8: ???????????????????????
+.
 Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
 Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
 Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
@@ -13991,12 +14036,18 @@
 Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
 Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
 Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
+Processing index.wml: [zh_CN], [zh_TW]Invalid UTF8: ????????
+, [zh_HK]Invalid UTF8: ????????
+.
+Processing index.wml: [zh_CN]Invalid UTF8: ???????????
+, [zh_TW]Invalid UTF8: ???????????
+, [zh_HK]Invalid UTF8: ???????????
+.
 Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
-Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
-Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
-Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
-Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
-Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
+Processing index.wml: [zh_CN]Invalid UTF8: ????????<a href="../../../../events/talks">????????????</a>?????????
+, [zh_TW]Invalid UTF8: ????????<a href="../../../../events/talks">????????????</a>?????????
+, [zh_HK]Invalid UTF8: ????????<a href="../../../../events/talks">????????????</a>?????????
+.
 Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
 Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
 Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
@@ -14035,7 +14086,8 @@
 Processing ftplist.wml: [zh_CN], [zh_TW], [zh_HK].
 Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
 Processing netinst.wml: [zh_CN], [zh_TW], [zh_HK].
-Processing packages.wml: [zh_CN], [zh_TW], [zh_HK].
+Processing packages.wml: [zh_CN]Invalid UTF8: ???????
+, [zh_TW], [zh_HK].
 Processing pre-installed.wml: [zh_CN], [zh_TW], [zh_HK].
 make[2]: Leaving directory '??????ROOT??????webwml/chinese/distrib'
 make -C doc
@@ -14044,7 +14096,10 @@
 make[2]: Leaving directory '??????ROOT??????webwml/chinese/doc'
 make -C international
 make[2]: Entering directory '??????ROOT??????webwml/chinese/international'
-Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
+Processing index.wml: [zh_CN]Malformed UTF-8 character: \xe9\x98\x0a (unexpected non-continuation byte 0x0a, 2 bytes after start byte 0xe9; need 3 bytes, got 2) in substitution (s///) at ../bin/tocn.pl line 108, <> line 89.
+, [zh_TW]Invalid UTF8: ???????
+, [zh_HK]Invalid UTF8: ???????
+.
 make -C Chinese
 make[3]: Entering directory '??????ROOT??????webwml/chinese/international/Chinese'
 Processing about.wml: [zh_CN], [zh_TW], [zh_HK].
@@ -14054,7 +14109,10 @@
 Processing related_links.wml: [zh_CN], [zh_TW], [zh_HK].
 Processing software.wml: [zh_CN], [zh_TW], [zh_HK].
 Processing software_non-free.wml: [zh_CN], [zh_TW], [zh_HK].
-Processing thanks.wml: [zh_CN], [zh_TW], [zh_HK].
+Processing thanks.wml: [zh_CN]Invalid UTF8: ??????????????
+, [zh_TW]Invalid UTF8: ??????????????
+, [zh_HK]Invalid UTF8: ??????????????
+.
 make -C News
 make[4]: Entering directory '??????ROOT??????webwml/chinese/international/Chinese/News'
 Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
@@ -14093,7 +14151,9 @@
 Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
 Processing organization.wml: [zh_CN], [zh_TW], [zh_HK].
 Processing search.wml: [zh_CN], [zh_TW], [zh_HK].
-Processing why_debian.wml: [zh_CN], [zh_TW], [zh_HK].
+Processing why_debian.wml: [zh_CN], [zh_TW]Invalid UTF8: ??????????????
+, [zh_HK]Invalid UTF8: ??????????????
+.
 make[2]: Leaving directory '??????ROOT??????webwml/chinese/intro'
 make -C mirror
 make[2]: Entering directory '??????ROOT??????webwml/chinese/mirror'
@@ -14119,7 +14179,9 @@
 make -C releases
 make[2]: Entering directory '??????ROOT??????webwml/chinese/releases'
 Processing index.wml: [zh_CN], [zh_TW], [zh_HK].
-Processing proposed-updates.wml: [zh_CN], [zh_TW], [zh_HK].
+Processing proposed-updates.wml: [zh_CN], [zh_TW]Invalid UTF8: ????????????????? proposed-updates
+, [zh_HK]Invalid UTF8: ????????????????? proposed-updates
+.
 make -C buster
 make[3]: Entering directory '??????ROOT??????webwml/chinese/releases/buster'
 Processing credits.wml: [zh_CN], [zh_TW], [zh_HK].
@@ -14134,7 +14196,8 @@
 make[3]: Leaving directory '??????ROOT??????webwml/chinese/releases/etch'
 make -C hamm
 make[3]: Entering directory '??????ROOT??????webwml/chinese/releases/hamm'
-Processing HOWTO.upgrade.wml: [zh_CN], [zh_TW], [zh_HK].
+Processing HOWTO.upgrade.wml: [zh_CN], [zh_TW]Malformed UTF-8 character: \xe5\x8c\x0a (unexpected non-continuation byte 0x0a, 2 bytes after start byte 0xe5; need 3 bytes, got 2) in substitution (s///) at ../../bin/totw.pl line 111, <> line 71.
+, [zh_HK].
 Processing errata.wml: [zh_CN], [zh_TW], [zh_HK].
 make[3]: Leaving directory '??????ROOT??????webwml/chinese/releases/hamm'
 make -C jessie

Reply to: