On Tue, Jun 07, 2011 at 11:14:14AM +0200, Tollef Fog Heen wrote:
> ]] Steve Langasek
>
> Hi,
>
> | 4) Turn on direct use of 'debian/rules build-arch' on the autobuilders for
> | all packages in unstable and experimental immediately, with no fallback
> | if the target does not exist; requires a corresponding update to Policy
> | and mass updates to fix packages that fail to build as a result.
>
> I'd be happy to provide hardware to do a full scale rebuild test of the
> archive if somebody does the actual work of doing the rebuilds. rleigh
> did it for his sbuild resolver test so I've Cc-ed him to see if he's
> interested in doing a test for this too.
This has been completed now, and the summarised results follow.
All of the build logs, scripts and datafiles are available from
http://people.debian.org/~rleigh/dpkg-rebuild/
Summarised:
- autodetection with "make -qn" breaks few, if any packages.
- unconditional use of build-arch breaks approximately 45% of
arch-any packages (the fraction not providing a build-arch
target).
Note that the builds were done in the order
unpatched (8/6/11) - stock git commit dc927a01
autodetect (9/6/11) - dc927a01 + make -qn patch
unconditional (10/6/11) - dc927a01 with build-features fallback
removed
Because unstable was changing between the rebuilds, some of the
failures are likely due to churn, including multiarch work, so a
failure does not necessarily implicate the patch being tested.
Build logs were parsed to extract
- build status (success/failure)
- fail stage (if failed)
- if autodetection found build-arch to be missing in debian/rules
("found", sorry that's counterintutive)
- if build-arch was used and the build failed due to its absence
("used" when missing)
The data were put into a PostgreSQL database and queried to give
the results below.
\d builds
Table "public.builds"
┌───────────────┬────────────┬──────────────────────────────────┐
│ Column │ Type │ Modifiers │
├───────────────┼────────────┼──────────────────────────────────┤
│ strategy │ text │ not null │
│ package │ text │ not null │
│ version │ debversion │ not null │
│ status │ text │ not null default 'invalid'::text │
│ failstage │ text │ │
│ found_missing │ boolean │ │
│ used_missing │ boolean │ │
└───────────────┴────────────┴──────────────────────────────────┘
Indexes:
"builds_strategy_package_version_key" UNIQUE, btree (strategy, package, version)
SELECT b.strategy, b.status, b.failstage, b.found_missing AS found,
b.used_missing AS used, COUNT(*)
FROM builds AS b
GROUP by b.strategy,b.status,b.failstage,b.found_missing,b.used_missing
ORDER BY strategy,status,failstage,b.found_missing,b.used_missing;
found=build-arch target autodetected t=missing, false=present
used=build-arch target used when missing (f is OK, t is failure)
┌───────────────┬────────────┬────────────────┬───────┬──────┬───────┐
│ strategy │ status │ failstage │ found │ used │ count │
├───────────────┼────────────┼────────────────┼───────┼──────┼───────┤
│ autodetect │ attempted │ abort │ f │ f │ 1 │
│ autodetect │ attempted │ build │ f │ f │ 497 │
│ autodetect │ attempted │ build │ t │ f │ 357 │
│ autodetect │ failed │ apt-get-update │ │ f │ 3 │
│ autodetect │ failed │ fetch-src │ │ f │ 160 │
│ autodetect │ failed │ init │ │ f │ 2 │
│ autodetect │ failed │ install-deps │ │ f │ 162 │
│ autodetect │ failed │ unpack │ │ f │ 3 │
│ autodetect │ skipped │ arch-check │ │ f │ 6962 │
│ autodetect │ successful │ │ f │ f │ 4675 │⁵
│ autodetect │ successful │ │ t │ f │ 3877 │⁴
├───────────────┼────────────┼────────────────┼───────┼──────┼───────┤
│ unconditional │ attempted │ abort │ │ f │ 2 │
│ unconditional │ attempted │ build │ │ f │ 500 │
│ unconditional │ attempted │ build │ │ t │ 4173 │³
│ unconditional │ failed │ fetch-src │ │ f │ 178 │
│ unconditional │ failed │ install-deps │ │ f │ 163 │
│ unconditional │ failed │ unpack │ │ f │ 3 │
│ unconditional │ skipped │ arch-check │ │ f │ 6963 │
│ unconditional │ successful │ │ │ f │ 4717 │²
├───────────────┼────────────┼────────────────┼───────┼──────┼───────┤
│ unpatched │ attempted │ build │ │ f │ 743 │
│ unpatched │ failed │ fetch-src │ │ f │ 89 │
│ unpatched │ failed │ init │ │ f │ 2 │
│ unpatched │ failed │ install-deps │ │ f │ 161 │
│ unpatched │ failed │ unpack │ │ f │ 3 │
│ unpatched │ skipped │ arch-check │ │ f │ 6993 │
│ unpatched │ successful │ │ │ f │ 8708 │¹
└───────────────┴────────────┴────────────────┴───────┴──────┴───────┘
(26 rows)
¹Without the patch, all builds use the build target
²Always using the build-arch target, only packages with build-arch
(or dh/cdbs users) build successfully
³Failed builds are mainly due to lack of a build-arch target
⁴Autodetected presence of build-arch and built using build-arch
⁵Failed autodetection of build-arch; build used instead
From this you can see some clear trends, as summarised at the top.
To remove the noise, I selected only builds which were successful
in the unpatched version, and which didn't suffer from arbitrary
failure in the patched version (failed to download source etc.), i.e.
not build failures (attempted) but (failed).
CREATE VIEW orig AS SELECT package,version,status,failstage,found_missing,used_missing FROM builds WHERE (strategy='unpatched');
CREATE VIEW auto AS SELECT package,version,status,failstage,found_missing,used_missing FROM builds WHERE (strategy='autodetect');
CREATE VIEW break AS SELECT package,version,status,failstage,found_missing,used_missing FROM builds WHERE (strategy='unconditional');
SELECT a.status AS ad_status, a.found_missing AS ad_missing,
a.used_missing as ad_used,
b.status AS uc_status, b.used_missing AS uc_used, COUNT(*)
FROM orig AS o
INNER JOIN auto AS a
ON o.package=a.package AND o.version=a.version
INNER JOIN break AS b
ON o.package=b.package AND o.version=b.version
WHERE
o.status='successful'
AND (a.status='successful'
OR (a.status='attempted' AND a.failstage='build'))
AND (b.status ='successful'
OR (b.status='attempted' AND b.failstage='build'))
GROUP BY ad_status, ad_missing, ad_used, uc_status, uc_used
ORDER BY ad_status, uc_status, ad_missing, ad_used, uc_used;
ad=autodetect, uc=unconditional
┌────────────┬────────────┬─────────┬────────────┬─────────┬───────┐
│ ad_status │ ad_missing │ ad_used │ uc_status │ uc_used │ count │
├────────────┼────────────┼─────────┼────────────┼─────────┼───────┤
│ attempted │ f │ f │ attempted │ f │ 109 │¹
│ attempted │ t │ f │ attempted │ t │ 19 │¹
├────────────┼────────────┼─────────┼────────────┼─────────┼───────┤
│ successful │ f │ f │ attempted │ f │ 3 │¹
│ successful │ t │ f │ attempted │ f │ 2 │¹
│ successful │ t │ f │ attempted │ t │ 3818 │²
├────────────┼────────────┼─────────┼────────────┼─────────┼───────┤
│ successful │ f │ f │ successful │ f │ 4662 │³
│ successful │ t │ f │ successful │ f │ 40 │⁴
└────────────┴────────────┴─────────┴────────────┴─────────┴───────┘
(7 rows)
¹Outlying failures. Probably due to changes in unstable.
²Successful only with autodetection; unconditional tries to build
using nonexistent build-arch target while autodetection falls back
to build target.
³Autodetection found a build-arch rule and built using build-arch;
unconditional used build-arch.
⁴Possible failure of autodetection since autodetection failed to find
build-arch and fell back to build. However, unconditional still
succeeded here.
Hope this is useful. If you want any additional work doing, or any
further reanalysis or more detail about specific cases above, please
let me know. If you want to look at the data yourself, the data is
in CSV and PostgreSQL dumps on p.d.o. at the URI above.
Regards,
Roger
--
.''`. Roger Leigh
: :' : Debian GNU/Linux http://people.debian.org/~rleigh/
`. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/
`- GPG Public Key: 0x25BFB848 Please GPG sign your mail.
Attachment:
signature.asc
Description: Digital signature