Re: autodep8 test for C/C++ header
On Mon, 2023-08-07 at 22:52 +0300, Peter Pentchev wrote:
> On Mon, Aug 07, 2023 at 06:43:36PM +0000, Benjamin Drung wrote:
> > Hi,
> >
> > while working a whole week on fixing failing C/C++ header compilations
> > for armhf time_t [1], I noticed a common pattern: The library -dev
> > packages was missing one or more dependencies on another -dev package.
> > Over 200 -dev packages are affected.
> >
> > I propose to add an autodep8 test for C/C++ header files that tries to
> > compile the header file. This will catch issues like missing
> > dependencies and other issues.
> >
> > Here is a draft for the autopkgtest check script that takes a binary
> > -dev package as parameter:
> > https://github.com/bdrung/bdrung-scripts/blob/compile-header-check/compile-header-check
> >
> > What do you think? Should I proceed developing and packaging this check
> > script and submit support for it in autodep8?
> >
> > [1] https://salsa.debian.org/vorlon/armhf-time_t/-/merge_requests/103
>
> Whlie it does seem an interesting tool at first glance (and thanks for
> doing the work and presenting a proof of concept), I can think of
> several cases when compilation of the concatenated header files would
> fail:
Yes, I expect that several packages would fail to compile and the check
script would need several knobs to tweak (skip certain headers, add or
exclude certain include directories, define a specific C/C++ standand
version, use only C and not C++, etc). A set of common approaches to
solve such failures should be added to the documentation of the check
script (probably including skipping the whole check).
> 1) The library has a "main" header file that must be included before
> any of the others, and it does not come first in lexicographical
> order. It may define typedefs and structure definitions that
> the other header files can use, it may define preprocessor symbols
> that reflect the availability of this or that system header file or
> type; there are also other ways in which another header file
> distributed by the -dev package may depend on the main one.
In this case the non-"main" header could just include the "main" header
as first step. Alternatively, an option to specify headers that should
be included first could be added to the check script.
> 2) As a variation of the above, the -dev package may distribute
> the full set of header files included in the library, and some of
> them may only be included if certain preprocessor symbols are
> defined. Since their use is guarded by conditionals, they are
> allowed to unconditionally include system-specific header files
> that are only available on e.g. Windows or NetBSD or Darwin, etc.
> Unconditionally compiling the contents of those files would fail.
The -dev packages could drop shipping unneeded headers (e.g. Windows or
BSD specific ones), some headers could be excluded by a parameter to the
check script, and/or the check script could allow defining some names
(pass -D name=definition to gcc).
> 3) The -dev package may distribute the full set of header files
> included in the library, and some of them may be mutually exclusive
> and impossible to combine. For example, a header file may include
> either this or that other header file based on preprocessor
> defintions (OS type, features enabled, etc), and the included
> files may both define a function with the same name, but different
> contents.
That one is trickier to reflect in the check script.
> 4) Some of the library's header files may not be supposed to be
> included in all cases; the library's -dev package may suggest
> (but not depend on or recommend) another -dev package as
> an optional dependency, and a library's header file may
> unconditionally include some header file from the latter package.
Either exclude the header from the check script or not ship this header
file in the -dev package.
> All of these cases are things I've seen in complex libraries with
> many header files; maybe not all of them can be found in Debian
> right now.
When you look at check-armhf-time_t [2] you will find all those
mentioned cases and some more.
> So... yeah. Thanks for your work, I know you mean well and you are
> trying to make the life of Debian developers better, but this
> particular approach will likely fail on a non-trivial set of
> Debian -dev packages.
I expect this approach to work for most packages, fail for several
packages due to bugs in the package (missing dependencies and a bunch of
other issues like missing double include protection) and some will fail
with the reasons above.
According to [3] there are 5360 C/C++ -dev package and only 1900 -dev
packages (35%) did not compile out of the box [4]. The majority of those
1900 -dev packages do not compile due to bugs in the package (judging
from the workaround that we apply in [2]).
IMO having this check for 90% of the packages to catch bugs and
disabling it for 10% is better than not having it. If there were such
check script in place before the armhf time_t analysis started, we would
have had significantly fewer packages to analyze.
[2] https://salsa.debian.org/vorlon/armhf-time_t/-/blob/main/check-armhf-time_t
[3] https://wiki.debian.org/ReleaseGoals/64bit-time
[4] https://lists.debian.org/debian-devel/2023/05/msg00168.html
--
Benjamin Drung
Debian & Ubuntu Developer
Reply to: