Re: catgets 'make check' failure
On Tue, Oct 01, 2002 at 07:41:21PM -0700, Jeff Bailey wrote:
> On Tue, Oct 01, 2002 at 07:30:41PM -0400, Ben Collins wrote:
>
> > > > annexc is an expected failure. Check the output sometime.
>
> > > I have, but I haven't looked at what the test is supposed to do
> > > yet. =)
>
> > > I want to either fix the test or patch it so it doesn't run.
>
> > Don't keep it from running. Leave it. It's been there for years and
> > hurts nothing. There's no reason to get rid of it now.
>
> One of my goals is to remove the '-k' from the make check line. I
> want the build to fail if there are bad tests - that way regressions
> are caught and dealt with very quickly on arch's that the glibc
> maintainers aren't using.
You can't remove the -k. If you do, it will stop on the first test
failure, which means you wont see trends, or the overall picture.
Besides the -k does not make it ignore all the failures, the '-' in
front of the check target makes it not stop on errors.
Not only that but it is quite frequent that new architectures don't want
to fail for test suite failures, especially known failures. They accept
a certain amount of problems while developing the architecture.
The only way to catch regressions is for people to manually check the
test results, as should be done with glibc, gcc and binutils (i.e. the
toolchain). You can't get by with automating this to the point where you
don't check the testsuite results manually before an upload.
> > Just read the output, you'll see what it is for. Basically it checks
> > certain headers to see if they define macros that they should and
> > also don't define macros that they shouldn't.
>
> > Do not remove the test.
>
> You haven't explained why not. A test that fails and is ignored is
> useless and misleading.
It's there for the glibc maintainers. It's not in our best interest to
remove any of the tests, regardless of the need to make everything
appear neat and tidy.
--
Debian - http://www.debian.org/
Linux 1394 - http://www.linux1394.org/
Subversion - http://subversion.tigris.org/
Deqo - http://www.deqo.com/
Reply to: