On Wed, Dec 12, 2012 at 11:49:01PM +0200, Niko Tyni wrote: > so this failure mode seems to be specific to s390* unless I goofed > up some way. Yes, it is. > This isn't exhaustive as some of the failures (like shogun, mentioned > in this bug) were not given back before a new source upload superseded > them. I can't see any way to determine all failed builds on > buildd.debian.org without reading all the logs in; the best thing I > found is /srv/buildd.debian.org/db/log which I'm using as an index. I did start a grep on all s390x build logs yesterday. And it did complete over night. > FWIW I can't see any clear correlation between the buildd host and the > failure rate so a hardware problem seems to be out of the picture. > > Cc'ing the s390 list in case somebody has clever ideas. This is > unreproducible so far; if somebody can reproduce it reliably, please > take a snapshot of the whole system or something before it goes away. abort()s don't leave coredumps as far as I know and those problems are unreproducible as a given back build usually works. Normally these are one-offs. The build log grep seems to confirm that. So we are talking about these: ./g/gimp/2.8.2-2/s390x_1353706105_log.bz2:*** glibc detected *** /usr/bin/perl: malloc(): memory corruption: 0x00000200004480b8 *** ./l/linux/3.2.30-1/s390x_1348704958_log.bz2:*** glibc detected *** /bin/sh: malloc(): memory corruption: 0x00000000800288e0 *** ./s/sofa-framework/1.0~beta4-6.1/s390x_1338312919_log.bz2:*** glibc detected *** /usr/bin/perl: malloc(): memory corruption (fast): 0x0000000083aebdd0 *** ./p/pdl/1:2.4.11-3/s390x_1338241813_log.bz2:*** glibc detected *** /usr/bin/perl: malloc(): memory corruption: 0x0000000080c28b60 *** ./p/pdl/1:2.4.11-3/s390x_1338241813_log.bz2:*** glibc detected *** /usr/bin/perl: malloc(): memory corruption: 0x0000000080c28dc0 *** ./p/pdl/1:2.4.11-4/s390x_1338393181_log.bz2:*** glibc detected *** /usr/bin/perl: malloc(): memory corruption: 0x0000000080c28b60 *** ./p/pdl/1:2.4.11-4/s390x_1338393181_log.bz2:*** glibc detected *** /usr/bin/perl: malloc(): memory corruption: 0x0000000080c28dc0 *** ./p/pdl/1:2.4.11-1/s390x_1338082289_log.bz2:*** glibc detected *** /usr/bin/perl: malloc(): memory corruption: 0x0000000080c28b40 *** ./p/pdl/1:2.4.11-1/s390x_1338082289_log.bz2:*** glibc detected *** /usr/bin/perl: malloc(): memory corruption: 0x0000000080c28d80 *** ./p/pdl/1:2.4.11-2/s390x_1338139001_log.bz2:*** glibc detected *** /usr/bin/perl: malloc(): memory corruption: 0x0000000080c28b40 *** ./p/pdl/1:2.4.11-2/s390x_1338139001_log.bz2:*** glibc detected *** /usr/bin/perl: malloc(): memory corruption: 0x0000000080c28d80 *** ./p/pdl/1:2.4.10+dfsg-1/s390x_1333684026_log.bz2:*** glibc detected *** /usr/bin/perl: malloc(): memory corruption: 0x0000000080c154f0 *** ./p/pdl/1:2.4.10+dfsg-1/s390x_1333684026_log.bz2:*** glibc detected *** /usr/bin/perl: malloc(): memory corruption: 0x0000000080c15670 *** ./h/haskell-attempt/0.3.1.1-1+b2/s390x_1328292955_log.bz2:*** glibc detected *** /usr/bin/perl: malloc(): memory corruption: 0x00000200001f7838 *** Indeed I saw it mostly with perl but it's not confined to it. It doesn't make me happy but it happens quite rarely. Maybe one would have the best chance with pdl to debug that. 0x80c28d80 is on the heap in any case (i.e. not an anonymous mmap), my assumption is still that something in the allocator or the kernel's fishy. In any case feel free to give back builds failing with this reason. That's what I did so far. Kind regards Philipp Kern
Attachment:
signature.asc
Description: Digital signature