Re: Please give back prometheus on armel
On 24/11/16 05:37, Martín Ferrari wrote:
> On 23/11/16 19:13, Emilio Pozuelo Monfort wrote:
>
>>> prometheus 1.2.3+ds2-2 failed to build on armel, but I am not being able
>>> to reproduce this failure in porter boxes. Can you please trigger a
>>> rebuild to see if it was a transient problem?
>
>> Given back.
>
> Thanks!
>
> Now I have a problem, that maybe somebody can help with... The build
> failed again on armel[0], but I still can't reproduce this.
>
> I have just tried on abel. I created a sid chroot, and rebuild from the
> source present in the archive. I checked all the build-dependencies, and
> version numbers match exactly.
>
> But in abel the tests run just fine. Is there any significant difference
> with the armel buildds? I have the same failure both in henze and antheli.
Hardware wise, they look similar:
https://db.debian.org/machines.cgi?host=abel
https://db.debian.org/machines.cgi?host=henze
https://db.debian.org/machines.cgi?host=antheil
henze is an armhf buildd that also builds armel, but antheil is an armel buildd,
like abel, so that seems fine.
Your failing test seems to be:
=== RUN TestDropMetrics
time="2016-11-18T02:42:54Z" level=info msg="Loading series map and head
chunks..." source="storage.go:354"
time="2016-11-18T02:42:54Z" level=info msg="0 series loaded."
source="storage.go:359"
time="2016-11-18T02:43:04Z" level=info msg="Maintenance loop stopped."
source="storage.go:1224"
panic: runtime error: index out of range
goroutine 655 [running]:
panic(0x423540, 0x10928028)
/usr/lib/go-1.7/src/runtime/panic.go:500 +0x33c
github.com/prometheus/prometheus/storage/local.(*memorySeries).maybeCloseHeadChunk(0x11b7a050,
0x618714)
/«BUILDDIR»/prometheus-1.2.3+ds2/build/src/github.com/prometheus/prometheus/storage/local/series.go:280 +0x290
github.com/prometheus/prometheus/storage/local.(*MemorySeriesStorage).maintainMemorySeries(0x10998600,
0x94b33977, 0x2ee1b8f2, 0x341bb166, 0xfffffe7a, 0xfffffd00)
/«BUILDDIR»/prometheus-1.2.3+ds2/build/src/github.com/prometheus/prometheus/storage/local/storage.go:1327 +0x1d4
github.com/prometheus/prometheus/storage/local.(*MemorySeriesStorage).loop(0x10998600)
/«BUILDDIR»/prometheus-1.2.3+ds2/build/src/github.com/prometheus/prometheus/storage/local/storage.go:1252 +0x400
created by
github.com/prometheus/prometheus/storage/local.(*MemorySeriesStorage).Start
/«BUILDDIR»/prometheus-1.2.3+ds2/build/src/github.com/prometheus/prometheus/storage/local/storage.go:370 +0x3e8
exit status 2
FAIL github.com/prometheus/prometheus/storage/local 12.879s
However a recent build succeeded on henze[1]:
=== RUN TestDropMetrics
time="2016-11-09T22:21:33Z" level=info msg="Loading series map and head
chunks..." source="storage.go:354"
time="2016-11-09T22:21:33Z" level=info msg="0 series loaded."
source="storage.go:359"
time="2016-11-09T22:21:43Z" level=info msg="Checkpointing in-memory metrics and
chunks..." source="persistence.go:549"
time="2016-11-09T22:21:43Z" level=info msg="Done checkpointing in-memory metrics
and chunks in 894.349µs." source="persistence.go:573"
time="2016-11-09T22:21:43Z" level=info msg="Stopping local storage..."
source="storage.go:377"
time="2016-11-09T22:21:43Z" level=info msg="Stopping maintenance loop..."
source="storage.go:379"
time="2016-11-09T22:21:43Z" level=info msg="Completed maintenance sweep through
2 in-memory fingerprints in 825.475555ms." source="storage.go:1167"
time="2016-11-09T22:21:43Z" level=info msg="Maintenance loop stopped."
source="storage.go:1224"
time="2016-11-09T22:21:43Z" level=info msg="Stopping series quarantining..."
source="storage.go:383"
time="2016-11-09T22:21:43Z" level=info msg="Series quarantining stopped."
source="storage.go:1635"
time="2016-11-09T22:21:43Z" level=info msg="Stopping chunk eviction..."
source="storage.go:387"
time="2016-11-09T22:21:43Z" level=info msg="Chunk eviction stopped."
source="storage.go:1044"
time="2016-11-09T22:21:43Z" level=info msg="Checkpointing in-memory metrics and
chunks..." source="persistence.go:549"
time="2016-11-09T22:21:43Z" level=info msg="Done checkpointing in-memory metrics
and chunks in 296.116µs." source="persistence.go:573"
time="2016-11-09T22:21:43Z" level=info msg="Checkpointing fingerprint
mappings..." source="persistence.go:1372"
time="2016-11-09T22:21:43Z" level=info msg="Done checkpointing fingerprint
mappings in 195.958µs." source="persistence.go:1395"
time="2016-11-09T22:21:43Z" level=info msg="Local storage stopped."
source="storage.go:402"
--- PASS: TestDropMetrics (10.87s)
Those first few debug messages aren't the same in both runs. That "Maintenance
loop stopped" happens earlier in the failing run. No idea if that's significant
as I haven't looked at the code, but you may want to investigate that. Also
since this is failing some times but has also passed, I would try to run this in
a loop for a while in the porterbox. Might be a race condition.
> Thanks again. And if this is not the correct place to ask, please
> redirect me accordingly.
Cc'ing debian-arm@ and armel@, in case they have any other comments.
Cheers,
Emilio
[1] https://buildd.debian.org/status/logs.php?pkg=prometheus&arch=armel
Reply to: