Bug#915613: libksysguard: autopkgtest intermittently fails: testCPUGraphHistory() 'percentageHist.size() > 0' returned FALSE

[Simon McVittie <smcv@debian.org>]

Source: libksysguard
Version: 4:5.14.3-1
Severity: important
X-Debbugs-CC: debian-ci@lists.debian.org
User: debian-ci@lists.debian.org
Usertags: flaky

The autopkgtest for libksysguard appears to be unreliable, with intermittent
failures. The testing migration infrastructure can't distinguish between an
unreliable test failing due to bad luck and a genuine regression caused by
an updated package, so this is likely to stall packages' migration to
testing (it's currently affecting dbus). For now I've retried the most
recent failed test.

For comparison, a successful test run looks like this:

> https://ci.debian.net/data/autopkgtest/unstable/amd64/libk/libksysguard/1424303/log.gz
> /usr/bin/ctest --force-new-ctest-process -j2
> Test project /tmp/autopkgtest-lxc.f73y3vo9/downtmp/build.UaB/src/obj-x86_64-linux-gnu
>     Start 1: processtest
>     Start 2: signalplotterbenchmark
> dbus-daemon[15640]: [session uid=1000 pid=15640] Activating service name='org.kde.kglobalaccel' requested by ':1.1' (uid=1000 pid=15657 comm="/tmp/autopkgtest-lxc.f73y3vo9/downtmp/build.UaB/sr")
> dbus-daemon[15640]: [session uid=1000 pid=15640] Successfully activated service 'org.kde.kglobalaccel'
> 1/5 Test #2: signalplotterbenchmark ...........   Passed    1.65 sec
>     Start 3: graphicssignalplotterbenchmark
> 2/5 Test #1: processtest ......................   Passed    1.74 sec
>     Start 4: signalplottertest
> 3/5 Test #4: signalplottertest ................   Passed    0.05 sec
>     Start 5: chronotest
> 4/5 Test #5: chronotest .......................   Passed    0.00 sec
> 5/5 Test #3: graphicssignalplotterbenchmark ...   Passed    1.12 sec
> 
> 100% tests passed, 0 tests failed out of 5

An example of an unsuccessful test run:

https://ci.debian.net/data/autopkgtest/unstable/amd64/libk/libksysguard/1428569/log.gz
> /usr/bin/ctest --force-new-ctest-process -j2
> Test project /tmp/autopkgtest-lxc.2643_tf7/downtmp/build.h17/src/obj-x86_64-linux-gnu
>     Start 1: processtest
>     Start 2: signalplotterbenchmark
> dbus-daemon[15668]: [session uid=1000 pid=15668] Activating service name='org.kde.kglobalaccel' requested by ':1.1' (uid=1000 pid=15685 comm="/tmp/autopkgtest-lxc.2643_tf7/downtmp/build.h17/sr")
> dbus-daemon[15668]: [session uid=1000 pid=15668] Successfully activated service 'org.kde.kglobalaccel'
> 1/5 Test #2: signalplotterbenchmark ...........   Passed    1.68 sec
>     Start 3: graphicssignalplotterbenchmark
> 2/5 Test #1: processtest ......................***Failed    1.72 sec
> ********* Start testing of testProcess *********
> Config: Using QtTest library 5.11.2, Qt 5.11.2 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 8.2.0)
> PASS   : testProcess::initTestCase()
> PASS   : testProcess::testTimeToUpdateAllProcesses()
> RESULT : testProcess::testTimeToUpdateAllProcesses():
>      3.5 msecs per iteration (total: 56, iterations: 16)
> QWARN  : testProcess::testTimeToUpdateModel() QDBusConnection: name 'org.kde.kglobalaccel' had owner '' but we thought it was ':1.2'
> PASS   : testProcess::testTimeToUpdateModel()
> RESULT : testProcess::testTimeToUpdateModel():
>      9.6 msecs per iteration (total: 77, iterations: 8)
> PASS   : testProcess::testProcesses()
> PASS   : testProcess::testProcessesTreeStructure()
> PASS   : testProcess::testProcessesModification()
> QWARN  : testProcess::testHistories() History was not available
> QWARN  : testProcess::testHistories() History was not available
> PASS   : testProcess::testHistories()
> RESULT : testProcess::testHistories():
>      0 msecs per iteration (total: 0, iterations: 1)
> PASS   : testProcess::testHistoriesWithWidget()
> PASS   : testProcess::testUpdateOrAddProcess()
> FAIL!  : testProcess::testCPUGraphHistory() 'percentageHist.size() > 0' returned FALSE. ()
>    Loc: [/tmp/autopkgtest-lxc.2643_tf7/downtmp/build.h17/src/tests/processtest.cpp(238)]
> PASS   : testProcess::cleanupTestCase()
> Totals: 10 passed, 1 failed, 0 skipped, 0 blacklisted, 1656ms

I don't know whether this is correlated with the warning "QDBusConnection:
name 'org.kde.kglobalaccel' had owner '' but we thought it was ':1.2'",
which might indicate a race condition in QtDBus - CTest suppresses the
test's detailed output for successful test cases. Can CTest be configured
to not do that, or can the tests be made to output successful test logs
afterwards?

If this test cannot be made reliable, please mark it as flaky
("Restrictions: flaky" in d/tests/control) so that failure is not treated
as serious.

Thanks,
    smcv