unity8-dash crashes during power tests

[Selene ToyKeeper]

Unity8-dash crashes fairly often while running power usage tests. I hope someone will be able to determine the root cause based on crash dumps and other logs.

Here are some of the 170 instances I've seen so far... inside each are numbered directories for each test run, and the crash dumps are inside of those under var-crash/ .

Arale rc-proposed 61, runs 2, 3, and 4:
http://people.canonical.com/~platform-qa/power-results/2015-07-16_18:21:18-arale-61-power_usage_display_on/

Arale devel-proposed 61, runs 2, 5, and 8:
http://people.canonical.com/~platform-qa/power-results/2015-07-16_21:58:20-arale-61-power_usage_idle/

Arale devel-proposed 60, runs 1, 2, 6, and 7:
http://people.canonical.com/~platform-qa/power-results/2015-07-16_15:26:03-arale-60-power_usage_flight_mode_on/

Krillin rc-proposed 62, runs 1, 3, 4, 5, 6, 7, and 8:
http://people.canonical.com/~platform-qa/power-results/2015-07-13_02:02:39-krillin-62-power_usage_idle/

Krillin devel-proposed 132, runs 1, 3, 6, and 7:
http://people.canonical.com/~platform-qa/power-results/2015-07-13_22:02:04-krillin-132-power_usage_display_on/

These are merely samples from recent test runs. I have plenty more if you need them, or can add additional logging to get more data from future tests.

[Albert Astals Cid (aacid)]

So they all have the same stacktrace

#0 0xb61659a6 in ?? () from /lib/arm-linux-gnueabihf/libc.so.6
 No symbol table info available.
 #1 0xb617362e in raise () from /lib/arm-linux-gnueabihf/libc.so.6
 No symbol table info available.
 #2 0xb6174332 in abort () from /lib/arm-linux-gnueabihf/libc.so.6
 No symbol table info available.
 #3 0xb638ecdc in QMessageLogger::fatal(char const*, ...) const () from /usr/lib/arm-linux-gnueabihf/libQt5Core.so.5
 No symbol table info available.
 #4 0xb3ed8a34 in ?? () from /usr/lib/arm-linux-gnueabihf/qt5/plugins/platforms/libqpa-ubuntumirclient.so
 No symbol table info available.
 #5 0xb3edb2a4 in ?? () from /usr/lib/arm-linux-gnueabihf/qt5/plugins/platforms/libqpa-ubuntumirclient.so
 No symbol table info available.
 #6 0xb6d0e95e in QPlatformIntegrationFactory::create(QString const&, QStringList const&, int&, char**, QString const&) () from /usr/lib/arm-linux-gnueabihf/libQt5Gui.so.5
 No symbol table info available.
 #7 0xb6d16878 in QGuiApplicationPrivate::createPlatformIntegration() () from /usr/lib/arm-linux-gnueabihf/libQt5Gui.so.5
 No symbol table info available.
 #8 0xb6d17096 in QGuiApplicationPrivate::createEventDispatcher() () from /usr/lib/arm-linux-gnueabihf/libQt5Gui.so.5
 No symbol table info available.
 #9 0xb6501540 in QCoreApplication::init() () from /usr/lib/arm-linux-gnueabihf/libQt5Core.so.5
 No symbol table info available.
 #10 0xb65015ca in QCoreApplication::QCoreApplication(QCoreApplicationPrivate&) () from /usr/lib/arm-linux-gnueabihf/libQt5Core.so.5
 No symbol table info available.
 #11 0xb6d183fa in QGuiApplication::QGuiApplication(int&, char**, int) () from /usr/lib/arm-linux-gnueabihf/libQt5Gui.so.5
 No symbol table info available.
 #12 0x000134f2 in ?? ()
 No symbol table info available.
 #13 0xb6165776 in __libc_start_main () from /lib/arm-linux-gnueabihf/libc.so.6
 No symbol table info available.
 #14 0x0001401c in _start ()

Which as far as i understand means that it could not find a Mir server and is aborting.

That makes sense for the cases where unity8 had also crashed, but for ones like
http://people.canonical.com/~platform-qa/power-results/2015-07-13_02:02:39-krillin-62-power_usage_idle/1/var-crash/
where unity8 did not crash it may indeed be a real bug.

Maybe we're starting unity8-dash too early?

[Gerry Boland (gerboland)]

This does suggest unity8-dash is starting but either
1. there is no mir server, such as unity8, running
2. MIR_SOCKET is not set to the correct mir server socket
3. the permissions on the socket file mean dash cannot connect

We rely on upstart to ensure Dash starts when unity8 claims to be ready to accept connections.
Upstart starts unity8, and puts it in the "starting" state. When unity8 is ready to accept client connections, it emits a SIGSTOP. Upstart intercepts this and SIGCONTs unity8. It then can start anything which requires the unity8 job to be "running" - such as unity8-dash.

It's also upstart's job to set the MIR_SOCKET env var, and other env vars required. And to ensure permissions are correct.

So something in the above has gone wrong.

I am aware of unity8 not always shutting down correctly, which might leave cruft behind. But I wold have thought upstart was cleaning that cruft up, and only launching unity8-dash after a working unity8 had started.

[Sebastien Bacher (seb128)]

we could perhaps add an apport hook to collect extra info if needed (socket permissions for example)