MySql log file problem prevents Akonadi startup

Bug #578357 reported by Davor Cubranic on 2010-05-10
48
This bug affects 10 people
Affects Status Importance Assigned to Milestone
Akonadi
Incomplete
Medium
akonadi (Ubuntu)
Undecided
Unassigned

Bug Description

I upgraded an older PC from KUbuntu Karmic to Lucid by doing a clean install to the root partition, and keeping existing files in the separate '/home' partition untouched. I now cannot use KMail any more, because Akonadi fails to start up. This is the error output that I get when I tried running 'akonadictl start' from the console:

I tried running 'akonadictl start' from the command line, and it fails to start -- apparently because I haven't installed MySql. Console output is below:
Connecting to deprecated signal QDBusConnectionInterface::serviceOwnerChanged(QString,QString,QString)
 [akonadiserver] search paths: ("/home/cubranic/bin", "/home/cubranic/bin", "/usr/local/sbin", "/usr/local/bin", "/usr/sbin", "/usr/bin", "/sbin", "/bin", "/usr/games", "/usr/sbin", "/usr/local/sbin", "/usr/local/libexec", "/usr/libexec", "/opt/mysql/libexec", "/opt/local/lib/mysql5/bin")
 [akonadiserver] Found mysql_install_db: "/usr/bin/mysql_install_db"
 [akonadiserver] Found mysql_upgrade: "/usr/bin/mysql_upgrade"
 "akonadiserver" [out] "Looking for 'mysql' as: /usr/bin/mysql
 "
 "akonadiserver" [out] "Looking for 'mysqlcheck' as: /usr/bin/mysqlcheck
 "
 "akonadiserver" [out] "Running 'mysqlcheck' with connection arguments: '--port=3306' '--socket=/var/run/mysqld/mysqld.sock' '--socket=/home/cubranic/.local/share/akonadi/db_misc/mysql.socket'
 "
 [akonadiserver] /usr/bin/mysqlcheck: Got error: 2002: Can't connect to local MySQL server through socket '/home/cubranic/.local/share/akonadi/db_misc/mysql.socket' (2) when trying to connect
 [akonadiserver] FATAL ERROR: Upgrade failed
 [akonadiserver] Database process exited unexpectedly during initial connection!
 [akonadiserver] executable: "/usr/sbin/mysqld-akonadi"
 [akonadiserver] arguments: ("--defaults-file=/home/cubranic/.local/share/akonadi//mysql.conf", "--datadir=/home/cubranic/.local/share/akonadi/db_data/", "--socket=/home/cubranic/.local/share/akonadi/db_misc/mysql.socket")
 [akonadiserver] stdout: ""
 [akonadiserver] stderr: ""
 [akonadiserver] exit code: 1
 [akonadiserver] process error: "Unknown error"
 [akonadiserver] "[
 [akonadiserver] 0: akonadiserver(_Z11akBacktracev+0x35) [0x8052265]
 [akonadiserver] 1: akonadiserver() [0x8052726]
 [akonadiserver] 2: [0xb5d400]
 [akonadiserver] 3: [0xb5d422]
 [akonadiserver] 4: /lib/tls/i686/cmov/libc.so.6(gsignal+0x51) [0x4d1651]
 [akonadiserver] 5: /lib/tls/i686/cmov/libc.so.6(abort+0x182) [0x4d4a82]
 [akonadiserver] 6: /usr/lib/libQtCore.so.4(_Z17qt_message_output9QtMsgTypePKc+0x8c) [0x16d2dc]
 [akonadiserver] 7: akonadiserver(_ZN15FileDebugStream9writeDataEPKcx+0xc4) [0x80537a4]
 [akonadiserver] 8: /usr/lib/libQtCore.so.4(_ZN9QIODevice5writeEPKcx+0x8e) [0x20546e]
 [akonadiserver] 9: /usr/lib/libQtCore.so.4(+0x10a275) [0x21a275]
 [akonadiserver] 10: /usr/lib/libQtCore.so.4(_ZN11QTextStreamD1Ev+0x3d) [0x21b78d]
 [akonadiserver] 11: akonadiserver(_ZN6QDebugD1Ev+0x43) [0x804dea3]
 [akonadiserver] 12: /usr/lib/libakonadiprivate.so.1(_ZN7Akonadi13AkonadiServer25startMysqlDatabaseProcessEv+0x2071) [0xe50cb1]
 [akonadiserver] 13: /usr/lib/libakonadiprivate.so.1(_ZN7Akonadi13AkonadiServer20startDatabaseProcessEv+0x263) [0xe53da3]
 [akonadiserver] 14: /usr/lib/libakonadiprivate.so.1(_ZN7Akonadi13AkonadiServerC1EP7QObject+0x7d) [0xe5403d]
 [akonadiserver] 15: /usr/lib/libakonadiprivate.so.1(_ZN7Akonadi13AkonadiServer8instanceEv+0x58) [0xe554f8]
 [akonadiserver] 16: akonadiserver(main+0x374) [0x804d204]
 [akonadiserver] 17: /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6) [0x4bdbd6]
 [akonadiserver] 18: akonadiserver() [0x804cdc1]
 [akonadiserver] ]
 [akonadiserver] "
 ProcessControl: Application 'akonadiserver' returned with exit code 255 (Unknown error)
[repeated three more times]

I had the same problem, and it was fixed running:

sudo aa-complain mysqld
sudo aa-complain mysqld-akonadi
sudo /etc/init.d/apparmor reload
akonadictl start

Davor Cubranic (cubranic) wrote :

Unfortunately, that didn't help. Even stopping AppArmor (by running '/etc/init.d/apparmor stop') makes no difference whatsoever -- I still get the same stack trace.

Davor Cubranic (cubranic) wrote :

I found interesting error messages in ~/.local/share/akonadi/db_data/mysql.err. It looks like the problem might have to do with the (zero) size of a MySQL log file in that directory, which causes MySQL startup to abort. (see attachment)

Davor Cubranic (cubranic) wrote :

Deleting both ib_logfile0 and ib_logfile1 in ~/.local/share/akonadi/db_data/, and then running 'akonadictl start' fixed the problem for me. I don't know how the log file got corrupted, but the system should be more robust than that. I mean, deleting a (corrupted) log file does not need to be an unrecoverable error!

It's completely reasonable in such a situation to delete it and proceed with startup. The database should still be in a consistent state, and the worst that could happen is that the last few transactions are missing. Or at the very least notify the user what the real problem is. As it is, akonadi server's output is utterly unhelpful, and it took me forever to find mysql.err, where the problem was actually clearly identified.

Davor Cubranic (cubranic) wrote :

Mysql.err that identified the problem:

100519 9:49:38 [Note] Plugin 'FEDERATED' is disabled.
InnoDB: Error: log file ./ib_logfile1 is of different size 0 0 bytes
InnoDB: than specified in the .cnf file 0 67108864 bytes!
100519 9:49:38 [ERROR] Plugin 'InnoDB' init function returned error.
100519 9:49:38 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
100519 9:49:38 [ERROR] Unknown/unsupported table type: innodb
100519 9:49:38 [ERROR] Aborting

100519 9:49:38 [Note] /usr/sbin/mysqld-akonadi: Shutdown complete

Davor Cubranic (cubranic) wrote :

I opened a related KDE bug (https://bugs.kde.org/show_bug.cgi?id=238217) to bring up the issue of Akonadi's fragility to the attention of KMail devs.

Version: (using KDE 4.4.2)
OS: Linux
Installed from: Ubuntu Packages

I originally reported this bug in Ubuntu Launchpad: https://bugs.launchpad.net/ubuntu/+source/akonadi/+bug/578357

I am opening it here as a wider issue: Akonadi is very fragile in the face of mysql startup problems, and that has a knock-on effect of preventing basic user applications (mail, PIM) from running.

The most visible symptom of the problem was that after starting up KMail, I would get an error dialog stating that Akonadi is not running. After closing the dialog, the app would close also.

I started investigating, and got voluminous, but not very helpful error output when running 'akonadictl start' from the command line. (I'll attach a copy.) Further poking and twiddling with apparmor and other workarounds suggested in various bugs and forums was unsuccessful. Eventually, I stumbled upon mysql.err in ~/.local/share/akonadi/db_data/ with the following contents:

100519 9:49:38 [Note] Plugin 'FEDERATED' is disabled.
 InnoDB: Error: log file ./ib_logfile1 is of different size 0 0 bytes
 InnoDB: than specified in the .cnf file 0 67108864 bytes!
 100519 9:49:38 [ERROR] Plugin 'InnoDB' init function returned error.
 100519 9:49:38 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
 100519 9:49:38 [ERROR] Unknown/unsupported table type: innodb
 100519 9:49:38 [ERROR] Aborting
100519 9:49:38 [Note] /usr/sbin/mysqld-akonadi: Shutdown complete

File 'ib_logfile1' was indeed of size zero. Deleting it and 'ib_logfile0', and then restarting 'akonadictl' seemed to fix the problem and I now have a working mail reader.

I don't know how the log file got corrupted, but the system should be more robust than that. I mean, having a zero-size or corrupted log file does not need to be an unrecoverable error!

It's completely reasonable in such a situation to delete it and proceed with startup. The database should still be in a consistent state, and the worst that could happen is that the last few transactions are missing. Or at the very least notify the user what the real problem is. As it is, akonadi server's output is utterly unhelpful, and it took me forever to find Akonadi's mysql.err, where the problem was actually clearly identified.

Created attachment 43736
Output of 'akonadictl start' when MySql fails to start

This is the output of running 'akonadictl start' from the console before I vixed the problem by deleting the corrupted logfiles. This output actually repeats three times. (Why?)

summary: - Akonadi startup error with no MySql
+ MySql log file problem prevents Akonadi startup
yope (djander) wrote :

I had the same problem, when using NFS home directories. Somehow local users didn't have trouble.
The instructions from comment #1 fixed it for me.

Jürgen (j-w-ott) wrote :

I have the same problem but even a new generated user (avoiding damaged files somewhere in the user subdirs) cannot start the akonadi database. The creation obviously is successful.

I have also started the appamor in complain mode, allthough I didn't knew what I was doing.

For me it seems to be a permission problem but I don't know how to find the right spot where to fix it.

This makes KDE completely useless !

Download full text (4.3 KiB)

I got a similar issue with kontact/akonadi (No resource agents have been found).

$ sudo aa-complain mysqld
[sudo] password for gerhard:
Setting /usr/sbin/mysqld to complain mode.

$ sudo aa-complain mysqld-akonadi
Setting /etc/apparmor.d/usr.sbin.mysqld-akonadi to complain mode.

$ sudo /etc/init.d/apparmor reload
 * Reloading AppArmor profiles Skipping profile in /etc/apparmor.d/disable: usr.bin.firefox
Skipping profile in /etc/apparmor.d/disable: usr.bin.firefox-3.7
Warning: found usr.bin.freshclam in /etc/apparmor.d/force-complain, forcing complain mode
Warning: found usr.sbin.mysqld in /etc/apparmor.d/force-complain, forcing complain mode
Warning: found usr.sbin.ntpd in /etc/apparmor.d/force-complain, forcing complain mode

$ akonadictl restart
[...]
[akonadiserver] Found mysql_upgrade: "/usr/bin/mysql_upgrade"
"akonadiserver" [out] "Looking for 'mysql' as: /usr/bin/mysql
"
"akonadiserver" [out] "Looking for 'mysqlcheck' as: /usr/bin/mysqlcheck
"
"akonadiserver" [out] "Running 'mysqlcheck' with connection arguments: '--port=3306' '--socket=/var/run/mysqld/mysqld.sock' '--socket=/home/gerhard/.local/share/akonadi/db_misc/mysql.socket'
"
[akonadiserver] /usr/bin/mysqlcheck: Got error: 2002: Can't connect to local MySQL server through socket '/home/gerhard/.local/share/akonadi/db_misc/mysql.socket' (2) when trying to connect
[akonadiserver] FATAL ERROR: Upgrade failed
[akonadiserver] Database "akonadi" opened using driver "QMYSQL"
[akonadiserver] DbInitializer::run()
[...]

$ mysqlcheck --port=3306 --socket=/var/run/mysqld/mysqld.sock --socket=/home/gerhard/.local/share/akonadi/db_misc/mysql.socket -g -a --auto-repair akonadi
akonadi.collectionattributetable OK
akonadi.collectionmimetyperelation OK
akonadi.collectionpimitemrelation OK
akonadi.collectiontable OK
akonadi.flagtable OK
akonadi.mimetypetable OK
akonadi.parttable OK
akonadi.pimitemflagrelation OK
akonadi.pimitemtable OK
akonadi.resourcetable OK
akonadi.schemaversiontable OK

$ cat $HOME/.local/share/akonadi/db_data/mysql.err
100717 13:59:35 [Note] Plugin 'FEDERATED' is disabled.
100717 13:59:36 InnoDB: Started; log sequence number 0 3093356
100717 13:59:36 [Note] /usr/sbin/mysqld-akonadi: ready for connections.
Version: '5.1.41-3ubuntu11-log' socket: '/home/gerhard/.local/share/akonadi/db_misc/mysql.socket' port: 0 (Ubuntu)

$ grep -i ERR $HOME/.local/share/akonadi/db_data/mysql.err.old|sort -u
100608 19:05:39 [ERROR] Aborting
100608 19:05:39 [ERROR] Plugin 'InnoDB' init function returned error.
100608 19:05:39 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
100608 19:05:39 [ERROR] Unknown/unsupported table type: innodb
100608 19:05:39 InnoDB: Operating system error number 11 in a file operation.
InnoDB: Error in opening ./ibdata1
Inno...

Read more...

mue.de (emuede) wrote :
Download full text (5.4 KiB)

I think, i have the same problem (even after 5 Trials, to install a fresh Lucid Lynx (Kubuntu) without the old /home-Folder).
My last error-log ($HOME/.local/share/akonadi/db_data/mysql.err) is this:

100728 8:12:58 [Note] Plugin 'FEDERATED' is disabled.
100728 8:12:59 InnoDB: Started; log sequence number 0 130014
100728 8:13:00 [Warning] Can't open and lock time zone table: Table 'mysql.time_zone_leap_second' doesn't exist trying to live without them
100728 8:13:00 [ERROR] Can't open and lock privilege tables: Table 'mysql.servers' doesn't exist
100728 8:13:00 [Note] /usr/sbin/mysqld-akonadi: ready for connections.
Version: '5.1.41-3ubuntu11-log' socket: '/home/muelux/.local/share/akonadi/db_misc/mysql.socket' port: 0 (Ubuntu)

** First, akonadi came with:

Test 14: ERROR
--------

No resource agents found.
Details: No resource agents have been found, Akonadi is not usable without at least one. This usually means that no resource agents are installed or that there is a setup problem. The following paths have been searched: '/usr/share/akonadi/agents /usr/share/akonadi/agents'. The XDG_DATA_DIRS environment variable is set to '/usr/share:/usr/share:/usr/local/share', make sure this includes all paths where Akonadi agents are installed to.

Test 16: ERROR
--------

Previous Akonadi server error log found.
Details: The Akonadi server did report error during its previous startup into <a href='/home/muelux/.local/share/akonadi/akonadiserver.error.old'>/home/muelux/.local/share/akonadi/akonadiserver.error.old</a>.

File content of '/home/muelux/.local/share/akonadi/akonadiserver.error.old':
Control process died, committing suicide!

*** I tried:

sudo apt-get install virtuoso-server mysql-server-5.1
rm -r $HOME/.local/share/akonadi
akonadictl start
mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/
mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket
akonadictl stop
akonadictl start

*** after that it came with:

"akonadiserver" [out] "Running 'mysqlcheck' with connection arguments: '--port=3306' '--socket=/var/run/mysqld/mysqld.sock' '--socket=/home/muelux/.local/share/akonadi/db_misc/mysql.socket'
"
[akonadiserver] /usr/bin/mysqlcheck: Got error: 2002: Can't connect to local MySQL server through socket '/home/muelux/.local/share/akonadi/db_misc/mysql.socket' (2) when trying to connect
[akonadiserver] FATAL ERROR: Upgrade failed
[akonadiserver] Failed to use database "akonadi"
[akonadiserver] Query error: "Unknown database 'akonadi' QMYSQL: Unable to execute query"
[akonadiserver] Database error: "Can't connect to local MySQL server through socket '/home/muelux/.local/share/akonadi/db_misc/mysql.socket' (2) QMYSQL: Unable to connect"
[akonadiserver] Trying to create database now...
[akonadiserver] Database "akonadi" opened using driver "QMYSQL"

...]
### it creates a new database with log-messages (extract):

[akonadiserver] DbInitializer::run() done
[akonadiserver] skipping update 2
[akonadiserver] skipping update 3
[akonadiserver] skipping update 4
[akonadiserver] skipping update 8
[akonadiserver] skipping update 10
[akonadiserver] skipping update 12
[akonadiserver] skipping update 13
[akonadiserver]...

Read more...

I have the same problem too. Moving to akonadi has been a disaster for me as a user. Beside this breakage, how does one even backup one's email files even?

Orang_Gila (bug-thug) wrote :

Almost exactly the same errors:
..........
[akonadiserver] /usr/bin/mysqlcheck: Got error: 2002: Can't connect to local MySQL server through socket '/home/lmpam/.local/share/akonadi/db_misc/mysql.socket' (2) when trying to connect
[akonadiserver] FATAL ERROR: Upgrade failed ............. Then repeats several time before aborting.

Linux Mint 9 Isadora,
Linux 2.6.32.21.22-generic (i686)
Desktop Environment: GNOME 2.30.2
akonadi-server: 1.3.1-0ubuntu3 (lucid)
mysql-server-core-5.1: 5.1.41-3ubuntu12.6 (lucid)
kontact & kmail: 4:4.4.2-0ubuntu5 (lucid)

*** Bug 256527 has been marked as a duplicate of this bug. ***

Same comment as #7: I have the problem with my mom's installation (she uses kmail) on an nfs-mounted home dir. That's quite strange (apart from the fact that a mail client needs a fulblown database server); also: I fixed it by moving the entire akonadi directory out of the way (bit of a brute-force way I admit).

*** This bug has been confirmed by popular vote. ***

Shriramana Sharma (jamadagni) wrote :

Domenichelli's fix worked for me on Natty Narwhal where I had the same problem (also see bug 707464).

Maarten Bezemer (veger) wrote :

Thank you for taking the time to report this bug and helping to make Ubuntu better. We are sorry that we do not always have the capacity to look at all reported bugs in a timely manner. There have been many changes in Ubuntu since that time you reported the bug and your problem may have been fixed with some of the updates. It would help us a lot if you could test it on a currently supported Ubuntu version. When you test it and it is still an issue, kindly upload the updated logs by running apport-collect 578357 and any other logs that are relevant for this particular issue.

Changed in akonadi (Ubuntu):
status: New → Incomplete
Davor Cubranic (cubranic) wrote :

Maarten, I haven't tried reproducing this and so it's not a "testable" bug. A KUbuntu upgrade two years ago failed and corrupted some MySQL file, so that Akonadi wouldn't start. If I see that again when I upgrade to Precise, I'll post here.

Davor Cubranic (cubranic) wrote :

As for all the AppArmor comments, you guys should open a separate bug. This one is about Akonadi failing to start if there is a problem with the MySQL log file. Look at "~/.local/share/akonadi/db_data/mysql.err" for the error message if you're not sure.

Maarten Bezemer (veger) wrote :

Thank you for your response. It would be nice if you could update this report when you upgrade to Precise, either with good or bad news :)
For now I'll leave the report status 'Incomplete' awaiting your response.

About the AppArmor comments, that should indeed go into a new bug report, so if anyone is still able to (re)produces these errors: Please open a new report and post the details there. So those issues can be solved as well.

Harald Sitter (apachelogger) wrote :

Closing in favor of upstream report.

Changed in akonadi (Ubuntu):
status: Incomplete → Invalid

This bug has only been reported for versions older than KDEPIM 4.14 (at most akonadi-1.3). Can anyone tell if this bug still present?

If noone confirms this bug for a recent version of akonadi (part of KDE Applications 15.08 or later), it gets closed in about three months.

Changed in akonadi:
importance: Unknown → Medium
status: Unknown → Incomplete
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.