Can't migrate summary to sqlite

[Jan Newmarch]

Binary package hint: evolution

Trying Ubuntu 8.10 alpha 5 from Ubuntu 8.04. Evolution wants to migrate existing maliboxes to sqlite, but runs out of memory before completing. I have about 2Gb of mail in 14,000 files in my Mail directory. Errors are

1st attempt)
(evolution:8296): camel-local-provider-WARNING **: failed a quick-sync, trying a full sync

(evolution:8296): camel-local-provider-WARNING **: failed a quick-sync, trying a full sync
Saving 2/2 dirty records of gmail
Failure for store_db_path : [/home/newmarch/.evolution/mail/local/folders.db]
Retry with /home/newmarch/.evolution/mail/mbox/home/newmarch/.evolution/mail/local/folders.db failed

(evolution:8296): camel-WARNING **: Could not save object state file to '/home/newmarch/.evolution/mail/local/.#wireless.cmeta': Too many open files

(evolution:8296): camel-local-provider-WARNING **: Could not open/create index file: Too many open files: indexing not performed
Segmentation fault (core dumped)

2nd attempt)
(evolution:9497): camel-local-provider-WARNING **: Could not save summary for local providers

GLib-ERROR **: /build/buildd/glib2.0-2.17.7/glib/gmem.c:156: failed to allocate 4352 bytes
aborting...
Trace/breakpoint trap (core dumped)

3rd attempt)
(evolution:7808): camel-local-provider-WARNING **: Could not save summary for local providers

GLib-ERROR **: /build/buildd/glib2.0-2.17.7/glib/gmem.c:136: failed to allocate 8200 bytes
aborting...
Trace/breakpoint trap (core dumped)

All later attempts duplicate one of these. No core dump in the current directory, sorry. Evolution cannot currently be used since it dies on the import.

Addition:

 the gdb trace is

GLib-ERROR **: /build/buildd/glib2.0-2.17.7/glib/gmem.c:136: failed to allocate 8200 bytes
aborting...

Program received signal SIGTRAP, Trace/breakpoint trap.
[Switching to Thread 0xb6360720 (LWP 6471)]
0xb6a4e6ff in g_logv () from /usr/lib/libglib-2.0.so.0

#0 0xb6a4e6ff in g_logv () from /usr/lib/libglib-2.0.so.0
#1 0xb6a4ea76 in g_log () from /usr/lib/libglib-2.0.so.0
#2 0xb6a4cece in g_malloc () from /usr/lib/libglib-2.0.so.0
#3 0xb789f1b9 in e_mempool_alloc () from /usr/lib/libedataserver-1.2.so.11
#4 0xb7b0adbe in ?? () from /usr/lib/libcamel-1.2.so.13
#5 0xb7b0b3b3 in ?? () from /usr/lib/libcamel-1.2.so.13
#6 0xb7b0ba6f in camel_mime_parser_step () from /usr/lib/libcamel-1.2.so.13
#7 0xb7d86066 in camel_folder_summary_info_new_from_parser ()
   from /usr/lib/libcamel-provider-1.2.so.13
#8 0xb7d88039 in camel_folder_summary_add_from_parser ()
   from /usr/lib/libcamel-provider-1.2.so.13
#9 0xb4f4e437 in ?? ()
   from /usr/lib/evolution-data-server-1.2/camel-providers/libcamellocal.so
#10 0xb4f4e843 in ?? ()
   from /usr/lib/evolution-data-server-1.2/camel-providers/libcamellocal.so
#11 0xb4f48577 in camel_local_summary_check ()
   from /usr/lib/evolution-data-server-1.2/camel-providers/libcamellocal.so
#12 0xb4f46362 in camel_local_folder_construct ()
   from /usr/lib/evolution-data-server-1.2/camel-providers/libcamellocal.so
#13 0xb4f49b23 in camel_mbox_folder_new ()
   from /usr/lib/evolution-data-server-1.2/camel-providers/libcamellocal.so
#14 0xb4f4c399 in ?? ()
   from /usr/lib/evolution-data-server-1.2/camel-providers/libcamellocal.so
---Type <return> to continue, or q <return> to quit---
#15 0xb7da74a3 in camel_store_get_folder ()
   from /usr/lib/libcamel-provider-1.2.so.13
#16 0xb5e484e9 in ?? ()
   from /usr/lib/evolution/2.24/components/libevolution-mail.so
#17 0xb5e48510 in ?? ()
   from /usr/lib/evolution/2.24/components/libevolution-mail.so
#18 0xb5e486a4 in ?? ()
   from /usr/lib/evolution/2.24/components/libevolution-mail.so
#19 0xb5e4a60b in em_migrate ()
   from /usr/lib/evolution/2.24/components/libevolution-mail.so
#20 0xb5e5a30f in ?? ()
   from /usr/lib/evolution/2.24/components/libevolution-mail.so
#21 0xb7fd7576 in _ORBIT_skel_small_GNOME_Evolution_Component_upgradeFromVersion () from /usr/lib/evolution/2.24/libeshell.so.0
#22 0xb7533680 in ORBit_c_stub_invoke () from /usr/lib/libORBit-2.so.0
#23 0xb7fd9045 in GNOME_Evolution_Component_upgradeFromVersion ()
   from /usr/lib/evolution/2.24/libeshell.so.0
#24 0x080554ce in e_shell_attempt_upgrade ()
#25 0x08055ab8 in e_shell_construct ()
#26 0x08055c27 in e_shell_new ()
#27 0x0805e680 in ?? ()
#28 0xb6a42881 in ?? () from /usr/lib/libglib-2.0.so.0
#29 0xb6a447e1 in g_main_context_dispatch () from /usr/lib/libglib-2.0.so.0
#30 0xb6a47e83 in ?? () from /usr/lib/libglib-2.0.so.0
#31 0xb6a483a2 in g_main_loop_run () from /usr/lib/libglib-2.0.so.0
#32 0xb758dce3 in bonobo_main () from /usr/lib/libbonobo-2.so.0
#33 0x0805ed13 in main ()

Valgrind not so useful:
    Valgrind's memory management: out of memory:
       newSuperblock's request for 68141056 bytes failed.
       3061645312 bytes have already been allocated.
    Valgrind cannot continue. Sorry.

I have 1Gb RAM, 2GB swap.

[Pedro Villavicencio (pedro)]

Thanks for your bug report. Please try to obtain a backtrace http://wiki.ubuntu.com/DebuggingProgramCrash and attach the file to the bug report. This will greatly help us in tracking down your problem.

[Sebastien Bacher (seb128)]

you can try to run gconf-editor, go to apps, evolution and change the *version there to 2.24.0, that should make the boxes being migrated when you open those and not when evolution is starting

[Jan Newmarch (jan-newmarch)]

The workaround from Sebastien works okay. I'll keep running evolution under gdb just in case anything else happens. I didn't mention that the out-of-memory would occur after about half-hour of converting the mailboxes. Not sure if it had reached the end of a looooong process involving thousands of maiboxes or ran out before finishing converting.

[Pedro Villavicencio (pedro)]

Closing this bug report as no further information has been provided. Please feel free to reopen this bug if you can provide the information asked for. Thanks!.

[Pierre-François Dutot (pfdutot)]

I just had this very same bug, and solved it using the same workaround. Before applying the workaround, I installed evolution-dbgsym and did a backtrace.

[Pierre-François Dutot (pfdutot)]

I also did a valgrind log to show the memory usage of the faulty process.

[Pierre-François Dutot (pfdutot)]

re-opening the bug since I uploaded traces

[Pierre-François Dutot (pfdutot)]

A final comment on how the bug appeared : Migrating from 8.04 to 8.10, while migrating mailboxes to SQLite (after a few seconds of migration) on a large mailbox.

Linux sans 2.6.27-7-generic #1 SMP Tue Nov 4 19:33:06 UTC 2008 x86_64 GNU/Linux
evolution 2.24.1-0ubuntu2

[Pedro Villavicencio (pedro)]

trace looks different please open a new bug rather and also install the evolution-dbgsym and libcamel dbgsym packages since the trace you attached is missing a few symbols, thanks in advance.