[SRU req.] kdepim stability fixes; calendar crashes when using google calendar [regression]

[themroc]

This is an accumulated stability update fixing a number of crashes and information loss situations as well as regressions in kdepim 3.5.10.

Going through the changelog:

  * Add kubuntu_17_upstream_r860376.diff which fixed a pointer problem
    in korganizer leading to objects being lost.
Rationale: Can lead to informaion loss in certain cases. Extremely minor fix.

  * Add kubuntu_18_upstream_r857911.diff to make sure that the inbox
    folder doesn't stay hidden if content is added.
    - See http://bugs.kde.org/show_bug.cgi?id=168544
Rationale: kde 3.5.10 introduced hiding of the local inbox if it wasn't in use; if hidden starting to use the inbox would however not make it appear again. This introduced a "practical" mail loss. This patch fixes this and will reveal the mail again.
TEST CASE: Open a fresh kmail, inbox is hidden. Add a new account that delivers to the hidden inbox, the inbox doesn't show up again. Install the version from hardy-proposed and the inbox now appears.

  * Add kubuntu_19_fix_gcal_crash.diff which fixes problems with remote
    resources crashing kontact, most notably when using GCal. (LP: #286905)
Rationale: Regression from kdepim 3.5.9.
TEST CASE: Add google calendar to kontact; switch to kmail and back and kontact crashes. Install the version from hardy-proposed and the crashing stops.

  * Add kubuntu_20_upstream_r882942.diff which prevents incidence loss
    under certain conditions as the UID map wasn't being reloaded.
Rationale: Can lead to large amounts of informaion loss in certain cases. Minor fix.

  * Add kubuntu_21_upstream_r882976.diff to prevent KMail crashing
    due to certain malicious base64-encoded mail parts.
Rationale: Can crash KMail by tricking the user into opening a malicious mail.

The accumulated update in total has a very little regression potential; and furthermore fixes already introduced regressions.

Test build of the proposed SRU is avaible in my PPA: https://launchpad.net/~andreas-wenning/+archive

[Harald Sitter (apachelogger)]

Please install kdepim-dbg and get a new crashreport.
Thank you.

[themroc (rauchweihe)]

enclosed the new crashreport with kdepim-dbg installed.

[Andreas Wenning (andreas-wenning)]

Does the crash seem to happen at any special event? Eg. when starting the calendar, opening an event, mouse over an event etc.

[themroc (rauchweihe)]

I normaly start contact with the overview. Then I can see the dates from google and local calendar. I can switch to the emails (kmail), notes and journal and aggregator. All works normal. But when I switch to one of the both calendarviews, contact crash, when the mouse left the calendar window or when I want to switch to an other contactpart (e.g. Email).

It also crash, when I click anything in contact if calendar window is activate.

It crash too, when overview is activate and I doubleclick on a date from the calendar (normally to edit the date).

[themroc (rauchweihe)]

contact crash on all of my notebooks (three: one 64bit Kubuntu 8.04 and two 32bit Kubuntu 8.04) after the update.
I

[Andreas Wenning (andreas-wenning)]

Confirmed due to duplicates, and as I can manage to make it crash as well.

[themroc (rauchweihe)]

Now I realize, that the google calendar does probably crash contact.
Contact crash due to a calendar on a webdav server:

[Resource_FdnrB4vBO4]
DownloadUrl=https://mediacenter.gmx.net/aufgaben.ics
LastLoad=2935093,2,28,0,0,0
LastSave=2935093,2,28,0,0,0
ReloadInterval=10
ReloadPolicy=1
ResourceIdentifier=FdnrB4vBO4
ResourceIsActive=true
ResourceIsReadOnly=false
ResourceName=aufgaben
ResourceType=remote
SaveInterval=10
SavePolicy=0
UploadUrl=https://mediacenter.gmx.net/aufgaben.ics

[themroc (rauchweihe)]

Sorry, I mean:
"Now I realize, that the google calendar doesn't probably crash contact."

[Andreas Wenning (andreas-wenning)]

There seems to be a more generic problem with remote resources in Kontact, and that is what I'm trying to find the reason for.

[Andreas Wenning (andreas-wenning)]

I seem to get a similar crash, although the backtrace isn't exactly the same. Just to keep all the information gathered, I've attached the log here.

[Andreas Wenning (andreas-wenning)]

And here is the valgrind output from the crash.

[Andreas Wenning (andreas-wenning)]

If any of you can still reproduce the original crash is able to get a valgrind trace for it, that would be very helpful to compare with the related crash. You can obtain it by running

valgrind /usr/bin/kontact 2>&1 | tee ~/valgrind.log

Be aware that kontact will run VERY slow when you do it; this is completely normal.

[Rich Ibbotson (richibb)]

I am not one of the people to report the original crash, but I do see a crash using Korganizer (within Kontact) when I have enabled a Google calendar in read-only mode. I believe that this is the same problem that Andreas has reproduced. The crash report is attached, and it looks very similar.

My steps to reproduce this problem (note that this started occurring with kde 3.5.10; I had done the same steps previously with kde 3.5.9 without a crash):

1) Open kontact, switch to calendar
2) "Enable" a Google calendar by checking the checkbox in the Resource View
3) Scroll the calendar window.

On scrolling the calendar window, kontact/korganizer will crash.

It's interesting to note that I can no longer even disable the Google calendar in the Resource View by clicking on the checkbox. Attempting to click anywhere in the korganizer view seems to crash kontact in the same way. I've disabled this resource by ~/.kde/share/config/kresources/calendar/stdrc by hand and setting ResourceIsActive=false for the Google calendar. The most recent change to my system was to install kdepim-dbg and valgrind.

I will attempt to attach a valgrind report to another comment.

[Rich Ibbotson (richibb)]

Here is the valgrind trace from my last crash

[themroc (rauchweihe)]

Here are two valgrind log's from two crashes:
With several remote resources: Google and Webdav.

1st Crash after open kalendar and moving mouse outside kontact.

[themroc (rauchweihe)]

2nd Crash after open kalendar, waiting a longer moment, then switch to kmailwindow, wait, then switch again to calendarwindow, wait not so long, push the kmail button-> contact chrash!

[themroc (rauchweihe)]

Sorry, here is the first crash:

after open kalendar and moving mouse outside kontact:

[Andreas Wenning (andreas-wenning)]

Thx a lot. That confirmed my suspision that my crash is in fact related to the crash originally described in the bug.

I'll try to look further into it and see if I'm able to figure out exactly what goes wrong here.

[devweasel (devweasel)]

In case you need yet another report, her is mine (Sorry for the delay; went abroad)...
The log is kind of split in two halfves. First part is startup of korganizer, Second part (starting with the "Invalid null key" entries) is on activation of the Groupwise Resource until the crash.

[Andreas Wenning (andreas-wenning)]

I think I've found the error and has come up with a fix for it. Please test kdepim 4:3.5.10-0ubuntu1~hardy3~ppa7 that is avaible from my PPA at https://launchpad.net/~andreas-wenning/+archive and see if it fixes the problem for you.

[devweasel (devweasel)]

Thanks for the effort so far, a read-only Google Calendar (via remote ICS) seems to work fine now.
Unfortunately a Novell Groupwise Calendar (via Soap) and/or an eGroupware Calendar (via remote ICS) still both crash with the "usual" symptoms (regardless whether flagged as read-only or not).

[Andreas Wenning (andreas-wenning)]

Alexander:
Thanks for reporting back; sounds like we're partly there now. Can I have you post a new valgrind log for the two types of crashes you mention there.

[devweasel (devweasel)]

Here you are, two Valgrind traces:
  valgrind_egw.log - eGroupware Resource
  valgrind_ngw.log - Novell Groupwise Resource

[scido (domi-mailing-list)]

Hi,
I've got the same symptoms and problems already discussed here. I upgraded the packages from the Andreas ppa repository. I don't have any problem with googlecalendar, but I still struggling with a webcal calendar (from "Remember the Milk" site). Maybe is it related to this kind of cal scheme.

Kubuntu 8.04.1
Kde 3.5.10

Bye,
Scido

[Andreas Wenning (andreas-wenning)]

scido and Alexander:
Thanks for testing things. After looking through the code I have a few possible fixes (they are actually trade-offs between speed and stability). I've uploaded the first one as kdepim - 4:3.5.10-0ubuntu1~hardy3~ppa8 in my PPA: https://launchpad.net/~andreas-wenning/+archive/ . I would be glad if you could test if it fixes any of the crashes you're still experiencing.

[scido (domi-mailing-list)]

I'm sorry but nothing change. Now I canceled each remote calendar and I just use local ical file. I maybe found a new important information: I think the crash is related to korganizer/kmail interaction. For example, I open kontact, I can choose korganizer and then kmail module, and everything is ok. After a while, I choose korganizer module and kontact crashes with these lines in my .xsession-errors:

*** KMail got signal 11 (Crashing)
KCrash: Application 'kontact' crashing...

So it seems that kmail is crashing, not korganizer. But if I remove korganizer module from kontact, everything goes fine.
I'm starting to have an headache...

Tell me if you need debug or something similar.

Bye,
Scido

[devweasel (devweasel)]

Sadly i have to second scodi's comment - nothing seems to have changed from a user's perspective. The crash occurs right after the download is reported to be complete.
Would it help to fetch valgrind traces from the working 3.5.9 for those two resources?
Attached are two valgrind traces for the latest crashes...

[Andreas Wenning (andreas-wenning)]

The problem is keeping the internal layers in korganizer in sync under some circumstances; this results in problems when redrawing some of the views, and that is why it could seem like a kontact problem, as it manifests itself when changing to kmail and back.

I have one last tes here (kdepim - 4:3.5.10-0ubuntu1~hardy3~ppa10) in my PPA, that you can test. If this one doesn't help I'm out of ideas for now and will prepare the fix for google calendar crashes on it's own. If it does change to the better, I'll need to finetune the fix performance-wise.

[scido (domi-mailing-list)]

I tried last packages but nothing changes... Thanks for your support, but now I think the only solution is downgrading kdepim packages to 3.5.9 (I tried and everything works smoothly).

Thank you.
Bye,
Scido

[themroc (rauchweihe)]

Thank you, for your support. I have upgraded to Kubuntu 8.10 (kde4), so I can no longer report to this Problem.
The remote ressources work in kde4 well. IMAP are now better suported.

[Andreas Wenning (andreas-wenning)]

debdiff for the SRU request.

[Scott Kitterman (kitterman)]

I'm currently offline and unable to sponsor this, but I've been following
awn's progress on this and am definitely in favor of the update.

[Jonathan Riddell (jr)]

Uploaded to hardy-proposed, waiting for archive accept.

[Martin Pitt (pitti)]

Is this an issue in jaunty/intrepid? Please close the task if it's already fixed there.

[Martin Pitt (pitti)]

Accepted into hardy-proposed, please test and give feedback here. Please see https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[Andreas Wenning (andreas-wenning)]

Not an issue in intrepid/jaunty. Mostly regressions due to changes prior to splitting the kde 4 stack out from kde3.

[Andreas Wenning (andreas-wenning)]

Both test cases confirmed using the version from hardy-proposed. The inbox shows up as it should, and google calendar resources no longer crashes korganizer. If any of you originally hit by the bug can confirm one or the other test-case, and that the package works in general, please do.

[Launchpad Janitor (janitor)]

This bug was fixed in the package kdepim - 4:3.5.10-0ubuntu1~hardy3

---------------
kdepim (4:3.5.10-0ubuntu1~hardy3) hardy-proposed; urgency=low

  * Add kubuntu_17_upstream_r860376.diff which fixed a pointer problem
    in korganizer leading to objects being lost.
  * Add kubuntu_18_upstream_r857911.diff to make sure that the inbox
    folder doesn't stay hidden if content is added.
    - See http://bugs.kde.org/show_bug.cgi?id=168544
  * Add kubuntu_19_fix_gcal_crash.diff which fixes problems with remote
    resources crashing kontact, most notably when using GCal. (LP: #286905)
  * Add kubuntu_20_upstream_r882942.diff which prevents incidence loss
    under certain conditions as the UID map wasn't being reloaded.
  * Add kubuntu_21_upstream_r882976.diff to prevent KMail crashing
    due to certain malicious base64-encoded mail parts.

 -- Andreas Wenning <email address hidden> Tue, 18 Nov 2008 13:39:44 +0100

[Martin Pitt (pitti)]

Copied to hardy-updates.

[Rich Ibbotson (richibb)]

I have just updated to 3.5.10-0ubuntu1~hardy3 from hardy-updates, but I still see the GCal crash (LP: #286905).
dpkg-query -l 'kdepim*' shows the following:
kdepim-dbg 4:3.5.10-0ubuntu1~hardy3
kdepim-dev
kdepim-doc 4:3.5.10-0ubuntu1~hardy3
kdepim-kio-plugins 4:3.5.10-0ubuntu1~hardy3
kdepim-kresources 4:3.5.10-0ubuntu1~hardy3
kdepim-wizards 4:3.5.10-0ubuntu1~hardy3

I've attached the kcrash. I can produce a valgrind report if you would like to see one.

[Andreas Wenning (andreas-wenning)]

Hi Rich

There has been two types of crashes happening regarding remote resources. I have tried to split them up into this bug and bug 284911 and from the trace I can see that you experience the one tracked in bug 284911 . I haven't been able to find a fix for the issue in bug 284911, but I'll keep looking to see if anything comes up in an upstream commit.

[Rich Ibbotson (richibb)]

Andreas,

Thanks for the update. I am actually seeing this crash when accessing a Google calendar in a "Calendar in Remote File" resource. That bug report seems to be (at least from the title) related to Novell Groupwise resources. I don't know if that is useful information in tracking the problem down... at least, the problem doesn't seem to be restricted to that type of resource.

If I can send you any other info that helps, just let me know.

[Andreas Wenning (andreas-wenning)]

Hi Rich

Both bugs hit remote resources more generally; and the title pretty much reflected where it was discovered first.

The problem is, that even a valgrind-log doesn't give much clue to, where the exact problem is located; but if you have any suspicion against anything, just add it in the other bug.