[REGRESSION] Unable to modify CalDAV items ("this.mItemInfoCache[aNewItem.id] is undefined")

[Daniel Gnoutcheff]

This bug showed up with the 1.9+build1-0ubuntu0.12.10.1 update that I pulled in yesterday.

Steps to reproduce:
1. Create an event inside a calendar hosted by a CalDAV sever (in my case, a Darwin Calendar Server instance running on the local machine).
2. Restart Thunderbird.
3. Try to modify that event's start time.

Expected:
The new start time should be shown on the calendar display, and it should be saved to the CalDAV sever as well

Actual:
The event remains unchanged, both on the local display and on the server. The Thunderbird error console contains:
> Timestamp: 11/22/2012 11:08:42 AM
> Error: this.mItemInfoCache[aNewItem.id] is undefined
> Source File: file:///usr/lib/mozilla/extensions/%7B3550f703-e582-4d05-9a08-453d09bdfdc6%7D/%7Be2fda1a4-762b-4020-b5ad-a41df1933103%7D/components/calDavCalendar.js
> Line: 660

It should be noted that other calendar programs can modify the same event on the CalDAV server, and those modifications show up in Lightning after a synchronize. Furthermore, this bug does not affect events that were initially created on the CalDAV server by other calendar programs, and it also doesn't effect events that were created on that server by previous versions of Lightning. It only seems to occur when one creates an event using the latest version of Lightning and then tries to modify the event (under the same profile) in Lightning. (I'm betting that other instances of Lighting using different profiles would also be able to edit it, just like other calendar programs can -- I'll verify this shortly.)

[Spamcop (spamcop)]

When selecting the task and trying to delete it, the error console prints:

Error: this.mItemInfoCache[aItem.id] is undefined
Source File: file:///Users/xxx/Library/Thunderbird/Profiles/lujdvccv.default/extensions/%7Be2fda1a4-762b-4020-b5ad-a41df1933103%7D/components/calDavCalendar.js
Line: 869

When I try to edit the task in any way, I get:

Error: this.mItemInfoCache[aNewItem.id] is undefined
Source File: file:///Users/xxx/Library/Thunderbird/Profiles/lujdvccv.default/extensions/%7Be2fda1a4-762b-4020-b5ad-a41df1933103%7D/components/calDavCalendar.js
Line: 696

[Ssitter (ssitter)]

Did it work in previous releases? Does it work in most recent beta or nightly test builds?

Please provide more detailed information that can help developers to reproduce the problem in a fresh profile, including other information like type and version of CalDAV server, etc.

In addition you could go to Thunderbirds advanced preference editor and enable the calendar.debug... preferences. This should provide more information from CalDAV provider in Error Console.

[Spamcop (spamcop)]

I never had a task to delete in any previous release and no, I cannot downgrade this system to test that (this is a production environment). The most recent beta or nightly doesn't work with my version of Thunderbird any longer and when being used with a nightly build of Thunderbird as well, they behave absolutely identical in really every aspect (including this bug).

No, I cannot create a fresh profile (again, this is a production environment, especially the CalDAV server, which is crucial for the company I work at) and use it with this CalDav Server, which is an iCal Server, version unknown, OS X Version unknown (Snow Leopard or Lion, I suppose).

There is no option calendar.debug, however there is calendar.debug.log and calendar.debug.log.verbose and after setting both to true, and even restarting Thunderbird, the console log output looks exactly as in my initial report.

-----------

Was anything above really helpful? I don't think so. Sorry, but I think the information in my initial report were more than adequate to do something about this bug. This is a very simple bug: Don't access something that may be undefined, verify before accessing it and if undefined, either define it or work without that information. Form the perspective of a software developer, I'm very disappointed by the Lightning team so far. Every bug I report only leads to plenty of usually pointless questions, I'm getting pushed around to try old versions, new versions, beta versions, and try out other obscure pseudo-fixes or provide information, totally unrelated to the bug in question. I'm a long term bug reporter, my first bug on bugzilla.mozilla.org dates back to 2002-07-09(!) and yes, I'm a little bit pissed that Lighting bug reports only receive replies, that are intended to pass back the ball to the reporter, instead of ever showing any progress on the actual issue. I saw this happening with several other Lighting bug reports, too. This is not the professional level of bug tracking I'm used from Firefox.

[Philipp-bugzilla (philipp-bugzilla)]

Mohit, seems this bug is back :-(

TGOS: Stefan was suggesting you create a new profile to see if it happens with a fresh profile too. Creating a new profile will not harm your production profile and you can still use it.

Fixing this bug is unfortunately not that easy. Of course we could make the error message go away, but this won't fix the bug. Other things will break.

If you want to make this go away, please delete the cache.sqlite in your profile/calendar-data. This will make the cache items resynchronize and the error will go away.

I can't guarantee this bug won't reappear afterwards, but if you have been using subsequent versions of Lightning, the initial bug may have happened with a previous release. We fixed something that might make this not reappear, but it would be nice to find out about that.

So in short, specifically for your situation

1. delete you cache
2. use your calendar
3. if it shows up again, let us know.

--

I'm sorry this bug has bitten you and you have had such a negative experience with filing bugs. On the other hand, you have to understand us. We do the same for each bug reporter and confirming bugs is not something we can do alone without spending less time fixing bugs and more time confirming them.

If the user is using an older version, it may have been fixed in a newer build. If the user is using the newest version it would help very much to find the regression range, which is why we might ask to test an older version. As a software developer, I would have hoped you know this situation.

The more information we get, the better we can reproduce the bug. For example, when I repeat the steps you posted in comment 0, I don't get those error messages. Without getting further information, how do you expect me to get rid of the bug?

We are not asking these questions to be annoying, but to find out what situation causes the bug.

[Mohit Kanwal (mohit-kanwal)]

Hi TGOS,

Phillip had tried to fix this issue in a previous version via bug 700637. You can follow the discussion over there where it was usually occurring during the processing of recurring events.

Looks like it is happening now with tasks :(

Could you provide information, or just a scenario, where we can isolate the issue? In the case of bug 700637 we had managed to reproduce the issue by fiddling around with the system clock.

Maybe it happens only with recurring tasks. or something like that. I am, in the meanwhile trying to test out a new build to see if I can reproduce this issue with tasks the way I managed to do it in bug 700637.

[Spamcop (spamcop)]

@Phillip:

Okay, I quit Thunderbird, deleted cache.sqlite (not local.sqlite), and started Thunderbird again. After a short time, the task came back, so this task was in fact never deleted on the calendar server. I selected it and deleted it, this time without an error message. I restarted Thunderbird to make sure it won't come back and it didn't, so it was deleted locally and on the server. This makes me think, the issue is only related to a local cache "corruption"; though corruption is maybe a bit "harsh", it might just be that the local cache is out of date.

For testing purposes, I created three other tasks: One in the future, one in the past, one without any date; all three for the same calendar on the same calendar server that caused the problem before. I restarted Thunderbird again and tried to delete all three tasks. I was able to delete all three without an issue. So this is not a reproducible issue and thus most likely not related to my calendar server.

As a developer I see two bugs here:

  1. The bug that causes cache database corruption. I have no idea what causes this bug but fixing this bug will fix the "real issue". The problem is that once the database is already corrupted, there is no way to find out which past action caused this corruption. I understand that this is no easy bug to fix. It could be a cache update issue, like the cache not being updated correctly or only partially being updated, causing an inconsistent view on the data in question.

  2. The bug that once the database is corrupted, there is no way to recover from that state. You said making the error message go away will break other things but what if you just catch this error and handle it by "invalidating" the cached data for this "object"? In that case the data is re-fetched on next sync and the corruption will most likely go away, just as it did when I deleted the cache database. This does not solve the "real issue", it's just a pain killer for the symptoms, however, sometimes such pain killers are necessary, since if you have some headache, it's often much more important to get rid of it than it is to find out what the real cause of the headache is. Looking at the fix of bug 700637, it does exactly what I suggested: It verifies the value is defined and otherwise uses a fallback "method" if it isn't, as well as catching an exception where necessary. So the fix for 700637 does not really "solve" the problem (which may be server side and thus unsolvable), it also just works around the broken state.

I'm sorry about my harsh comment before, I know how "painful" bug tracking can be, especially in situations, where a bug keeps reoccurring every now and then but there is no way to reproduce it at will. However, I'm a software developer and I'm also a part time software support agent, and if I was telling a customer to go back to a previous version or try the latest beta version, I knew that I'm basically just pushing him around to buy us some more time, since the chances that the latest beta fixes this bug and I'm unaware of that fact are close to nonexistent and the chances that the bug was introduced with the last release are almost equally small. Also havin...

[Daniel Gnoutcheff (gnoutchd)]

I've just managed to reproduce this issue in a fresh Thunderbird profile, but it seems I'll need to revise my steps to reproduce. The offline support feature and the synchronize button seem to be relevant. Here are my steps to reproduce now:

1) Select a CalDAV calendar with "offline support" enabled
2) Create an event
3) Click "synchronize"
4) Restart Thunderbird
5) Try to change the start time of the event created earlier

Also, it turns out that it's not necessary to create the event in Lightning in order to trigger this bug: it's also possible to trigger it with events created elsewhere, like so:
1) Add a CalDAV calendar to Lightning with "Offline Support" enabled
2) Close Thunderbird
3) Use another calendar application (I used Evolution) to create an event on the same CalDAV calendar
4) Start Thunderbird, the new event should show up on the Lightning calendar display as well
5) Click "synchronize"
6) Restart Thunderbird
7) Try to change the start time of the event created earlier

HTH!

[Daniel Gnoutcheff (gnoutchd)]

Ugh, sorry about the noise, but I had better clarify:

I said in my initial bug report that this bug does not effect events created outside of Lightning: just to be clear, it appears that was wrong about this, and my second comment includes the counterexample I found. I suspect that I got fooled earlier because I wasn't paying attention to my use of the "synchronize" button.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[tschoie (tschoie)]

I see this too, but in my experience it seems solely connected to "offline support": as long as this feature is enabled, modifying calDAV events fails (i.e. dismissing alarms in my case). Everything seems to work as intendend while "offline support" is turned off.

[Tom Louwrier (tom-louwrier)]

This looks very much like the issue I'm experiencing. It may be related to 769118, I posted a comment there too.
It's not confined to recurring events, but also dismissing alarms that pop up, both for missed events and current ones.

I get this about 23 times in the log after I click 'dismiss all' once.

Timestamp: 11-12-12 10:51:13
Error: this.mItemInfoCache[aNewItem.id] is undefined
Source File: file:///usr/lib/mozilla/extensions/%7B3550f703-e582-4d05-9a08-453d09bdfdc6%7D/%7Be2fda1a4-762b-4020-b5ad-a41df1933103%7D/components/calDavCalendar.js
Line: 660

Running Tbird 17.0 / Lightning 1.9 on Ubuntu 12.04 amd64.
My CalDAV server is a Sogo one, current version. Using the server's web interface to dismiss, delete or edit the events does not solve the situation in Tbird/Lightning.

Switching off local calendar caching, restarting Tbird and enabling caching again (need to be able to work offline) clears the hanging events. After a while one comes up that can not be dismissed and then all following events will join the queue of missed events that can not be dismissed.

cheers
Tom

[Enzomas (enzomas)]

Same symptoms for me since I upgraded to lightning 1.9 (with TB 17.0).

This bug affect my calendars on different servers (some on MacOSX server, some others on Sogo).

Steps to reproduce (tested with a brand new installation/profile) :
- set a new calendar on a caldav server (offline mode disabled)
- create a new event
- enable offline support for this calendar
- refresh calendar
- restart TB
- trying to edit or delete the previous event is impossible :
(Error : this.mItemInfoCache[aNewItem.id] is undefined
in file:///C:/Users/xxxxxxx/AppData/Roaming/Thunderbird/Profiles/xxxxxxxx/extensions/%7Be2fda1a4-762b-4020-b5ad-a41df1933103%7D/components/calDavCalendar.js
Ligne : 660)

Disabling offline support (or deleting cache.sqlite) is the only way to modify/delete the event again.

[Mohit Kanwal (mohit-kanwal)]

Hi enzomas,

I suspect this might be happening due to bug 742528. If you are able to get your hands dirty with code, can you try editing

~/AppData/Roaming/Thunderbird/Profiles/xxxxxxxx/extensions/%7Be2fda1a4-762b-4020-b5ad-a41df1933103%7D/components/calDavCalendar.js

and attempt to add the lines that were removed in the patch for bug 742528.

Check if this solves the problem.

[Enzomas (enzomas)]

(In reply to Mohit Kanwal [:redDragon] from comment #8)
> Hi enzomas,
>
> I suspect this might be happening due to bug 742528. If you are able to get
> your hands dirty with code, can you try editing
>
> ~/AppData/Roaming/Thunderbird/Profiles/xxxxxxxx/extensions/%7Be2fda1a4-762b-
> 4020-b5ad-a41df1933103%7D/components/calDavCalendar.js
>
> and attempt to add the lines that were removed in the patch for bug 742528.
>
> Check if this solves the problem.

In Lightning 1.9, these two lines are still present in calDavCalendar.js.

[Kaulich (kaulich)]

This bug is still true for lightning 2.1a2.

The two lines mentioned in comment #8 are still there, but setting a task to completed works only for online calendar for me.

Also resetting the cache from the right mouse button leads to a crash. Maybe this is connected.

[Bugmail-a (bugmail-a)]

I encountered this condition attempting to edit an event. It occurred after Thunderbird crashed for another, unrelated reason (sending a message). Resetting the calendar cache via the contextual menu on the calendar itself resolved it.

Does the cache maintain itself in a way that an application crash won't corrupt it?

Should the cache be automatically reset after a Thunderbird crash? Is it?

[Ingo-goeppert (ingo-goeppert)]

Same problem on my installation, TB 17 ESR with lightning 1.9 and zarafa as server.

I stopped TB and deleted the cache.sqlite. Then I can create new events and edit the old events. After I restarted TB several times and created a new event in Outlook, the problem happens again.

Lightning is unusable for me at the moment, because reminders pop up again and again. I have to switch off and on the cached mode. Very frustrating.

What can I do to help to fix the bug? Generate logfiles? How?

[Www-mozilla-org (www-mozilla-org)]

TB 17.0.2, Lightning 1.9, Davmail 4.1.0-2042 talking to Exchange. All running on Debian Linux but with the standard Mozilla installation packages.

I do not get this problem for events that I have created myself. I regularly get this error for events created by others to which I have been invited. The Davmail debug log shows no attempt by Lightning to talk CalDAV to it.

TB console shows this:

Timestamp: 09/01/13 11:26:41
Error: this.mItemInfoCache[aNewItem.id] is undefined
Source File: file:///home/chris/.thunderbird/0bm31s7m.default/extensions/%7Be2fda1a4-762b-4020-b5ad-a41df1933103%7D/components/calDavCalendar.js
Line: 660

My calendar is not read-only. Usually the calendar is marked as available for offline support. The problem appears to go away when I untick the offline support flag.

Going back into the calendar properties I notice it's switched on again (possibly a separate fault?); at this stage I cannot determine whether it is the flag display or the flag setting that is incorrect.

[Marc-v (marc-v)]

I have developed a Sabre based Caldav/Carddav Server
which supports webdav-sync(RFC6578) AND getctag(caldav-ctag-02)!

I can confirm the bug for TB 17-ESR/Lighning 1.9/SOGo Connector 17.0.2!

I discovered that lightning uses a combination of getctag and webdavsync if the server supports webdav-sync!

If I DISABLE the getctag Property on the Server the bug is gone ...
every refresh is done by webdav-sync
and the cache-data seems to be updated and stable after Restart/Reboot!

[Philipp-bugzilla (philipp-bugzilla)]

Marc, could you check if this bug still occurs with the patch for bug 827078 applied?

[Marc-v (marc-v)]

After serveral days of testing I can confirm:

the patch for bug 827078 solves the problem!

Great Job - Thanx!!!!

[Ssitter (ssitter)]

Thanks for testing. I'll resolve this problem as fixed for 1.9.1 by Bug 827078.

[Paul White (paulw2u)]

Upstream bug showing "RESOLVED FIXED" on 2013-02-27
Target milestone: Lightning 1.9.1
Marking "Fix Released" to close