Upgrading Firefox/Thunderbird when an add-on update is waiting to be installed hides the add-on

[Micah Gersten]

This affects users by temporarily hiding their addon until another addon update/install prompts the addon manifest to be rebuilt.

[Abhay-saraf (abhay-saraf)]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:7.0) Gecko/20100101 Firefox/7.0
Build ID: 20110816154714

Steps to reproduce:

Upgrade from 6.0 to 7.0 on beta channel.

Actual results:

NoScript addon was uninstalled

Expected results:

Addons should not have been affected

[Robert-bugzilla (robert-bugzilla)]

Are you sure it isn't listed in the bottom section of the add-ons manager?

[Abhay-saraf (abhay-saraf)]

Yeah pretty sure. I checked about:addons first thing.

[Robert-bugzilla (robert-bugzilla)]

Please double check. The installer and the updater both don't touch the user's profile so it is extremely unlikely that the add-on was by removed by either of them. It is possible that the add-ons manager code which does have code that touches the extensions in the user's profile could have removed it.

[Abhay-saraf (abhay-saraf)]

I am a s/w developer myself and understand the importance of repros. So I double checked it on both my machines.

I have the same situation on my laptop so next time I restart FF I will come back and report on it when the update gets applied on the restart

[Robert-bugzilla (robert-bugzilla)]

Have you had a chance to check yet?

[Patrik-nyborg (patrik-nyborg)]

I can confirm this (6.0.2=>7.0 release channel).

Mozilla/5.0 (Windows NT 6.1; WOW64; rv:7.0) Gecko/20100101 Firefox/7.0
and
Mozilla/5.0 (Windows NT 6.1; rv:7.0) Gecko/20100101 Firefox/7.0

[Savagebiscuits (savagebiscuits)]

I had a similar issue. While I was running the Firefox 6.0beta Linux build, I went to Tools>Addons and noticed that I had a pending update for NoScript (green stripes, etc). I ignored it and went on browsing, thinking that I'll wait till I restart my browser for that update to occur (I generally leave my browser open till the end of the day, then shut it down before shutting down my computer). However, some hours later, I noticed that Firefox was wanting to update to the 7.0beta, so I decided to let it update and restart the browser. It was once that I restarted the browser and it was running as 7.0beta, that I noticed that NoScript was missing. Its .xpi file was in the extension's directory in my profile, but it wasn't listed as an addon by Firefox (other extensions were fine). I then reinstalled NoScript, with no further issues, and the extension had updated fine since.

This was on a Debian Squeeze (32bit) machine.

[Robert-bugzilla (robert-bugzilla)]

That sounds like the add-ons manager is not handling add-on updates properly when the application version changes after an application update. What ever the case, application update doesn't touch the profile, modify extensions, etc. so moving over to add-ons manager.

[Dtownsend (dtownsend)]

Confirmed, this line is throwing NS_ERROR_UNEXPECTED for some reason: http://mxr.mozilla.org/mozilla-central/source/toolkit/mozapps/extensions/XPIProvider.jsm#1172

[G-maone (g-maone)]

So, if I understand it right, the downloaded file is kept in the profile (comment 7).
Does this mean that another Firefox update can detect the mess and fix it?

[G-maone (g-maone)]

And could someone figure out whether it's better removing latest add-on version from AMO or declaring it incompatible with Fx 7?

[kmaglione (maglione-k)]

Giorgio: I'd recommend removing it. The AMO update check mechanism is supposed to Do The Right Thing in circumstances where the newest version is not marked compatible with a given version and an older is, but that's unfortunately not reliable. If you do remove it, just let me know when you've re-uploaded and I'll review it right away.

[Jah (jah)]

Had the same problem today. Went from latest release of v6 to v7 on WinXP SP3 and there was no incompatibility warning regarding noscript, it was silently uninstalled. I'm not sure which version of noscript I was using before it was removed, but it was one of the dev channel releases.

[Savagebiscuits (savagebiscuits)]

(In reply to Giorgio Maone from comment #10)
> So, if I understand it right, the downloaded file is kept in the profile
> (comment 7).

Yes, I noticed that the new .xpi file was in the profile (in the extensions directory), yet wasn't installed and also missing in about:addons after the browser update and restart.

[Dtownsend (dtownsend)]

I've figured out what the problem here is and it will likely affect any user that has a new add-on waiting to install or an update to an existing add-on waiting to be installed at the time they update from anything earlier than Firefox 7 to Firefox 7 or later.

The problem stems from how the add-on install process works. We download the XPI and store it in a staging directory. We also load the information from install.rdf into a JS object. Any updated compatibility information (as well as other things I'm sure I'm not remembering) is applied to this JS object and then it is written alongside the XPI as a JSON text file. After restarting Firefox loads the JSON data rather than needing to parse install.rdf and do compatibility checks a second time.

Normally that works out fine however if the version of Firefox changes in the meantime and new fields are now supported in install.rdf the loaded JSON data won't contain those new fields, best case the install/update will complete but the database won't contain the correct data from install.rdf, worst case the install fails. In this case the latter is happening. Bug 653637 introduced the new fields that is causing this.

I have a hack that sort of solves the problem but I have a strong feeling that it may break things. The better solution is probably to do a better job of passing the data between Firefox instances and I have a few ideas of how to do that that I need to investigate further.

[Dtownsend (dtownsend)]

(In reply to Graham Gordon from comment #14)
> (In reply to Giorgio Maone from comment #10)
> > So, if I understand it right, the downloaded file is kept in the profile
> > (comment 7).
>
> Yes, I noticed that the new .xpi file was in the profile (in the extensions
> directory), yet wasn't installed and also missing in about:addons after the
> browser update and restart.

That is a strange thing and it has the weird side effect that if the user installs a different add-on then NoScript would suddenly be detected and installed again.

[G-maone (g-maone)]

(In reply to Dave Townsend (:Mossop) from comment #15)

> I have a hack that sort of solves the problem but I have a strong feeling
> that it may break things. The better solution is probably to do a better job
> of passing the data between Firefox instances and I have a few ideas of how
> to do that that I need to investigate further.

"Solve" and "solution" like in just "this won't happen again on next major update" or also "next minor update will complete broken installation/updates"?

[Dtownsend (dtownsend)]

(In reply to Giorgio Maone from comment #17)
> (In reply to Dave Townsend (:Mossop) from comment #15)
>
> > I have a hack that sort of solves the problem but I have a strong feeling
> > that it may break things. The better solution is probably to do a better job
> > of passing the data between Firefox instances and I have a few ideas of how
> > to do that that I need to investigate further.
>
> "Solve" and "solution" like in just "this won't happen again on next major
> update" or also "next minor update will complete broken
> installation/updates"?

The code changes would mean that it won't happen again the next time we introduce new fields to install.rdf. I suspect that the next update to Firefox would make it re-detect the present XPI in the profile already.

[kmaglione (maglione-k)]

(In reply to Dave Townsend (:Mossop) from comment #18)
> The code changes would mean that it won't happen again the next time we
> introduce new fields to install.rdf. I suspect that the next update to
> Firefox would make it re-detect the present XPI in the profile already.

Can we confirm that? If it is the case, is there something else we can do to automatically re-install the add-ons that were uninstalled from this bug for users with automatic updates? If not, can we push out a minor release with this fixed so that it happens anyway? Millions of users having their favorite add-ons disappear is not good for Firefox's public image.

[Dtownsend (dtownsend)]

(In reply to Kris Maglione [:kmag] from comment #19)
> (In reply to Dave Townsend (:Mossop) from comment #18)
> > The code changes would mean that it won't happen again the next time we
> > introduce new fields to install.rdf. I suspect that the next update to
> > Firefox would make it re-detect the present XPI in the profile already.
>
> Can we confirm that? If it is the case, is there something else we can do to
> automatically re-install the add-ons that were uninstalled from this bug for
> users with automatic updates? If not, can we push out a minor release with
> this fixed so that it happens anyway? Millions of users having their
> favorite add-ons disappear is not good for Firefox's public image.

I'm working with the release drivers as we speak to figure out the best path forwards

[Hskupin (hskupin)]

Dave, is there anything QA could help at this stage?

[Dtownsend (dtownsend)]

(In reply to Kris Maglione [:kmag] from comment #19)
> (In reply to Dave Townsend (:Mossop) from comment #18)
> > The code changes would mean that it won't happen again the next time we
> > introduce new fields to install.rdf. I suspect that the next update to
> > Firefox would make it re-detect the present XPI in the profile already.
>
> Can we confirm that? If it is the case, is there something else we can do to
> automatically re-install the add-ons that were uninstalled from this bug for
> users with automatic updates? If not, can we push out a minor release with
> this fixed so that it happens anyway? Millions of users having their
> favorite add-ons disappear is not good for Firefox's public image.

I've also just confirmed that this works, it reverts back to the previous version of NoScript, but that'd then automatically updated in the next day.

[Dtownsend (dtownsend)]

(In reply to Henrik Skupin (:whimboo) from comment #21)
> Dave, is there anything QA could help at this stage?

Not right now thanks

[Clegnitto (clegnitto)]

(In reply to Dave Townsend (:Mossop) from comment #22)
> (In reply to Kris Maglione [:kmag] from comment #19)
> > (In reply to Dave Townsend (:Mossop) from comment #18)
> > > The code changes would mean that it won't happen again the next time we
> > > introduce new fields to install.rdf. I suspect that the next update to
> > > Firefox would make it re-detect the present XPI in the profile already.
> >
> > Can we confirm that? If it is the case, is there something else we can do to
> > automatically re-install the add-ons that were uninstalled from this bug for
> > users with automatic updates? If not, can we push out a minor release with
> > this fixed so that it happens anyway? Millions of users having their
> > favorite add-ons disappear is not good for Firefox's public image.
>
> I've also just confirmed that this works, it reverts back to the previous
> version of NoScript, but that'd then automatically updated in the next day.

A) I'm not exactly sure this is millions of users. Yes, we have reports in this bug but we aren't seeing widespread problems yet (admittedly could be due to a lag in feedback channels)

B) Does uninstalling a different add-on trigger the "hidden" ones to show up / reinstall as well?

[kmaglione (maglione-k)]

(In reply to Christian Legnitto [:LegNeato] from comment #24)
> A) I'm not exactly sure this is millions of users. Yes, we have reports in
> this bug but we aren't seeing widespread problems yet (admittedly could be
> due to a lag in feedback channels)

I'm basing my assumption on the fact that this seems to be 100%
reproducable when an add-on update was downloaded just before a
Firefox update. NoScript has over 2 million users, and Giorgio
pushed out an update several hours before the Firefox release team
did. There have also been reports of affected AdBlock Plus users. So
this easily has the potential to affect millions of users and I'd
rather err on the side of caution.

[Dtownsend (dtownsend)]

(In reply to Christian Legnitto [:LegNeato] from comment #24)
> B) Does uninstalling a different add-on trigger the "hidden" ones to show up
> / reinstall as well?

Yes

[Fligtar-h (fligtar-h)]

(In reply to Christian Legnitto [:LegNeato] from comment #24)
> A) I'm not exactly sure this is millions of users. Yes, we have reports in
> this bug but we aren't seeing widespread problems yet (admittedly could be
> due to a lag in feedback channels)
>

I wouldn't expect widespread reports of this for days or weeks -- most users won't notice their add-ons are gone immediately.

The day before and of a Firefox release is always the busiest time for putting out add-on updates.

[G-maone (g-maone)]

I removed the latest versions of my add-on both from the beta and the dev channel, but as a more general stop-gap measure couldn't AMO temporarily stop answering add-on update pings which come from Gecko 4-6.x?

[Fligtar-h (fligtar-h)]

(In reply to Giorgio Maone from comment #28)
> I removed the latest versions of my add-on both from the beta and the dev
> channel, but as a more general stop-gap measure couldn't AMO temporarily
> stop answering add-on update pings which come from Gecko 4-6.x?

We'd have to devise a pretty complicated solution in order to do that, because most add-ons aren't explicitly compatible with Firefox releases anymore and depend on AMO update responses saying they are compatible.

[kmaglione (maglione-k)]

(In reply to Justin Scott [:fligtar] from comment #29)
> (In reply to Giorgio Maone from comment #28)
> > I removed the latest versions of my add-on both from the beta and the dev
> > channel, but as a more general stop-gap measure couldn't AMO temporarily
> > stop answering add-on update pings which come from Gecko 4-6.x?
>
> We'd have to devise a pretty complicated solution in order to do that,
> because most add-ons aren't explicitly compatible with Firefox releases
> anymore and depend on AMO update responses saying they are compatible.

I think he means pings coming from Gecko 4-6. I.e., for requests where currentAppVersion is less than 7, return no results until this issue has been sorted out.

[Mardeg (mardeg)]

*** Bug 689842 has been marked as a duplicate of this bug. ***

[Shadowfyr55 (shadowfyr55)]

Just to be clear, I ran into this too. Its not uncommon for Firefox to state that some addons are "incompatible", and it will, usually, update some of them, when installing the new version. In my case, I got a list, which I mostly didn't pay attention to, except that I know it included Linkification (self adjusted, because its one of those that half the time doesn't update at all), and HTTPS-Everywhere, which I very recently added. I don't know if Noscript was "in the list" or not, but a number where listed as, "Current version is not compatible, check for updates?" As soon as the client loaded, every single thing that was in that list was just flat missing. Once I "installed" another plugin, as suggested by someone here, and restarted, they all reappeared.

It should be noted that, for me, installing a new Noscript "failed" to make this happen, or make it reappear, I had to install a completely different plugin, picked semi-randomly out of the ones available. So, something in the process of checking for compatibility and updating the list of new versions to load, failed. Its possible it effected only very recent additions, since it killed ones that I installed and/or manually updated, the last time I got such an update message as well. Everything installed "prior" to that update, stayed. And I know Noscript updated since then, I added the HTTPS one after that, and I also hand patched Linkification, to make it compatible (since I didn't figure what it did what likely broken between versions). I will bet donuts to dollars that everything else listed was in the same category, with the exception of a few that really are out of date, and I just haven't gotten rid of (or can't... I just love ones that refuse to let you uninstall them...).

[Dtownsend (dtownsend)]

(In reply to Patrick Elliott from comment #32)
> Just to be clear, I ran into this too. Its not uncommon for Firefox to state
> that some addons are "incompatible", and it will, usually, update some of
> them, when installing the new version. In my case, I got a list, which I
> mostly didn't pay attention to, except that I know it included Linkification
> (self adjusted, because its one of those that half the time doesn't update
> at all), and HTTPS-Everywhere, which I very recently added. I don't know if
> Noscript was "in the list" or not, but a number where listed as, "Current
> version is not compatible, check for updates?" As soon as the client loaded,
> every single thing that was in that list was just flat missing. Once I
> "installed" another plugin, as suggested by someone here, and restarted,
> they all reappeared.

The list you refer to, was it something that displayed while you were still running Firefox 6, part of the UI telling you that a new version of Firefox was available, or was it something that displays after you first started Firefox 7?

[Shadowfyr55 (shadowfyr55)]

Hmm. Hard to say, actually. I think the list was shown when starting the new one, maybe, then there was an error, during the actual start up, in a sort of off color, yellowish, box, during the step where updates to the addons would normally happen, which said that there had been a problem installing new version of the addons. The original list is the one that always shows up when patching versions, and a conflict happens, so I am pretty sure it happened when ever that does normally. But the error happened "during" the update for new addons, so.. I think that happens during start up, not in the old one.

In any case, I think Firefox does realize something went wrong, hence the error message, during the step where download of the addons would happen, after starting up, but it didn't know what to do about it.

[Dtownsend (dtownsend)]

(In reply to Patrick Elliott from comment #34)
> Hmm. Hard to say, actually. I think the list was shown when starting the new
> one, maybe, then there was an error, during the actual start up, in a sort
> of off color, yellowish, box, during the step where updates to the addons
> would normally happen, which said that there had been a problem installing
> new version of the addons. The original list is the one that always shows up
> when patching versions, and a conflict happens, so I am pretty sure it
> happened when ever that does normally. But the error happened "during" the
> update for new addons, so.. I think that happens during start up, not in the
> old one.

The bug we have here happens before the UI you're talking about appears so the UI simply wouldn't contain add-ons that got hit by it. You're also saying you lost a lot of add-ons, some of which definitely wouldn't have had any updates waiting to install before you upgraded. So either we don't yet correctly understand this bug, or there is a different bug that you hit.

[Cbeard (cbeard)]

Our China team reports that they are seeing this across their entire user base as their localized distribution of Firefox relies upon a bundle of add-ons to package and deliver their local market adaptations -- and we updated all of those add-ons this week ahead of the release

[Shadowfyr55 (shadowfyr55)]

Well, as I said, my guess is that it may effect addons that "only" where updated "recently", like within weeks of the new patch. The bug may be happening before that UI appears, but the UI itself, in my case, just happened to catch that something went odd. I am not entirely certain which one though. Thinking on it.. I may have seen one, on shutdown, that said they where not compatible, then one on start, which showed "others" which where not up to date (legitimately), but not the ones that had gone missing, and *then* a message, telling me that something went badly wrong. But I am not 100% certain of the order.

But, now that you mention it, its likely the bug, and the initial list, happened in the process of the first one closing down. In any case, all the "lost" ones where, as with Chris, *recent*, and they all popped up, and without "incompatibility" messages, as soon as a new addon was put in the queue, and a new restart was done.

[Dtownsend (dtownsend)]

Created attachment 562976
patch rev 1

This is the patch that seems like the best solution. We still cache the entire manifest when preparing to install (for backwards compatibility) however on the startup it completely loads the install manifest from install.rdf (thus parsing it and getting all the fields that the new Firefox supports) and then also loads the cached manifest and copies across only certain appropriate properties. If any of the properties expected seem to be missing then we just ignore and carry on. This should guarantee a safe and working add-on install.

There is a small performance penalty associated here because we have to read install.rdf on startup for every new add-on. I haven't measured the cost of that yet but I expect it to be low and as it is only for the case when a new add-on is installed I'd expect it to be worth it for correcting this.

I think the only risk here is that if I have missed any properties to copy across. That would appear as the add-on changing state in some way on upgrade (maybe getting disabled etc.) but I am growing confident that I have them all. I'll be double checking that in the morning but figured I might as well put the patch up now as adding properties to that list won't really change the review. We can also be over-zealous about listing properties there (maybe even just try to copy all properties) because the new code will simply ignore any it doesn't find and the cached properties will always be the same or newer than those loaded from install.rdf.

The testcase from bug 664833 already checks installing an update when the DB schema and app version changes. The only thing missing was to make the cached manifest have a necessary property missing so I updated the test to do that. Without this fix the test fails in the same way that I've been seeing in this bug.

I've pushed this to the try server, if people want to test builds they will be showing up at http://<email address hidden>

[Dtownsend (dtownsend)]

(In reply to Dave Townsend (:Mossop) from comment #38)
> The testcase from bug 664833 already checks installing an update when the DB
> schema and app version changes. The only thing missing was to make the
> cached manifest have a necessary property missing so I updated the test to
> do that. Without this fix the test fails in the same way that I've been
> seeing in this bug.

Forgot to add, the testcase here is good and I think an accurate test for this, however it suffers from the same problem that much of our automated tests for updates do, it doesn't actually test moving from one version of Firefox to another in reality. So we certainly want a litmus test for this scenario.

[Robert-bugzilla (robert-bugzilla)]

Comment on attachment 562976
patch rev 1

Looks like a decent approach.

btw: there is a bug to test app updates after the nightly build. Perhaps a similar test for this case should be done at the same time.

[Hskupin (hskupin)]

(In reply to Dave Townsend (:Mossop) from comment #39)
> tests for updates do, it doesn't actually test moving from one version of
> Firefox to another in reality. So we certainly want a litmus test for this
> scenario.

We already have a litmus test for it. Sadly it's only part of the major update test scenario. Since we do minor updates now, those tests haven't been executed. We should create a test group for minor updates and include all appropriate tests from the MU test group. I have already started this discussion so we can hopefully proceed on it quickly.

[Steffen Wilberg (steffen-wilberg)]

This has been reported by several users in a German IT news site:
the update deleted Adblock Plus (http://www.heise.de/newsticker/foren/S-Firefox-Update-loescht-Adblock-Plus/forum-210502/msg-20851562/read/)

[Epinal99-bugzilla (epinal99-bugzilla)]

I guess bug 689752 (NoScript icon disappeared from add-on manager after auto-updating to Firefox 7) is a dupe of this bug.

Issue reported here http://forums.informaction.com/viewtopic.php?p=31326#p31326 with NoScript.

[Hskupin (hskupin)]

*** Bug 689752 has been marked as a duplicate of this bug. ***

[Trev-moz (trev-moz)]

I did some testing and can confirm that uninstalling another extension resolves the problem - as does any other change to your extensions (enabling/disabling an extension, updating some extension etc). The add-on isn't really uninstalled and stays in the extensions/ directory, the update is actually applied correctly. It is only that the add-on manager temporarily "forgets" about it. I also verified that a minor browser update fixes the issue automatically - it forces the add-on manager to check the list of add-ons again.

So far I have only seen one report of this issue (https://addons.mozilla.org/addon/adblock-plus/reviews/313717/), no idea how many Adblock Plus users are affected. Is removing the update from AMO really the best course of action? I am pretty certain that I will receive lots of complains about Adblock Plus not being compatible with Firefox 7 then - install.rdf of Adblock Plus 1.3.9 lists Firefox 7.0a1 as maxVersion which means trouble even if AMO says something different.

[VanillaMozilla (vanillamozilla)]

*** Bug 601937 has been marked as a duplicate of this bug. ***

[ashughes (anthony-s-hughes)]

Just want to get a clear set of steps to reproduce this bug so QA can quickly verify the fix. Please confirm these steps:

1) Install Firefox <7.0
2) Install an add-on which is compatible to Firefox 7.0 but not the latest version of the add-on
3) Restart Firefox
4) Check for and download the update for the add-on (DO NOT INSTALL IT)
5) Check for and download the update for Firefox 7.0.1
6) Restart Firefox to install the update

Expected Result:
Firefox and add-ons updated

Please confirm the above steps. Also, what is the expected update path / test for:
 * 4.* users
 * 5.* users
 * 6.* users
 * 7.0 users
 * 4b*/ 5b* / 6b* users
 * 3.6 Major Update

[Dtownsend (dtownsend)]

(In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #47)
> Just want to get a clear set of steps to reproduce this bug so QA can
> quickly verify the fix. Please confirm these steps:
>
> 1) Install Firefox <7.0
> 2) Install an add-on which is compatible to Firefox 7.0 but not the latest
> version of the add-on
> 3) Restart Firefox
> 4) Check for and download the update for the add-on (DO NOT INSTALL IT)
> 5) Check for and download the update for Firefox 7.0.1
> 6) Restart Firefox to install the update
>
> Expected Result:
> Firefox and add-ons updated

These look like the correct STR to me.

I would also suggest a second set of steps for verification:

1) Install Firefox <7.0
2) Install an add-on which is compatible to Firefox 7.0 but not the latest version of the add-on
3) Restart Firefox
4) Check for and download the update for the add-on (DO NOT INSTALL IT)
5) Exit and install Firefox 7.0
6) Check that the add-on is now missing from the add-ons manager
7) Check for and download the update for Firefox 7.0.1
8) Restart Firefox to install the update

Expected Result:
The add-on that disappeared in step 6 should have reappeared.

[Vlad-mozbugs (vlad-mozbugs)]

Can confirm the fix for upgrading from 6.0 to try server build
http://<email address hidden>/

[Chimel (chimel31)]

I lost AddBlock Plus, had to reinstall manually.
http://www.reddit.com/r/firefox/comments/ktvyl/upgrading_to_firefox_7_uninstalls_adblock_plus/

In addition to fixing the bug, can you:
1) Add this test case (add-on that has a pending update) to your normal test cases
2) Notify users of any add-on that was disabled or removed after a Firefox upgrade

Given the new fast dev cycle between upgrades, it is really important to log every anomaly instead of just ignoring it.

I would have liked to test the patch, but my add-ons are now up to date.
It would be great if you could create a test add-on that users can update locally to try and repro this type of bug, instead of relying on actual add-ons that may or may not have been updated.

[Dtownsend (dtownsend)]

Created attachment 563099
patch rev 2

Made the minor change of adding applyBackgroundUpdates to the list of properties copied across, otherwise identical, probably doesn't need re-review but better safe than sorry

[Dtownsend (dtownsend)]

Pushed to trunk: https://hg.mozilla.org/mozilla-central/rev/80b55a615ac9

[Dojnd (dojnd)]

*** Bug 690020 has been marked as a duplicate of this bug. ***

[Clegnitto (clegnitto)]

Comment on attachment 563099
patch rev 2

Approved everywhere. Land all the places!

[Clegnitto (clegnitto)]

To be clear we'll want to land this on the "default" branch of:

releases/mozilla-aurora
releases/mozilla-beta
releases/mozilla-release

as well. Thanks!

[Clegnitto (clegnitto)]

http://hg.mozilla.org/releases/mozilla-aurora/rev/8aa82e4926dc
http://hg.mozilla.org/releases/mozilla-beta/rev/4d8d9922dfca
http://hg.mozilla.org/releases/mozilla-release/rev/a494b8cd79bf

[Hskupin (hskupin)]

(In reply to Chimel from comment #50)
> It would be great if you could create a test add-on that users can update
> locally to try and repro this type of bug, instead of relying on actual
> add-ons that may or may not have been updated.

I have created an extension which demonstrates that and will be compatible with all future versions of Firefox. We will use it for our Litmus test which needs to be added for this particular case. You can find it here:

https://www.mozqa.com/data/firefox/addons/update-nonrestartless/

[Fligtar-h (fligtar-h)]

Since it hasn't been linked here yet, in addition to the fix going into Firefox, we've also created a workaround to recover any hidden add-ons.

http://blog.mozilla.com/addons/2011/09/28/issue-discovered-with-firefox-add-on-upgrades/
http://support.mozilla.com/en-US/kb/add-ons-hidden-after-updating-firefox-7

[Stefan-blumenrath (stefan-blumenrath)]

(In reply to Justin Scott [:fligtar] from comment #58)
> Since it hasn't been linked here yet, in addition to the fix going into
> Firefox, we've also created a workaround to recover any hidden add-ons.

Great, unfortunaly this won't work in SeaMonkey. I don't dare just to change the install.rdf - would that help?

(I've lost AB, but was able to reinstall, after reinstalling some filters disappeared).

[Antoine-mechelynck-gmail (antoine-mechelynck-gmail)]

(In reply to stefan.blumenrath from comment #59)
> (In reply to Justin Scott [:fligtar] from comment #58)
> > Since it hasn't been linked here yet, in addition to the fix going into
> > Firefox, we've also created a workaround to recover any hidden add-ons.
>
> Great, unfortunaly this won't work in SeaMonkey. I don't dare just to change
> the install.rdf - would that help?
>
> (I've lost AB, but was able to reinstall, after reinstalling some filters
> disappeared).

My guess is that the following, placed where it belongs in install.rdf, ought to be enough:

      <!-- SeaMonkey version -->
        <em:targetApplication>
            <Description>
                <em:id>{92650c4d-4b8e-4d2a-b7eb-24ecf4f6b63a}</em:id>
                <em:minVersion>2.4a1</em:minVersion>
                <em:maxVersion>2.7.*</em:maxVersion>
            </Description>
        </em:targetApplication>

but Mossop knows better than me if there's no other subtle caveat.

[Justin Wood (Callek) (bugspam-callek)]

(In reply to stefan.blumenrath from comment #59)
> (In reply to Justin Scott [:fligtar] from comment #58)
> > Since it hasn't been linked here yet, in addition to the fix going into
> > Firefox, we've also created a workaround to recover any hidden add-ons.
>
> Great, unfortunaly this won't work in SeaMonkey. I don't dare just to change
> the install.rdf - would that help?

That would work, but there is also a version on people.m.o that installs into SeaMonkey just fine: https://people.mozilla.com/%7Edtownsend/shared/bug680802/hotfix.xpi

The trick is the addon has toolkit@ entry, but AMO needs a specific app entry, sadly.

[Fligtar-h (fligtar-h)]

You don't need to make any changes for that to work with SeaMonkey; simply installing, removing, or enabling/disabling any add-on and restarting Firefox will fix the issue.

[Antoine-mechelynck-gmail (antoine-mechelynck-gmail)]

Update: the version at AMO now supports SeaMonkey too.

[Launchpad Janitor (janitor)]

This bug was fixed in the package firefox - 7.0.1+build1+nobinonly-0ubuntu1

---------------
firefox (7.0.1+build1+nobinonly-0ubuntu1) oneiric; urgency=low

  * New upstream stable release (FIREFOX_7_0_1_BUILD1)
    - Fixes LP: #861664

  * Add Mongolian and Swahili to locale blacklist. These aren't meant to be
    built on the release channel, but they still appear in the upstream
    shipped-locales
    - update debian/locales.blacklist
  * Drop the temporary postinst magic for the distribution.ini folder ->
    symlink transition. The new upstream release means that the install
    location changes, so this is redundant now
    - update debian/firefox.postinst.in
 -- Chris Coulson <email address hidden> Wed, 28 Sep 2011 23:53:21 +0100

[Nicolae-cristian (nicolae-cristian)]

I've performed the following steps:

1. Install Firefox 6.0
2. Install an add-on which is compatible to Firefox 7.0 but not the latest version of the add-on (Try URL Fixer by installing Version 3.0)
3. Restart Firefox to install the add-on
4. Check for and download the update for the add-on (DO NOT INSTALL IT BY RESTARTING), it should update to 3.1
5. Exit or kill Firefox 6.0 and install Firefox 7.0.1 by replacing the install
6. Check that the add-on is not missing from the add-ons manager, and that it is active and usable.

It seems that this issue is not reproducible any more. I've verified it with both restartless (e.g., an add-on such as Cleary) and restart add-ons (e.g, Persona's), but works fine.

--
Mozilla/5.0 (Android;Linux armv7l;rv:7.0.1)Gecko/20110928
Firefox/7.0.1 Fennec/7.0.1
Device: Samsung Google Nexus S
OS: Android 2.3

[Vlad-ghetiu (vlad-ghetiu)]

Setting resolution to verified Fixed on:

Mozilla/5.0 (X11; Linux i686; rv:7.0.1) Gecko/20100101 Firefox/7.0.1
Mozilla/5.0 (X11; Linux x86_64; rv:7.0.1) Gecko/20100101 Firefox/7.0.1

Mozilla/5.0 (Windows NT 5.0; rv:7.0.1) Gecko/20100101 Firefox/7.0.1
Mozilla/5.0 (Windows NT 5.1; rv:7.0.1) Gecko/20100101 Firefox/7.0.1
Mozilla/5.0 (Windows NT 6.0; rv:7.0.1) Gecko/20100101 Firefox/7.0.1

Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:7.0.1) Gecko/20100101 Firefox/7.0.1
Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0.1) Gecko/20100101 Firefox/7.0.1

Mozilla/5.0 (Windows NT 6.1; rv:7.0.1) Gecko/20100101 Firefox/7.0.1
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:7.0.1) Gecko/20100101 Firefox/7.0.1

This bug is no longer reproducible on Firefox 7.0.1.

[ashughes (anthony-s-hughes)]

(In reply to Vlad [QA] from comment #65)
> This bug is no longer reproducible on Firefox 7.0.1.

Could you comment the steps you used to verify this (similar to how Cristian did in comment 64)? This way we have 100% visibility into the fix verification.

Thanks

[Dtownsend (dtownsend)]

I don't see it explicitly called out, did we also verify the steps in comment 48? I want to be sure we've fixing the problem for people that already saw it as well as for new updaters

[AaronMT (aaron-train)]

(In reply to Dave Townsend (:Mossop) from comment #67)
> I don't see it explicitly called out, did we also verify the steps in
> comment 48? I want to be sure we've fixing the problem for people that
> already saw it as well as for new updaters

Yes. I will update our mobile notes; 6.0 -> 7.0 -> 7.0.1 update process resolves the issue

[Launchpad Janitor (janitor)]

This bug was fixed in the package thunderbird - 7.0.1+build1+nobinonly-0ubuntu1

---------------
thunderbird (7.0.1+build1+nobinonly-0ubuntu1) oneiric; urgency=low

  * New upstream stable release (THUNDERBIRD_7_0_1_BUILD1)
    - Fixes LP: #861664
 -- Chris Coulson <email address hidden> Thu, 29 Sep 2011 16:50:36 +0100

[Hskupin (hskupin)]

Also can we get someone from the China team to verify the fix? Also for partner builds I would kinda like to see a confirmation that everything is ok. Carsten?

[Jbecerra-mozilla (jbecerra-mozilla)]

I've been trying to reproduce the problem using builds from firefox.com.cn, however I don't know where to get older versions of the bundled addons such that I can check for addon updates in order to reproduce the problem.

What I've observed is that when I do an update from 6.0.2 to the latest version, there is a dialog for addon updates, which it then downloads and the application starts up fine with no missing addons.

[A-geek (a-geek)]

Maybe the addon, and 7.0.1, only work if one does the upgrade in one go?

I recently upgraded to 7.0 and lost a number of addons, then reinstalled some from addons.mozilla.org, then, today, found that a number of addons were disabled again. Seeing that, I exited FF7 and installed the upgrade to 7.0.1 (systemwide), then restarted FF again. But my addons are still missing. Then I also installed this "addon recovery tool" addon and followed the instructions, but my addons are still hidden/not there. But addons.sqlite in the profile still lists them. It would be nice if FF would recover all the addons automatically.

This is with FF 7.0.1 (Mozilla/5.0 (X11; Linux i686; rv:7.0.1) Gecko/20100101 Firefox/7.0.1).

Imho, the bug is not fixed.

[Steffen Wilberg (steffen-wilberg)]

(In reply to a_geek from comment #71)
> I recently upgraded to 7.0 and lost a number of addons, then reinstalled
> some from addons.mozilla.org, then, today, found that a number of addons
> were disabled again.
Are they present in the Addons-Manager and marked as disabled?
Or are they hidden, not visible in the Addons-Manager?

Can you find them in your profile folder (about:support has a button to open it), in the "extensions" subfolder?

[A-geek (a-geek)]

They are not visible in the addons manager. I've reinstalled most of them in the meantime because I needed them, but they were present in the addons.sqlite file. For those that I didn't reinstall yet, they are not listed under "Extensions" on "about:support". They have also been removed from the extensions subdir of the profile, as a cursory look indicates, but are (were?) still present in addons.sqlite.

[Vlad-mozbugs (vlad-mozbugs)]

Litmus test https://litmus.mozilla.org/show_test.cgi?id=33510

[ashughes (anthony-s-hughes)]

(In reply to a_geek from comment #73)
> They are not visible in the addons manager. I've reinstalled most of them in
> the meantime because I needed them, but they were present in the
> addons.sqlite file. For those that I didn't reinstall yet, they are not
> listed under "Extensions" on "about:support". They have also been removed
> from the extensions subdir of the profile, as a cursory look indicates, but
> are (were?) still present in addons.sqlite.

I'm thinking this is a separate issue which is a fallout case from this bug. Should a new bug be filed and investigated?

[Dtownsend (dtownsend)]

(In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #75)
> (In reply to a_geek from comment #73)
> > They are not visible in the addons manager. I've reinstalled most of them in
> > the meantime because I needed them, but they were present in the
> > addons.sqlite file. For those that I didn't reinstall yet, they are not
> > listed under "Extensions" on "about:support". They have also been removed
> > from the extensions subdir of the profile, as a cursory look indicates, but
> > are (were?) still present in addons.sqlite.
>
> I'm thinking this is a separate issue which is a fallout case from this bug.
> Should a new bug be filed and investigated?

Yeah almost all the symptoms listed are different to those that this bug causes so we should get a new bug filed. I wouldn't suspect it was fallout from this though as apparently some add-ons went missing in Firefox 7.0.

[A-geek (a-geek)]

I didn't mean to confuse things. If I understand you correctly, I should submit a new bug. But please note that I currently can't reproduce the problem, as I have destroyed most of the original evidence (by running installs and upgrades). Any suggestions for components etc., please?

[Hskupin (hskupin)]

(In reply to a_geek from comment #77)
> installs and upgrades). Any suggestions for components etc., please?

Same product and component as this bug, please.

[Adamw-c (adamw-c)]

The (first) workaround link above is no use because the information has been removed from the page and replaced with an instruction to update to 7.0.1. Folks, just because you've released 7.0.1 doesn't mean your downstreams all have too. Many Linux users run Firefox from distro packages, not direct from upstream, and not all downstreams have pushed out the 7.0.1 update yet. Removing the workaround text is plain rude.

The later workaround link in comment #61 did work for me, if anyone else is looking for a workaround.

[A-geek (a-geek)]

(In reply to Henrik Skupin (:whimboo) from comment #78)
> Same product and component as this bug, please.

Done:

https://bugzilla.mozilla.org/show_bug.cgi?id=691402

[Hskupin (hskupin)]

(In reply to Adam Williamson from comment #79)
> The (first) workaround link above is no use because the information has been
> removed from the page and replaced with an instruction to update to 7.0.1.

About which workaround link you are talking? Can you please tell us the comment number or the URL directly? Thanks.

[Adamw-c (adamw-c)]

https://support.mozilla.com/en-US/kb/add-ons-hidden-after-updating-firefox-7

[Epinal99-bugzilla (epinal99-bugzilla)]

(In reply to Henrik Skupin (:whimboo) from comment #81)
> (In reply to Adam Williamson from comment #79)
> > The (first) workaround link above is no use because the information has been
> > removed from the page and replaced with an instruction to update to 7.0.1.
>
> About which workaround link you are talking? Can you please tell us the
> comment number or the URL directly? Thanks.

The workaround has been removed at the revision #17583 of the webpage:
http://support.mozilla.com/en-US/kb/add-ons-hidden-after-updating-firefox-7/compare?locale=en-US&to=17583&from=17582

[Mverdi (mverdi)]

(In reply to Adam Williamson from comment #79)
> Many Linux users run Firefox from distro packages, not direct from
> upstream, and not all downstreams have pushed out the 7.0.1 update yet.
> Removing the workaround text is plain rude.

Hi Adam,
I didn't intend to be rude by removing the workaround from the article. I was trying to simplify the instructions but obviously missed a circumstance where the workaround is still needed. I'm in the process of correcting that now. Thanks for bringing this to my attention.

[Vlad-ghetiu (vlad-ghetiu)]

I have followed the steps from comment47 and comment 48 in order to verify this on Firefox 7.0.1 (comment65).
If the target milestone set for this bug is mozilla 10, in order to verify this bug, it's enough if I update from an older nightly build to the latest one and check that no add-ons are missing or I have to follow some additional steps? (the same applies to Aurora as well).
Thanks.

[Hskupin (hskupin)]

(In reply to Vlad [QA] from comment #85)
> If the target milestone set for this bug is mozilla 10, in order to verify
> this bug, it's enough if I update from an older nightly build to the latest
> one and check that no add-ons are missing or I have to follow some
> additional steps? (the same applies to Aurora as well).

You should update from a Nightly builds with a lower version number, means install a Nightly 9.0a1 build first and update to 10.0a1 for your check.

[Vlad-ghetiu (vlad-ghetiu)]

I have verified using the following steps:

Note: Install Firefox 7.0.1 and change the update channel to "releasetest"
1. Start Firefox 7.0.1.
2. Install NoScript 2.1.4. RESTART.
3. After restart, install Ab Block Plus 1.3.10. DO NOT RESTART.
4. Go to Tools > Add-ons > Extensions. AbBlock should require to restart Firefox.DO NOT RESTART
5. In Add-ons Manager, check for updates. NoScript should update to the latest version and should require to restart Firefox. DO NOT RESTART.
6. Go to Menu > About Firefox check for updates.

Expected results:
After updating to Firefox 8 RC, both add-ons should be present in the Add-ons Manager, NoScript should be updated to the latest version (2.1.8)

The same behavior applied to Aurora and Nightly. After updating from 8.0a2 to 9.0a2 and 9.0a1 to 10.0a1, both add-ons were present in Add-ons Manager.

I have verified on:
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0) Gecko/20100101 Firefox/8.0
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0a2) Gecko/20111107 Firefox/9.0a2
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0a1) Gecko/20111107 Firefox/10.0a1

Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:8.0) Gecko/20100101 Firefox/8.0
Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:9.0a2) Gecko/20111107 Firefox/9.0a2
Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:10.0a1) Gecko/20111107 Firefox/10.0a1

Mozilla/5.0 (X11; Linux x86_64; rv:8.0) Gecko/20100101 Firefox/8.0
Mozilla/5.0 (X11; Linux x86_64; rv:9.0a2) Gecko/20111107 Firefox/9.0a2
Mozilla/5.0 (X11; Linux x86_64; rv:10.0a1) Gecko/20111107 Firefox/10.0a1

Mozilla/5.0 (Windows NT 5.1; rv:8.0) Gecko/20100101 Firefox/8.0
Mozilla/5.0 (Windows NT 5.1; rv:9.0a2) Gecko/20111107 Firefox/9.0a2
Mozilla/5.0 (Windows NT 5.1; rv:10.0a1) Gecko/20111107 Firefox/10.0a1

Considering this, setting resolution to Verified Fixed across platforms.