Firefox 67 in Ubuntu 18.10 thinks it's an older version

Bug #1830096 reported by Maia Everett on 2019-05-22
18
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Mozilla Firefox
Won't Fix
Medium
firefox (Ubuntu)
High
Olivier Tilloy

Bug Description

Today I installed the Firefox 67 update on Ubuntu 18.04, then updated to Ubuntu 18.10. After this, launching Firefox gave me a "Using an older version of Firefox" message with only two options: create a new profile or quit.

Apparently the Firefox 67 build for 18.10 thinks it's an older version than the Firefox 67 build for 18.04.

For now I worked around the issue by manually installing the Firefox 67 build for Ubuntu 18.04, which did load the existing user profile correctly.

Created attachment 9066781
compatibility.ini

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:67.0) Gecko/20100101 Firefox/67.0

Steps to reproduce:

On 2019-05-21 I updated my computer from Ubuntu 18.04 to 19.04. I'm not sure if Firefox was updated to Firefox 67 prior to my upgrade. I was not using any nightly or developer versions.

Actual results:

When loading Firefox an error window came up with the following text:
> Using an older version of Firefox can corrupt bookmarks and browsing history already saved to an existing Firefox profile. To protect your information, create a new profile for this installation of Firefox.

Expected results:

Firefox should have loaded my profile without issue.

Just a comment that I was directed here from Reddit and told to upload the compatibility.ini file from the old profile.

Maia Everett (linneris) wrote :

From this thread https://www.reddit.com/r/firefox/comments/brjj82/firefox_67_thinks_its_an_old_version_and_wont/?ref=readnext

"From looking at the compatibility.ini for the old profile vs the new one it looks like the long version number for the newer Ubuntu is slightly smaller.

18.04 profile: 67.0_20190517140819/20190517140819

19.04 profile: 67.0_20190517140251/20190517140251"

Just to check, are you using the Ubuntu provided version of Firefox or are you using one downloaded directly from Mozilla?

This was reported as an ubuntu bug: https://bugs.launchpad.net/firefox/+bug/1830096.

It seems unfortunate that firefox uses MOZ_BUILD_DATE as its build ID (if I followed correctly the code path, see https://hg.mozilla.org/mozilla-central/file/tip/build/variables.py#l15).

The official build from Mozilla might not be affected by that bug, but on Ubuntu we build firefox for each supported release (at present that's 16.04, 18.04, 18.10, 19.04 and the development series, 19.10), which means that the same package of firefox for two different Ubuntu releases will be getting different build IDs, and there's no way to guarantee that the build ID for release X will be < the build ID for release Y, where X < Y (other than building the packages sequentially, instead of in parallel).

I wonder whether setting MOZ_BUILDID to the Ubuntu package version string (e.g. "67.0+build2-0ubuntu0.19.04.1") when building would fix the problem, and wouldn't introduce undesirable side effects?

Changed in firefox (Ubuntu):
status: New → Triaged
importance: Undecided → High

(In reply to Dave Townsend [:mossop] (he/him) from comment #2)
> Just to check, are you using the Ubuntu provided version of Firefox or are you using one downloaded directly from Mozilla?

I'm using the Ubuntu provided version, both before and after upgrading.

Changed in firefox:
importance: Unknown → Medium
status: Unknown → New
Olivier Tilloy (osomon) wrote :

An effective workaround would be to rebuild the packages for all supported series sequentially, to ensure the build IDs are in ascending order.

Currently, at least for amd64, they are reversed:

$ grep -r --include="application.ini" BuildID | sort
16.04/usr/lib/firefox/application.ini:BuildID=20190517141553
18.04/usr/lib/firefox/application.ini:BuildID=20190517140819
18.10/usr/lib/firefox/application.ini:BuildID=20190517140354
19.04/usr/lib/firefox/application.ini:BuildID=20190517140251
19.10/usr/lib/firefox/application.ini:BuildID=20190517114929

Changed in firefox (Ubuntu):
assignee: nobody → Olivier Tilloy (osomon)
status: Triaged → In Progress

(In reply to Olivier Tilloy from comment #3)
> It seems unfortunate that firefox uses MOZ_BUILD_DATE as its build ID (if I followed correctly the code path, see https://hg.mozilla.org/mozilla-central/file/tip/build/variables.py#l15).

My apologies, I hadn't noticed that this was the case.

> The official build from Mozilla might not be affected by that bug, but on Ubuntu we build firefox for each supported release (at present that's 16.04, 18.04, 18.10, 19.04 and the development series, 19.10), which means that the same package of firefox for two different Ubuntu releases will be getting different build IDs, and there's no way to guarantee that the build ID for release X will be < the build ID for release Y, where X < Y (other than building the packages sequentially, instead of in parallel).

If the builds for each release are from the same source then I don't think it would be an issue for the Firefox build ID to match across those builds, though I also imagine that sharing profiles between different Ubuntu releases is going to be rare.

The chief thing that we care about is that if the Firefox code is newer then it should have a larger compatibility version than previously. The compatibility version is defined as `<version>_<appbuildid>/<platformbuildid>` (the latter two are the same in Mozilla builds). We compare as if the string is a valid toolkit version (https://developer.mozilla.org/en-US/docs/Mozilla/Toolkit_version_format), and to make that the case we do some special parsing in the case where the buildid is the expected number of characters that are generated from Mozilla builds, otherwise we just use the literal compatibility version for comparison.

I am not sure about other uses of the build ID in Firefox, but I think one viable option is just to set the build ID to your "build2". This would lead to a compatibility version like `67.0_build2/build2`. This would fall back to just interpreting that as a full version string which looks like it compares correctly:

```
"67.0_build1/build1" < "67.0_build2/build2"
"67.0_build3/build3" > "67.0_build2/build2"
"67.0_build3/build3" < "67.1_build2/build2"
"67.0.1_build1/build1" < "67.1_build2/build2"
"67.0.1_build1/build1" > "67.0_build2/build2"
```

What do you think of that?

Olivier Tilloy (osomon) wrote :

Packages are being rebuilt, sequentially.

In the meantime I have reproduced the issue as originally reported, by updating firefox to 67.0+build2-0ubuntu0.18.04.1 in a bionic VM, launching it, then dist-upgrading to cosmic and launching firefox again. I have also verified that deleting the file compatibility.ini in the profile folder "fixes" the issue.

(In reply to Dave Townsend [:mossop] (he/him) from comment #6)

> If the builds for each release are from the same source then I don't think it would be an issue for the Firefox build ID to match across those builds, though I also imagine that sharing profiles between different Ubuntu releases is going to be rare.

Indeed this wouldn't be an issue, rather a logical thing: two binary packages of the same version of firefox (sharing a single source package) have the same build ID.

> The chief thing that we care about is that if the Firefox code is newer then it should have a larger compatibility version than previously. The compatibility version is defined as `<version>_<appbuildid>/<platformbuildid>` (the latter two are the same in Mozilla builds). We compare as if the string is a valid toolkit version (https://developer.mozilla.org/en-US/docs/Mozilla/Toolkit_version_format), and to make that the case we do some special parsing in the case where the buildid is the expected number of characters that are generated from Mozilla builds, otherwise we just use the literal compatibility version for comparison.
>
> I am not sure about other uses of the build ID in Firefox, but I think one viable option is just to set the build ID to your "build2". This would lead to a compatibility version like `67.0_build2/build2`.
>
> […]
>
> What do you think of that?

This makes sense to me, and sounds like it would work. Note that "build2" in that context comes from the upstream build candidate, it's not an Ubuntu-specific thing: https://ftp.mozilla.org/pub/firefox/candidates/67.0-candidates/.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package firefox - 67.0+build2-0ubuntu2

---------------
firefox (67.0+build2-0ubuntu2) eoan; urgency=medium

  * No-change rebuild to fix the BuildID ordering between the same version of
    firefox on all supported Ubuntu releases (LP: #1830096)

 -- Olivier Tilloy <email address hidden> Thu, 23 May 2019 13:00:20 +0200

Changed in firefox (Ubuntu):
status: In Progress → Fix Released
Olivier Tilloy (osomon) wrote :

Reverting the status to "Fix Committed" because the updates to bionic, cosmic and disco haven't been released yet.

Changed in firefox (Ubuntu):
status: Fix Released → Fix Committed

I am testing this suggestion, and it appears to work as expected.
Would the following patch be acceptable?

    --- a/build/variables.py
    +++ b/build/variables.py
    @@ -13,10 +13,12 @@ SOURCESTAMP_FILENAME = 'sourcestamp.txt'

     def buildid_header(output):
    - buildid = os.environ.get('MOZ_BUILD_DATE')
    - if buildid and len(buildid) != 14:
    - print('Ignoring invalid MOZ_BUILD_DATE: %s' % buildid, file=sys.stderr)
    - buildid = None
    + buildid = os.environ.get('MOZ_BUILDID')
    + if buildid is None:
    + buildid = os.environ.get('MOZ_BUILD_DATE')
    + if buildid and len(buildid) != 14:
    + print('Ignoring invalid MOZ_BUILD_DATE: %s' % buildid, file=sys.stderr)
    + buildid = None
         if not buildid:
             buildid = datetime.now().strftime('%Y%m%d%H%M%S')
         output.write("#define MOZ_BUILDID %s\n" % buildid)

I.e. prioritize the `MOZ_BUILDID` environment variable over `MOZ_BUILD_DATE`, if it is set.

(In reply to Olivier Tilloy from comment #8)
> I am testing this suggestion, and it appears to work as expected.

Ok, so the patch looks ok to me, though it would really need a build config peer to look at it. But we've discovered another issue related to how we compare the compatibility versions in bug 1554029 so we are going to be making changes to that. I suspect that what we change will still work for the suggestion above but while I realise this is an important issue for you it might be worth holding off any changes on your end until there is an accepted patch there. I'm hoping to have something written today.

Fair enough, I'll hold off changes until bug 1554029 is fixed. Thanks Dave!

Ok so it looks like bug 1554029 has landed and stuck and will be rolling out in a 67.0.1 soon. During testing of that I found that the suggestion I made in comment 6 would have actually only worked on some platforms due to a bug in the version comparator code. It probably would have worked on Linux but now the code is fixed it that changes things a little.

If you care about your users being able to upgrade from a version with the old-style build ID (14 digits based on the date) then either you need to:

1. Continue to use a purely numeric build ID that is larger each time you build (and the first must be larger than the last build ID your users have).

2. Switch to a build ID that can be parsed by the toolkit version code (https://developer.mozilla.org/en-US/docs/Mozilla/Toolkit_version_format) and must contain a character that is not a digit in it (or Firefox will just parse it as a number) and do the above.

For the second option in the first upgrade where users are switching from the old build ID to your new one the version comparator will consider the old build ID to be "0" because the number is too large and overflows. So for your new build ID to be larger than that then it has to be a valid version considered larger than or equal to 0. Mostly that means it must start with at least one digit so "1build1" would work (and then after "1build2" etc. would be considered larger.

It's a little uglier than I'd like unfortunately but it should work.

Seth Arnold (seth-arnold) wrote :

I saw a comment somewhere that suggested this was "rare".

I think this is actually pretty common -- it feels like this has been discussed more than any other single issue on irc lately.

The consequence for most users is losing their entire Firefox history, settings, addons, saved passwords, sessions, etc.

Can we please push the time-delayed-rebuilds, if not the actual fixes, sooner rather than later?

Thanks

Mason Loring Bliss (y-mason) wrote :

For what it's worth, my kids encountered this on two desktops today, moving from 16.04 to 18.04. As this would be a major issue for non-technical folk I'd urge getting the fix out the door sooner rather than later - especially on 18.04, to preserve LTS as being a safe choice.

The workaround I used here, prior to being shown this bug, was to let Firefox make a new profile and to follow these instructions to copy over all the critical files:

    https://support.mozilla.org/en-US/kb/recovering-important-data-from-an-old-profile

Jonathan Johnsson (jjohnsson) wrote :

I just upgraded from 18.10 to 19.04, since it looked like a good time given the release cycle, and I have this problem.

How do I "manually installing the Firefox 67 build for Ubuntu 18.04" (as mentioned in the description)? In my case I guess it would be "Firefox 67 build for Ubuntu 18.10".

Jonathan Johnsson (jjohnsson) wrote :

For those who come here and can't wait until this fix is released (thanks for your hard work, people fixing this!) you can find workarounds here (I can't say if the workarounds are safe or not):

https://old.reddit.com/r/firefox/comments/brjj82/firefox_67_thinks_its_an_old_version_and_wont/

Running
MOZ_ALLOW_DOWNGRADE=1 firefox --ProfileManager
from bash worked for me, since I accidentally created a new profile when exploring my options, and had to switch back to my original profile. I now have access to my session, bookmarks and containers again.

*** Bug 1557014 has been marked as a duplicate of this bug. ***

Based on this needing an ubuntu change at this point I'm going to close this out.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package firefox - 67.0.1+build1-0ubuntu0.18.10.1

---------------
firefox (67.0.1+build1-0ubuntu0.18.10.1) cosmic; urgency=medium

  * New upstream stable release (67.0.1build1)

 -- Olivier Tilloy <email address hidden> Thu, 30 May 2019 11:04:58 +0200

Changed in firefox (Ubuntu):
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package firefox - 67.0.1+build1-0ubuntu0.18.04.1

---------------
firefox (67.0.1+build1-0ubuntu0.18.04.1) bionic; urgency=medium

  * New upstream stable release (67.0.1build1)

 -- Olivier Tilloy <email address hidden> Thu, 30 May 2019 07:39:22 +0200

Changed in firefox (Ubuntu):
status: Fix Committed → Fix Released
Changed in firefox:
status: New → Won't Fix

*** Bug 1555986 has been marked as a duplicate of this bug. ***

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.