do-release-upgrade fails 19.10->20.04 AttributeError: 'UbuntuDistroInfo' object has no attribute 'version'

[Nigel Jones]

➜ ~ sudo do-release-upgrade -d
Checking for a new Ubuntu release
Get:1 Upgrade tool signature [1554 B]
Get:2 Upgrade tool [1344 kB]
Fetched 1346 kB in 0s (0 B/s)
authenticate 'focal.tar.gz' against 'focal.tar.gz.gpg'
extracting 'focal.tar.gz'
                                                                                                             81,25 Bot
Building dependency tree
Reading state information... Done

Traceback (most recent call last):
  File "/tmp/ubuntu-release-upgrader-dtwszzo7/DistUpgrade/DistUpgradeQuirks.py", line 186, in _get_from_and_to_version
    di.version('%s' % self.controller.fromDist).split()[0]
AttributeError: 'UbuntuDistroInfo' object has no attribute 'version'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/ubuntu-release-upgrader-dtwszzo7/focal", line 8, in <module>
    sys.exit(main())
  File "/tmp/ubuntu-release-upgrader-dtwszzo7/DistUpgrade/DistUpgradeMain.py", line 238, in main
    if app.run():
  File "/tmp/ubuntu-release-upgrader-dtwszzo7/DistUpgrade/DistUpgradeController.py", line 2082, in run
    return self.fullUpgrade()
  File "/tmp/ubuntu-release-upgrader-dtwszzo7/DistUpgrade/DistUpgradeController.py", line 1919, in fullUpgrade
    if not self.doPostInitialUpdate():
# cleanup[1] sre_constants
# cleanup[1] re
# cleanup[1] _codecs
# cleanup[1] _warnings
# cleanup[1] zipimport
# cleanup[1] _sysconfigdata
# cleanup[1] encodings.utf_8
# cleanup[1] codecs
# cleanup[1] readline
# cleanup[1] _sysconfigdata_nd
# cleanup[1] _locale
# cleanup[1] sitecustomize
# cleanup[1] signal
# cleanup[1] traceback
# cleanup[1] posix
# cleanup[1] encodings.aliases
# cleanup[1] exceptions
# cleanup[1] _weakref
# cleanup[1] sre_compile
# cleanup[1] _sre
# cleanup[1] sre_parse
# cleanup[2] copy_reg
# cleanup[2] posixpath
# cleanup[2] errno
# cleanup[2] _abcoll
# cleanup[2] types
# cleanup[2] genericpath
# cleanup[2] stat
# cleanup[2] warnings
# cleanup[2] UserDict
# cleanup[2] os.path
# cleanup[2] linecache
# cleanup[2] os
# cleanup sys
# cleanup __builtin__
# cleanup ints: 19 unfreed ints
# cleanup floats
➜ ~ type python
python is /usr/bin/python
➜ ~ sudo do-release-upgrade -d
Checking for a new Ubuntu release
Get:1 Upgrade tool signature [1554 B]
Get:2 Upgrade tool [1344 kB]
Fetched 1346 kB in 0s (0 B/s)
authenticate 'focal.tar.gz' against 'focal.tar.gz.gpg'
extracting 'focal.tar.gz'

ProblemType: Bug
DistroRelease: Ubuntu 19.10
Package: ubuntu-release-upgrader-core 1:19.10.15.4
ProcVersionSignature: Ubuntu 5.3.0-46.38-generic 5.3.18
Uname: Linux 5.3.0-46-generic x86_64
ApportVersion: 2.20.11-0ubuntu8.8
Architecture: amd64
CrashDB: ubuntu
Date: Wed Apr 22 12:02:00 2020
PackageArchitecture: all
SourcePackage: ubuntu-release-upgrader
UpgradeStatus: Upgraded to eoan on 2020-04-22 (0 days ago)
VarLogDistupgradeAptclonesystemstate.tar.gz:
 Error: command ['pkexec', 'cat', '/var/log/dist-upgrade/apt-clone_system_state.tar.gz'] failed with exit code 127: polkit-agent-helper-1: error response to PolicyKit daemon: GDBus.Error:org.freedesktop.PolicyKit1.Error.Failed: No session for cookie
 Error executing command as another user: Not authorized

 This incident has been reported.
VarLogDistupgradeAptlog:
 Log time: 2020-04-22 11:57:30.093261
 Log time: 2020-04-22 11:57:32.463320
mtime.conffile..etc.update-manager.release-upgrades: 2019-03-06T13:00:00.153336

[Brian Murray (brian-murray)]

Can you try this in a terminal and let us know the output?

python3 -c "import distro_info; print(distro_info.UbuntuDistroInfo().version('eoan'))"

[David S (daveman1010220)]

I ran your command:

python3 -c "import distro_info; print(distro_info.UbuntuDistroInfo().version('eoan'))" 35.7s  Thu 23 Apr 2020 09:38:42 PM EDT
Traceback (most recent call last):
  File "<string>", line 1, in <module>
AttributeError: 'UbuntuDistroInfo' object has no attribute 'version'

Basically, same error the installer gives me.

[David S (daveman1010220)]

Sorry about the extra timestamp in there.

[David S (daveman1010220)]

update-manager -c -d 39.6s  Thu 23 Apr 2020 09:33:17 PM EDT
WARNING:root:estimate_kernel_initrd_size_in_boot() returned '0' for kernel?
WARNING:root:estimate_kernel_initrd_size_in_boot() returned '0' for initrd?
Checking for a new Ubuntu release
Checking for a new Ubuntu release
/usr/lib/python3/dist-packages/DistUpgrade/DistUpgradeFetcher.py:23: PyGIWarning: Gtk was imported without specifying a version first. Use gi.require_version('Gtk', '3.0') before import to ensure that the right version gets loaded.
  from gi.repository import Gtk, Gdk
/usr/lib/python3/dist-packages/DistUpgrade/ReleaseNotesViewerWebkit.py:33: PyGIWarning: WebKit2 was imported without specifying a version first. Use gi.require_version('WebKit2', '4.0') before import to ensure that the right version gets loaded.
  from gi.repository import WebKit2 as WebKit
authenticate 'focal.tar.gz' against 'focal.tar.gz.gpg'
extracting 'focal.tar.gz'
Traceback (most recent call last):
  File "/tmp/ubuntu-release-upgrader-96b2bn8c/DistUpgrade/DistUpgradeQuirks.py", line 186, in _get_from_and_to_version
    di.version('%s' % self.controller.fromDist).split()[0]
AttributeError: 'UbuntuDistroInfo' object has no attribute 'version'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/ubuntu-release-upgrader-96b2bn8c/focal", line 8, in <module>
    sys.exit(main())
  File "/tmp/ubuntu-release-upgrader-96b2bn8c/DistUpgrade/DistUpgradeMain.py", line 238, in main
    if app.run():
  File "/tmp/ubuntu-release-upgrader-96b2bn8c/DistUpgrade/DistUpgradeController.py", line 2082, in run
    return self.fullUpgrade()
  File "/tmp/ubuntu-release-upgrader-96b2bn8c/DistUpgrade/DistUpgradeController.py", line 1919, in fullUpgrade
    if not self.doPostInitialUpdate():
  File "/tmp/ubuntu-release-upgrader-96b2bn8c/DistUpgrade/DistUpgradeController.py", line 925, in doPostInitialUpdate
    self.quirks.run("PostInitialUpdate")
  File "/tmp/ubuntu-release-upgrader-96b2bn8c/DistUpgrade/DistUpgradeQuirks.py", line 97, in run
    func()
  File "/tmp/ubuntu-release-upgrader-96b2bn8c/DistUpgrade/DistUpgradeQuirks.py", line 116, in focalPostInitialUpdate
    self._get_from_and_to_version()
  File "/tmp/ubuntu-release-upgrader-96b2bn8c/DistUpgrade/DistUpgradeQuirks.py", line 192, in _get_from_and_to_version
    (r.version for r in di.get_all("object")
AttributeError: 'UbuntuDistroInfo' object has no attribute 'get_all'
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/problem_report.py", line 477, in add_to_existing
    self.write(f)
  File "/usr/lib/python3/dist-packages/problem_report.py", line 430, in write
    block = f.read(1048576)
  File "/usr/lib/python3.7/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte

Original exception was:
Traceback (most recent call last):
  File "/tmp/ubuntu-release-upgrader-96b2bn8c/DistUpgrade/DistUpgradeQuirks.py", line 186, in _get_from...

[David S (daveman1010220)]

Ok, I've done some debugging, and it turns out the error is exactly right (surprise). There is no 'version' attribute on the UbuntuDistroInfo class. I made one. Now it wants a 'get_all' attribute, which is also not there. Is this file just out-of-date? Should I update it somehow?

[David S (daveman1010220)]

https://bugs.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/+bug/1848829

[David S (daveman1010220)]

This fixed the installer issue. ^^^

[Fork (nicolas-diener)]

This seems to typically stem from a pip-installed version of distro-info shadowing the apt version, like in the issue David S linked.
I'm not sure this can be prevented by the updater.

If you're here looking for a quick fix:

python3 -m pip uninstall distro-info
sudo apt reinstall python3-distro-info

[Jaroslaw Smorczewski (smoczyna)]

This bug affects all ubuntu version since I think 18.04
The problem is that upgrader tries to use UbuntuDistroInfo object (child of DistroInfo) in unsupported way. I believe it is upgrader issue.
At least one class of the upgrader, namely: DistUpgradeQuirks tries to call version field and get_all method of DistroInfo object and there are neither of them there.

The reason I think it's upgrader issue is this code I found in above class, this is the source of te installer failure.

def _get_from_and_to_version(self):
        di = distro_info.UbuntuDistroInfo()
        try:
            self._from_version = \
                di.version('%s' % self.controller.fromDist).split()[0]
            self._to_version = \
                di.version('%s' % self.controller.toDist).split()[0]
        # Ubuntu 18.04's python3-distro-info does not have version
        except AttributeError:
            self._from_version = next(
                (r.version for r in di.get_all("object")
                 if r.series == self.controller.fromDist),
                self.controller.fromDist).split()[0]
            self._to_version = next(
                (r.version for r in di.get_all("object")
                 if r.series == self.controller.toDist),
                self.controller.toDist).split()[0]

[Jaroslaw Smorczewski (smoczyna)]

in my case reinstalling distro-info doesn't help I compared the code of both packages and spoken class DistroInfo is exactly the same.
I think that upgrade scripts are outdated, not distro-info package.
In above snippet there is a comment saying:
  # Ubuntu 18.04's python3-distro-info does not have version
however the workaround provide for that is also wrong because Ubuntu 18.04's do not have also get_all()

[Brian Murray (brian-murray)]

bdmurray@clean-bionic-amd64:~$ apt-cache policy python3-distro-info
python3-distro-info:
  Installed: 0.23ubuntu1
  Candidate: 0.23ubuntu1
  Version table:
 *** 0.23ubuntu1 500
        500 http://192.168.10.7/ubuntu focal/main amd64 Packages
        500 http://192.168.10.7/ubuntu focal/main i386 Packages
        100 /var/lib/dpkg/status
bdmurray@clean-bionic-amd64:~$ ipython3
Python 3.8.5 (default, Jul 28 2020, 12:59:40)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.13.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import distro_info

In [2]: di = distro_info.UbuntuDistroInfo()

In [3]: [r.version for r in di.get_all("object")]
Out[3]:
['4.10',
 '5.04',
 '5.10',
 '6.06 LTS',
 '6.10',
 '7.04',
 '7.10',
 '8.04 LTS',
 '8.10',
 '9.04',
 '9.10',
 '10.04 LTS',
...
]

As you can see this works fine for me. What version of python3-distro-info do you have installed?

[Jaroslaw Smorczewski (smoczyna)]

Well, it doesn't for many others. As I said I compared all the disto-versions of ubuntu 18.04 and none f them has version or get_all() method.
In your case it seems to be a left over from older systems you upgraded from or so. Since 18.04 distro-info doesn't have version or get_all() methods and upgrade scripts are outdated.

[Jaroslaw Smorczewski (smoczyna)]

it would be a greater help if someone could attach DistroInfo object code here. The of which works of course, the fix of this issue is quite obvious but it require help from those who have no issue.

[Jaroslaw Smorczewski (smoczyna)]

to be fair with Brian, here is my output and no version can provide that you have

>>> import distro_info
>>> di = distro_info.UbuntuDistroInfo()
>>> [r.version for r in di.get_all("object")]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'UbuntuDistroInfo' object has no attribute 'get_all'

[Jaroslaw Smorczewski (smoczyna)]

instead of proving that everything is fine here (it is not), please paste you DistroInfo class code.

[Brian Murray (brian-murray)]

I'd be really helpful to know the version of python3-distro-info you have installed. You can check with 'apt list python3-distro-info'. Everything is fine with the official version of the package distro-info provided in the Ubuntu archive. Here's some information from the package's changelog.

distro-info (0.15) unstable; urgency=medium

  [ Mike Miller ]
  * debian-distro-info: Support oldstable with --alias. (Closes: #784284)

  [ Benjamin Drung ]
  * Fix flake8 and pylint issues found in Python code
  * Run flake8 and pylint checks on Python code in test suite
  * Python: Add DistroInfo.get_all() method
  * Bump Standard-Version to 3.9.8 (no changes required)
  * Use https for Vcs-Git and Vcs-Browser URLs

 -- Benjamin Drung <email address hidden> Fri, 28 Apr 2017 19:45:36 +0200

Ubuntu 18.04 includes package version 0.18ubuntu0.18.04.1 which does provide get_all().

Perhaps you have distro-info installed via pip?

bdmurray@clean-bionic-i386:~$ pip3 install distro-info
Collecting distro-info
  Downloading https://files.pythonhosted.org/packages/91/e0/ccd731d32238bc18bb8ac80353b4028394f80d5ff7db0f93a5ba488690d3/distro-info-0.10.tar
.gz
Building wheels for collected packages: distro-info
  Running setup.py bdist_wheel for distro-info ... done
  Stored in directory: /home/bdmurray/.cache/pip/wheels/30/c8/a5/895d8664df501a2d5192447e50fca0a9ef10fcb279c4518edc
Successfully built distro-info
Installing collected packages: distro-info
Successfully installed distro-info-0.0.0
bdmurray@clean-bionic-i386:~$ ipython3
Python 3.6.9 (default, Oct 8 2020, 12:12:24)
Type "copyright", "credits" or "license" for more information.

IPython 5.5.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]: import distro_info

In [2]: di = distro_info.UbuntuDistroInfo()

In [3]: [r.version for r in di.get_all("object")]
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-3-e9682ad5c584> in <module>()
----> 1 [r.version for r in di.get_all("object")]

AttributeError: 'UbuntuDistroInfo' object has no attribute 'get_all'

[Jaroslaw Smorczewski (smoczyna)]

well, I can't say exactly, that's the thing.
When I call Muon it shows me: python-distro-info to be 018ubuntu18.04.1, size 53kb
when I call pip3 it shows me: distro-info to be 0.0.0 - this package cannot be uninstalled becuase if is a dependency for many application.
However the code inside is the same and there is no field version and get_all() method.

[Jaroslaw Smorczewski (smoczyna)]

updating these packages or update manager do nothing. I mean no changes in the system.

[Jaroslaw Smorczewski (smoczyna)]

All right, cosidering Brian feedback I found the way to override the issue.

This is my modification to distro-info.py file sitting in:
 /usr/local/lib/python3.6/dist-packages/

I had to add 2 methods to get going, all the rest of the code is the same.

class DistroRecord(object):
    def __init__(self, row):
        self.version = row['version']
        self.codename = row['codename']
        self.series = row['series']
        self.created = row['created']
        self.release = row['release']
        self.eol = row['eol']
        # self.eol_server = row['eol-server'] if not None else None
        # self.eol_esm = row['eol-esm'] if not None else None

class DistroInfo(object):
    """Base class for distribution information.
    Use DebianDistroInfo or UbuntuDistroInfo instead of using this directly.
    """

    def __init__(self, distro):
        self._distro = distro
        filename = os.path.join(_get_data_dir(), distro.lower() + ".csv")
        csvfile = open(filename)
        csv_reader = csv.DictReader(csvfile)
        self._rows = []
        self._date = datetime.date.today()
        self.versions = []
        for row in csv_reader:
            for column in ("created", "release", "eol", "eol-server"):
                if column in row:
                    row[column] = convert_date(row[column])
            self._rows.append(row)
            self.versions.append(DistroRecord(row))

    @property
    def all(self, result="codename"):
        """List all known distributions."""
        return [self._format(result, x) for x in self._rows]

    def get_all(self, mode):
        if mode=='object':
            return self.versions

    def version(self, codename):
        return next((x.version for x in self.versions if x.series == codename), None)

...
all the same after this

with those changes distro upgrade gets going, so far.

[Jaroslaw Smorczewski (smoczyna)]

I can confirm that the system upgrade went all good and I have Ubuntu 20.04 LTS now with new KDE GUI, everything works fine and I have no issues so far.
For those having the same issue, my hack works.

[Jaroslaw Smorczewski (smoczyna)]

After upgrade my distro-info came to "normal" and it sports version and get_all methods indeed. They are slightly different that my mod but effect is the same.

[Pedro Miguel Baptista Machado (pedrmachado)]

Just run:
sudo python3 -m pip uninstall distro-info
sudo apt reinstall python3-distro-info