snappy removal of dpkg-query breaks lsb_release --all

[Ryan Harper]

root@localhost:~# lsb_release --all
Traceback (most recent call last):
  File "/usr/bin/lsb_release", line 95, in <module>
    main()
  File "/usr/bin/lsb_release", line 62, in main
    verinfo = lsb_release.check_modules_installed()
  File "/usr/lib/python3/dist-packages/lsb_release.py", line 136, in check_modules_installed
    close_fds=True).communicate()[0].decode('utf-8')
  File "/usr/lib/python3.5/subprocess.py", line 947, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.5/subprocess.py", line 1551, in _execute_child
    raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: 'dpkg-query'

cloud-init collects distro information via lsb_release --all

[Zygmunt Krynicki (zyga)]

Hmm

Could we switch to /etc/os-release instead?

[Oliver Grawert (ogra)]

yes please, use /etc/os-release if possible

[Gustavo Niemeyer (niemeyer)]

The problem here is likely the fact /etc/lsb-release is still pretending that this is a stock Ubuntu 16.04 system. We need to have Ubuntu Core in there, and then ensure it's not calling dpkg-query.

[Ryan Harper (raharper)]

The lsb_release --all triggers a lsb_release.distroinfo query

/usr/lib/python2.7/dist-packages/lsb_release.py

is owned by the python-2.7 package.

lsb_release.info invokes a dpkg-query to learn about what modules are
installed

# Find which LSB modules are installed on this system
    C_env = os.environ.copy(); C_env['LC_ALL'] = 'C'
    output = subprocess.Popen(['dpkg-query','-f',"${Version}
${Provides}\n",'-W'] + PACKAGES.split(),
                              env=C_env,
                              stdout=subprocess.PIPE,
                              stderr=subprocess.PIPE,

close_fds=True).communicate()[0].decode('utf-8')

On Thu, Sep 1, 2016 at 1:59 PM, Gustavo Niemeyer <email address hidden>
wrote:

> The problem here is likely the fact /etc/lsb-release is still pretending
> that this is a stock Ubuntu 16.04 system. We need to have Ubuntu Core in
> there, and then ensure it's not calling dpkg-query.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1619420
>
> Title:
> snappy removal of dpkg-query breaks lsb_release --all
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/cloud-init/+bug/1619420/+subscriptions
>

[Ryan Harper (raharper)]

root@localhost:~# cat /etc/issue
Ubuntu 16.04.1 LTS \n \l

root@localhost:~# cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.1 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.1 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
UBUNTU_CODENAME=xenial

Both those files say noting of Ubuntu-Core 16. Where does the image declare that is? In cloud-init, the 'system_is_snappy' checks for:

content = load_file("/etc/system-image/channel.ini", quiet=True)
    if 'ubuntu-core' in content.lower():
       return True
    if os.path.isdir("/etc/system-image/config.d/"):
        return True

On this Ubuntu Core 16 image (built from ubuntu-device-flash from edge),
/etc/system-image/config.d exists, but no channel.ini

[Gustavo Niemeyer (niemeyer)]

There is no dpkg and no deb information in Ubuntu Core, so dpkg-query doesn't exist and would not work even if it existed. lsb_release needs to be tweaked, or assumed to not work on Ubuntu Core.

[Ryan Harper (raharper)]

Might we not give a hint to lsb_release via info in the os-release or other
places?

On Thu, Sep 1, 2016 at 2:18 PM, Gustavo Niemeyer <email address hidden>
wrote:

> There is no dpkg and no deb information in Ubuntu Core, so dpkg-query
> doesn't exist and would not work even if it existed. lsb_release needs
> to be tweaked, or assumed to not work on Ubuntu Core.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1619420
>
> Title:
> snappy removal of dpkg-query breaks lsb_release --all
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/cloud-init/+bug/1619420/+subscriptions
>

[Oliver Grawert (ogra)]

we can surely add an os-release entry (i think there was even an older bug in the 15-04 timeframe for this that got never fixed) i'll dig that up tomorrow during my workday

[Zygmunt Krynicki (zyga)]

Fedora uses ID_VARIANT= and VARIANT for this AFAIK as can be seen here: https://github.com/zyga/os-release-zoo/blob/master/fedora/fedora-24-workstation.txt#L15

It would be good to use ID_VARIANT=classic, ID_VARIANT=core and appropriate VARIANT=... values to match.

[Oliver Grawert (ogra)]

bug 1481086 is the old bug i mentioned and bug 1520154 (leftover from our quest to get rid of python on the image) is also related

[Oliver Grawert (ogra)]

the lsb_release binary was now removed from the core snap.

[Robert Liu (robertliu)]

Hi @Ogra,

I cannot get the correct result on a all-snap system. Is this expected?

$ lsb_release
Traceback (most recent call last):
  File "/usr/bin/lsb_release", line 95, in <module>
    main()
  File "/usr/bin/lsb_release", line 59, in main
    distinfo = lsb_release.get_distro_information()
  File "/usr/lib/python3/dist-packages/lsb_release.py", line 383, in get_distro_information
    distinfo = guess_debian_release()
  File "/usr/lib/python3/dist-packages/lsb_release.py", line 281, in guess_debian_release
    get_distro_info(distinfo['ID'])
  File "/usr/lib/python3/dist-packages/lsb_release.py", line 41, in get_distro_info
    RELEASES_ORDER.sort(key=lambda n: float(n[0]))
  File "/usr/lib/python3/dist-packages/lsb_release.py", line 41, in <lambda>
    RELEASES_ORDER.sort(key=lambda n: float(n[0]))
ValueError: could not convert string to float: '8.04 LTS'
$ snap info core
name: core
summary: "snapd runtime environment"
publisher: canonical
description: |
  The core runtime environment for snapd
type: core
tracking: edge
installed: 16.04.1 (1251) 79MB -
refreshed: 2017-02-20 16:05:56 +0000 UTC
channels:
  stable: 16.04.1 (1240) 79MB -
  candidate: 16.04.1 (1240) 79MB -
  beta: 16.04.1 (1240) 79MB -
  edge: 16.04.1 (1251) 79MB -
$ cat /etc/os-release
NAME="Ubuntu Core"
VERSION="16"
ID=ubuntu-core
PRETTY_NAME="Ubuntu Core 16"
VERSION_ID="16"
HOME_URL="http://www.snapcraft.io/"
BUG_REPORT_URL="http://bugs.launchpad.net/snappy/"
$ cat /etc/issue
Ubuntu Core 16 on \4 (\l)

[Oliver Grawert (ogra)]

@robertliu, lsb_release should be completely gone from the core snap now ... i just discovered a minor typo in the removal code, thanks for reporting this.

please use /etc/os-release for checking for any distro data.

[Facundo Batista (facundo)]

Got here by trying to understand why "pip" exploded when tried to call "lsb_release" when being inside a snap (yes, no dpkg-query").

IIUC, "pip" survives to lsb_release not existing, so the solution for this bug is an improvement in that front too.

[Adam Conrad (adconrad)]

Why on earth are you doing "rm -f usr/bin/lsb_release" when you could be purging the "lsb-release" package entirely (which is more than just the stub binary).

Also, it would be nice if all these livecd-rootfs hacks actually happened on a branch that gets some review, rather than in PPA that no one looks at.

[Oliver Grawert (ogra)]

@adam: because purging tears out stuff we want to keep ...
also the hooks code went to https://github.com/snapcore/core/tree/master/live-build/hooks now and gets cross reviews, i'll SRU the remaining livecd-rootfs bits shortly so we dont need teh PPA version anymore

[Oliver Grawert (ogra)]

This has been fixed long ago, actually following the advice from Adam above to properly purge lsb-release from the images (in live-build/hooks/600-no-debian.binary).

[James Falcon (falcojr)]

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/2716