[MIR] python3-vitrageclient

[Michal Nasiadka]

== original bug report ==
Kolla built containers with heat-engine fail with following logs:
https://f20bec36e3f198a9ccf9-fa78c758d05065ae4af77013f9d746d5.ssl.cf2.rackcdn.com/749469/2/check/kolla-ansible-ubuntu-binary/059e66f/primary/logs/kolla/heat/heat-engine.txt

2020-09-02 10:50:52.676 6 CRITICAL heat-engine [-] Unhandled error: AttributeError: 'EntryPoint' object has no attribute 'module'
2020-09-02 10:50:52.676 6 ERROR heat-engine Traceback (most recent call last):
2020-09-02 10:50:52.676 6 ERROR heat-engine File "/usr/lib/python3/dist-packages/stevedore/extension.py", line 221, in _load_plugins
2020-09-02 10:50:52.676 6 ERROR heat-engine ext = self._load_one_plugin(ep,
2020-09-02 10:50:52.676 6 ERROR heat-engine File "/usr/lib/python3/dist-packages/stevedore/enabled.py", line 77, in _load_one_plugin
2020-09-02 10:50:52.676 6 ERROR heat-engine ext = super(EnabledExtensionManager, self)._load_one_plugin(
2020-09-02 10:50:52.676 6 ERROR heat-engine File "/usr/lib/python3/dist-packages/stevedore/extension.py", line 255, in _load_one_plugin
2020-09-02 10:50:52.676 6 ERROR heat-engine plugin = ep.load()
2020-09-02 10:50:52.676 6 ERROR heat-engine File "/usr/lib/python3.8/importlib/metadata.py", line 77, in load
2020-09-02 10:50:52.676 6 ERROR heat-engine module = import_module(match.group('module'))
2020-09-02 10:50:52.676 6 ERROR heat-engine File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
2020-09-02 10:50:52.676 6 ERROR heat-engine return _bootstrap._gcd_import(name[level:], package, level)
2020-09-02 10:50:52.676 6 ERROR heat-engine File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
2020-09-02 10:50:52.676 6 ERROR heat-engine File "<frozen importlib._bootstrap>", line 991, in _find_and_load
2020-09-02 10:50:52.676 6 ERROR heat-engine File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
2020-09-02 10:50:52.676 6 ERROR heat-engine File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
2020-09-02 10:50:52.676 6 ERROR heat-engine File "<frozen importlib._bootstrap_external>", line 783, in exec_module
2020-09-02 10:50:52.676 6 ERROR heat-engine File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
2020-09-02 10:50:52.676 6 ERROR heat-engine File "/usr/lib/python3/dist-packages/heat/engine/clients/os/vitrage.py", line 16, in <module>
2020-09-02 10:50:52.676 6 ERROR heat-engine from vitrageclient import client as vitrage_client
2020-09-02 10:50:52.676 6 ERROR heat-engine ModuleNotFoundError: No module named 'vitrageclient'

Seems the culprit is missing python-vitrageclient package, which is part of heat requirements.txt - but not installed as a dependency when installing heat-engine from UCA (https://github.com/openstack/kolla/blob/master/docker/heat/heat-engine/Dockerfile.j2)

==================================================================

[Impact]

heat depends on stevedore to load client entry points. In the past,
stevedore would just give a warning if a client failed to load (ie. due
to an uninstalled Suggests dependency). Then stevedore changed code
to use importlib.metadata which no longer supports ep.load(require=False), and now the failure to load a client entry point will error hard along the lines of that described above (ModuleNotFoundError: No module named 'vitrageclient'). In other words, all openstack clients that heat depends on must be installed or heat will fail hard. By getting python-vitrageclient into Ubuntu main, we can move it from Suggests to Depends and it will always be installed for heat binary packages.

[Testcase]

Taken from: https://bugs.launchpad.net/bugs/1958988 (Thanks to Francois Michaut)

Installing heat with the following commands :

add-apt-repository cloud-archive:xena
apt update
apt-get install heat-api heat-api-cfn heat-engine

will reproduce the issue.
heat-engine service will crash and restart continuously, all heat-related commands will fail ("openstack stack list" will output (after a very long time): ERROR: Internal Error).

In heat-engine logs, we can see that a module named "vitrageclient" is missing. Running "pip3 install python-vitrageclient" fixed the issue and heat started working correctly.

==================================================================

[MIR] python3-vitrageclient

[Availability]
Currently in universe

[Rationale]
Heat is an OpenStack project that we're ready to support in main, and it has a hard dependency on python-vitrageclient.

[Security]
No security history

[Quality Assurance]
Package works out of the box with no prompting. There are no major bugs in Ubuntu and there are no major bugs in Debian. Unit tests are run during build.

[Dependencies]
All are in main. Note: libjs-bootstrap is only needed for the python-vitrageclient-doc binary package which does not need to be in main.

[Standards Compliance]
FHS and Debian Policy compliant

[Maintenance]
Simple python package that the OpenStack Team will take care of

[Background]
This is a client library for OpenStack Vitrage built to interface with the Vitrage API. Vitrage is the OpenStack RCA (Root Cause Analysis) service for organizing, analyzing and expanding OpenStack alarms & events, yielding insights regarding the root cause of problems and deducing their existence before they are directly detected.

[Chris MacNaughton (chris.macnaughton)]

vitrageclient is currently in the Recommends section of python3-heat. As vitrageclient is an optional dependency (you only need it if you have heat and vitrage), this is fairly reasonable and can be resolved by doing something like: sudo apt-get --install-suggests install python3-heat

[Aurelien Lourot (aurelien-lourot)]

Re-opening as python3-vitrageclient should be declared as a hard dependency and not a recommended one. This is hitting charm-heat on focal-victoria: https://bugs.launchpad.net/charm-heat/+bug/1895300

[Aurelien Lourot (aurelien-lourot)]

Same with python3-zunclient

[Corey Bryant (corey.bryant)]

check-mir for python-vitrageclient:

Checking support status of binary dependencies...
 * libjs-bootstrap binary and source package is in universe
 * python3-pydot binary and source package is in universe
 * python3-pydotplus binary and source package is in universe

check-mir for python-zunclient:

Checking support status of binary dependencies...
 * libjs-bootstrap binary and source package is in universe
 * python3-docker binary and source package is in universe
 * python3-websocket binary and source package is in universe

We'll need to assess whether these can be moved to Suggests or if they also require MIRs.

Another option perhaps is to patch heat. zun server packages don't exist in Ubuntu and we don't maintain vitrage packages.

[Corey Bryant (corey.bryant)]

I think this is actually a regression in stevedore. I opened the following bug with more details. Hopefully it will get triaged soon and we'll find out. If it is a regression in stevedore then we won't need the MIRs.

https://bugs.launchpad.net/ubuntu/+source/stevedore/+bug/1897969

[Brian Murray (brian-murray)]

The Groovy Gorilla has reached end of life, so this bug will not be fixed for that release

[Corey Bryant (corey.bryant)]

I've dropped python3-zunclient from this MIR. Instead I'm dropping zunclient from heat since we do not package zun itself in Ubuntu.

[Corey Bryant (corey.bryant)]

check-mir for python3-vitrageclient:

Checking support status of build dependencies...
 * debhelper-compat does not exist (pure virtual?)
 * dh-python binary and source package is in universe
 * openstack-pkg-tools binary and source package is in universe
 * python3-coverage binary and source package is in universe
 * python3-hacking binary and source package is in universe
 * python3-mock binary and source package is in universe
 * python3-openstackdocstheme binary and source package is in universe
 * python3-oslotest binary and source package is in universe
 * python3-pydot binary and source package is in universe
 * python3-pydotplus binary and source package is in universe
 * python3-stestr binary and source package is in universe
 * python3-subunit binary and source package is in universe
 * python3-testrepository binary and source package is in universe

Checking support status of binary dependencies...
 * libjs-bootstrap binary and source package is in universe

libjs-bootstrap is only needed for the python-vitrageclient-doc binary package which does not need to be in main.

[Corey Bryant (corey.bryant)]

I've removed the python3-vitrageclient Depends for python3-pydot and python3-pydotplus as they are already Suggests for python3-netorkx, which python3-vitrageclient already depends on.

[Corey Bryant (corey.bryant)]

I have uploaded package changes required to fix this in jammy (which will also be backported to yoga). Since there is a work-around, there are no plans to fix this in prior releases.

[Corey Bryant (corey.bryant)]

This bug was fixed in the package heat - 1:17.0.0+git2022011216.9af3af156-0ubuntu3~cloud0
---------------

 heat (1:17.0.0+git2022011216.9af3af156-0ubuntu3~cloud0) focal-yoga; urgency=medium
 .
   * New update for the Ubuntu Cloud Archive.
 .
 heat (1:17.0.0+git2022011216.9af3af156-0ubuntu3) jammy; urgency=medium
 .
   * d/p/drop-zun(client).patch: Rename patch and drop zun.py as well.
 .
 heat (1:17.0.0+git2022011216.9af3af156-0ubuntu2) jammy; urgency=medium
 .
   * d/control, d/p/drop-zunclient.patch: Drop zunclient as zun is not
     packaged in Ubuntu.
   * d/control: Move python3-vitrageclient from Suggests to Depends in
     preperation for moving to Ubuntu main (LP: #1893935).

[James Page (james-page)]

[Summary]
LGTM +1 for promotion to main.

List of specific binary packages to be promoted to main: python3-virtageclient

Notes:
-doc package does not need to be promoted to main.

[Duplication]
There is no other package in main providing the same functionality.

[Dependencies]
OK:
no other Dependencies to MIR due to this

NOTE:
-doc package does not need to be pulled into main.

[Embedded sources and static linking]
OK:
no embedded source present
no static linking

[Security]
OK:
history of CVEs does not look concerning
does not run a daemon as root
does not use webkit1,2
does not use lib*v8 directly
does not parse data formats
does not open a port
does not process arbitrary web content
does not use centralized online accounts
does not integrate arbitrary javascript into the desktop
does not deal with system authentication (eg, pam), etc)

[Common blockers]
OK:
- does not FTBFS currently
- does have a test suite that runs at build time
  - test suite fails will fail the build upon error.
does have a test suite that runs as autopkgtest
The package has a team bug subscriber
no translation present, but none needed for this case (user visible)?
no new python2 dependency
Python package that is using dh_python

[Packaging red flags]
OK:
Ubuntu does carry a delta, but it is reasonable and maintenance under control
symbols tracking not applicable for this kind of code.
d/watch is present and looks ok
Upstream update history is good
Debian/Ubuntu update history is good
the current release is packaged
promoting this does not seem to cause issues for MOTUs that so far
maintained the package
no massive Lintian warnings
d/rules is rather clean
Does not have Built-Using

[Upstream red flags]
OK:
no Errors/warnings during the build
no incautious use of malloc/sprintf (as far as I can check it)
no use of sudo, gksu, pkexec, or LD_LIBRARY_PATH
no use of user nobody
no use of setuid
no important open bugs (crashers, etc) in Debian or Ubuntu
no dependency on webkit, qtwebkit, seed or libgoa-*
not part of the UI for extra checks

[Steve Langasek (vorlon)]

Override component to main
python-vitrageclient 4.4.0-0ubuntu2 in jammy: universe/misc -> main
python-vitrageclient-doc 4.4.0-0ubuntu2 in jammy amd64: universe/doc/optional/100% -> main
python-vitrageclient-doc 4.4.0-0ubuntu2 in jammy arm64: universe/doc/optional/100% -> main
python-vitrageclient-doc 4.4.0-0ubuntu2 in jammy armhf: universe/doc/optional/100% -> main
python-vitrageclient-doc 4.4.0-0ubuntu2 in jammy i386: universe/doc/optional/100% -> main
python-vitrageclient-doc 4.4.0-0ubuntu2 in jammy ppc64el: universe/doc/optional/100% -> main
python-vitrageclient-doc 4.4.0-0ubuntu2 in jammy riscv64: universe/doc/optional/100% -> main
python-vitrageclient-doc 4.4.0-0ubuntu2 in jammy s390x: universe/doc/optional/100% -> main
python3-vitrageclient 4.4.0-0ubuntu2 in jammy amd64: universe/python/optional/100% -> main
python3-vitrageclient 4.4.0-0ubuntu2 in jammy arm64: universe/python/optional/100% -> main
python3-vitrageclient 4.4.0-0ubuntu2 in jammy armhf: universe/python/optional/100% -> main
python3-vitrageclient 4.4.0-0ubuntu2 in jammy i386: universe/python/optional/100% -> main
python3-vitrageclient 4.4.0-0ubuntu2 in jammy ppc64el: universe/python/optional/100% -> main
python3-vitrageclient 4.4.0-0ubuntu2 in jammy riscv64: universe/python/optional/100% -> main
python3-vitrageclient 4.4.0-0ubuntu2 in jammy s390x: universe/python/optional/100% -> main
15 publications overridden.

[Launchpad Janitor (janitor)]

This bug was fixed in the package heat - 1:17.0.0+git2022011216.9af3af156-0ubuntu3

---------------
heat (1:17.0.0+git2022011216.9af3af156-0ubuntu3) jammy; urgency=medium

  * d/p/drop-zun(client).patch: Rename patch and drop zun.py as well.

 -- Corey Bryant <email address hidden> Thu, 27 Jan 2022 10:30:29 -0500