Docs job is broken in neutron

[Slawek Kaplonski]

Error like:

2021-05-19 02:18:30.683798 | ubuntu-focal | /home/zuul/src/opendev.org/openstack/neutron/.tox/docs/lib/python3.8/site-packages/dns/hash.py:23: DeprecationWarning: dns.hash module will be removed in future versions. Please use hashlib instead.
2021-05-19 02:18:30.683881 | ubuntu-focal | warnings.warn(
2021-05-19 02:18:30.683908 | ubuntu-focal | /home/zuul/src/opendev.org/openstack/neutron/.tox/docs/lib/python3.8/site-packages/dns/namedict.py:35: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
2021-05-19 02:18:30.683941 | ubuntu-focal | class NameDict(collections.MutableMapping):
2021-05-19 02:18:30.683965 | ubuntu-focal |
2021-05-19 02:18:30.683989 | ubuntu-focal | Extension error (oslo_config.sphinxconfiggen):
2021-05-19 02:18:30.684011 | ubuntu-focal | Handler <function generate_sample at 0x7f10982141f0> for event 'builder-inited' threw an exception (exception: No module named 'mitogen')
2021-05-19 02:18:31.013073 | ubuntu-focal | ERROR: InvocationError for command /home/zuul/src/opendev.org/openstack/neutron/.tox/docs/bin/sphinx-build -W -b html doc/source doc/build/html (exited with code 2)
2021-05-19 02:18:31.013153 | ubuntu-focal | docs finish: run-test after 2.96 seconds
2021-05-19 02:18:31.013688 | ubuntu-focal | docs start: run-test-post

For example https://9252355b19e6b9b2ecd6-c7c5829e17049c4cf47426e53829e931.ssl.cf2.rackcdn.com/783748/4/check/openstack-tox-docs/5a2fd4b/job-output.txt

It seems that it happens since 18.05.2021: http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22for%20event%20'builder-inited'%20threw%20an%20exception%20(exception%3A%20No%20module%20named%20'mitogen')%5C%22

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/792077

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

pyroute2 issue created to add this library to the pyroute2.core module installation requisites: https://github.com/svinota/pyroute2/issues/798

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/792180

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron/+/792181

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Rodolfo Alonso <email address hidden>" on branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron/+/792181

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron/+/792183

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/neutron/+/792188

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/neutron/+/792191

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/c/openstack/neutron/+/792192

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/792180
Committed: https://opendev.org/openstack/neutron/commit/9e6b7a2284a39ae74d2092083165e3fdf4cb8f33
Submitter: "Zuul (22348)"
Branch: master

commit 9e6b7a2284a39ae74d2092083165e3fdf4cb8f33
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed May 19 14:31:31 2021 +0000

    Install "pyroute2" as a doc job depedency

    By adding this library to the doc requirements file, tox will install
    it limiting the upper version according to
    "requirements/upper-constraints.txt".

    "pyroute2" is called in tox doc job as mentioned in [1]. Since version
    0.6.0, the library structure changed. To keep backwards compatibility,
    all modules are imported from the main namespace. As reported in the
    referred bug, the library "mitogen" must be installed first.

    In stable releases that use older "pyroute2" versions, this library is
    not needed but this patch should enforce the upper library version.

    Related-Bug: #1928913

    [1]https://github.com/openstack/neutron/blob/2a4074918c248adf84b0afa404d30c18710da4b2/doc/source/conf.py#L33-L40

    Change-Id: I0cf458f4532d313e593a943653e8310b93ac1e9c

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/792183
Committed: https://opendev.org/openstack/neutron/commit/c9a100db203eb204d58787a24064330619ce1b34
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit c9a100db203eb204d58787a24064330619ce1b34
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed May 19 14:31:31 2021 +0000

    Install "pyroute2" as a doc job depedency

    By adding this library to the doc requirements file, tox will install
    it limiting the upper version according to
    "requirements/upper-constraints.txt".

    "pyroute2" is called in tox doc job as mentioned in [1]. Since version
    0.6.0, the library structure changed. To keep backwards compatibility,
    all modules are imported from the main namespace. As reported in the
    referred bug, the library "mitogen" must be installed first.

    In stable releases that use older "pyroute2" versions, this library is
    not needed but this patch should enforce the upper library version.

    Related-Bug: #1928913

    [1]https://github.com/openstack/neutron/blob/2a4074918c248adf84b0afa404d30c18710da4b2/doc/source/conf.py#L33-L40

    Change-Id: I0cf458f4532d313e593a943653e8310b93ac1e9c
    (cherry picked from commit 9e6b7a2284a39ae74d2092083165e3fdf4cb8f33)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/792188
Committed: https://opendev.org/openstack/neutron/commit/6e775f35ebf7ec407dc8bec93b3edaa28c20747e
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 6e775f35ebf7ec407dc8bec93b3edaa28c20747e
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed May 19 14:31:31 2021 +0000

    Install "pyroute2" as a doc job depedency

    By adding this library to the doc requirements file, tox will install
    it limiting the upper version according to
    "requirements/upper-constraints.txt".

    "pyroute2" is called in tox doc job as mentioned in [1]. Since version
    0.6.0, the library structure changed. To keep backwards compatibility,
    all modules are imported from the main namespace. As reported in the
    referred bug, the library "mitogen" must be installed first.

    In stable releases that use older "pyroute2" versions, this library is
    not needed but this patch should enforce the upper library version.

    Related-Bug: #1928913

    [1]https://github.com/openstack/neutron/blob/2a4074918c248adf84b0afa404d30c18710da4b2/doc/source/conf.py#L33-L40

    Change-Id: I0cf458f4532d313e593a943653e8310b93ac1e9c
    (cherry picked from commit 9e6b7a2284a39ae74d2092083165e3fdf4cb8f33)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/792191
Committed: https://opendev.org/openstack/neutron/commit/ea029256306b4af9495f326fb5f6f2bf4e525b80
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit ea029256306b4af9495f326fb5f6f2bf4e525b80
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed May 19 14:31:31 2021 +0000

    Install "pyroute2" as a doc job depedency

    By adding this library to the doc requirements file, tox will install
    it limiting the upper version according to
    "requirements/upper-constraints.txt".

    "pyroute2" is called in tox doc job as mentioned in [1]. Since version
    0.6.0, the library structure changed. To keep backwards compatibility,
    all modules are imported from the main namespace. As reported in the
    referred bug, the library "mitogen" must be installed first.

    In stable releases that use older "pyroute2" versions, this library is
    not needed but this patch should enforce the upper library version.

    Related-Bug: #1928913

    [1]https://github.com/openstack/neutron/blob/2a4074918c248adf84b0afa404d30c18710da4b2/doc/source/conf.py#L33-L40

    Conflicts:
          doc/requirements.txt

    Change-Id: I0cf458f4532d313e593a943653e8310b93ac1e9c
    (cherry picked from commit 9e6b7a2284a39ae74d2092083165e3fdf4cb8f33)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/792192
Committed: https://opendev.org/openstack/neutron/commit/e00f444211bf8dd349d9a159387e7ee23058a923
Submitter: "Zuul (22348)"
Branch: stable/train

commit e00f444211bf8dd349d9a159387e7ee23058a923
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed May 19 14:31:31 2021 +0000

    Install "pyroute2" as a doc job depedency

    By adding this library to the doc requirements file, tox will install
    it limiting the upper version according to
    "requirements/upper-constraints.txt".

    "pyroute2" is called in tox doc job as mentioned in [1]. Since version
    0.6.0, the library structure changed. To keep backwards compatibility,
    all modules are imported from the main namespace. As reported in the
    referred bug, the library "mitogen" must be installed first.

    In stable releases that use older "pyroute2" versions, this library is
    not needed but this patch should enforce the upper library version.

    Related-Bug: #1928913

    [1]https://github.com/openstack/neutron/blob/2a4074918c248adf84b0afa404d30c18710da4b2/doc/source/conf.py#L33-L40

    Conflicts:
          doc/requirements.txt

    Change-Id: I0cf458f4532d313e593a943653e8310b93ac1e9c
    (cherry picked from commit 9e6b7a2284a39ae74d2092083165e3fdf4cb8f33)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/792077
Committed: https://opendev.org/openstack/neutron/commit/a75fc891bc17fd49b9194d2aed4678d623ec8caf
Submitter: "Zuul (22348)"
Branch: master

commit a75fc891bc17fd49b9194d2aed4678d623ec8caf
Author: Slawek Kaplonski <email address hidden>
Date: Wed May 19 09:25:15 2021 +0200

    Add mitogen to the docs requirements

    Mitogen is dependency of the pyroute2 and was added link to it
    the docs in pyroute2 0.6.1. Due to the missing mitogen in our
    docs venv, docs job was failing.
    So let's install mitogen as docs env requirement, at least for now
    to unblock our gate.
    Later we can think about better solution as e.g. adding it properly to
    the requirements or report and fix the issue in the pyroute2.

    Change-Id: Iba37581916842bb5fc9f42f7f483d3431747bf8d
    Closes-Bug: #1928913

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/793377

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/793377
Committed: https://opendev.org/openstack/neutron/commit/a660f77d388682fd464ba3a624c9fe5599dbd938
Submitter: "Zuul (22348)"
Branch: master

commit a660f77d388682fd464ba3a624c9fe5599dbd938
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Thu May 27 10:48:09 2021 +0000

    Remove "mitogen" library installation from "docs" job

    Since pyroute 0.6.2, this dependency is not needed anymore. The
    pyroute version is bumped both in requirements and doc/requirements.

    This patch bumps the pyroute2 version to 0.6.4 (newest version
    provided in upper-requirements).

    Change-Id: I627e902e32fe59c6829c3bc92af19abff4b70f9a
    Related-Bug: #1928913

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 19.0.0.0rc1 release candidate.