nova raises ConfigFileValueError for URLs with dashes

[George Shuklin]

nova version: newton
dpkg version: 2:14.0.1-0ubuntu1~cloud0
distribution: nova @ xenial with ubuntu cloud archive, amd64.

Nova fails with exception ConfigFileValueError: Value for option url is not valid: invalid URI: if url parameter of [neutron] section or novncproxy_base_url parameter contains dashes in url.

Steps to reproduce:

Take a working openstack with nova+neutron.

Put (in [neutron] section) url= http://nodash.example.com:9696 - it works

Put url = http://with-dash.example.com:9696 - it fails with exception:

nova[18937]: TRACE Traceback (most recent call last):
nova[18937]: TRACE File "/usr/bin/nova-api-os-compute", line 10, in <module>
nova[18937]: TRACE sys.exit(main())
nova[18937]: TRACE File "/usr/lib/python2.7/dist-packages/nova/cmd/api_os_compute.py", line 51, in main
nova[18937]: TRACE service.wait()
nova[18937]: TRACE File "/usr/lib/python2.7/dist-packages/nova/service.py", line 415, in wait
nova[18937]: TRACE _launcher.wait()
nova[18937]: TRACE File "/usr/lib/python2.7/dist-packages/oslo_service/service.py", line 568, in wait
nova[18937]: TRACE self.conf.log_opt_values(LOG, logging.DEBUG)
nova[18937]: TRACE File "/usr/lib/python2.7/dist-packages/oslo_config/cfg.py", line 2626, in log_opt_values
nova[18937]: TRACE _sanitize(opt, getattr(group_attr, opt_name)))
nova[18937]: TRACE File "/usr/lib/python2.7/dist-packages/oslo_config/cfg.py", line 3057, in __getattr__
nova[18937]: TRACE return self._conf._get(name, self._group)
nova[18937]: TRACE File "/usr/lib/python2.7/dist-packages/oslo_config/cfg.py", line 2668, in _get
nova[18937]: TRACE value = self._do_get(name, group, namespace)
nova[18937]: TRACE File "/usr/lib/python2.7/dist-packages/oslo_config/cfg.py", line 2711, in _do_get
nova[18937]: TRACE % (opt.name, str(ve)))
nova[18937]: TRACE ConfigFileValueError: Value for option url is not valid: invalid URI: 'http://with-dash.example.com:9696'.

Expected behavior: do not crash.

[George Shuklin (george-shuklin)]

I found source of the bug: python-rfc3986 is to blame (it is used by oslo-config). Version 0.2.0-2 contains bug which violates RFC3986. It was fixed in 0.2.2. Version of python-rfc3986 from zesty (0.3.1-2) fix this problem.

I believe this bug should be fixed by bumping up version of python-rfc3986 in UCA to 0.2.2 or higher.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Christoph Fiehe (fiehe)]

A fix is already on its way, but the new python-rfc3986 package is only available in the {xenial,yakkety}-proposed repositories.

Have a look:
https://people.canonical.com/~ubuntu-archive/pending-sru.html

This bug is already covered by this one:
https://bugs.launchpad.net/tripleo/+bug/1632538

[George Shuklin (george-shuklin)]

I found one more source of this bug (It was pointed by Marsikus at habrahabr.ru): https://github.com/openstack/oslo.config/compare/3.18.0...master

As you can see, oslo.config have dependency for python-rfc3986== 0.2.0 in version 3.18.0, and 0.2.2 in stable/newton.

And https://releases.openstack.org/newton/index.html#oslo-config says that 'Newton' is 3.17.0.

I think it is a maintenance mistake from oslo.config upstream.

[ChangBo Guo(gcb) (glongwave)]

rfc3986>=0.2.2 in stable/newton and 3.18
https://github.com/openstack/oslo.config/blob/stable/newton/requirements.txt
https://github.com/openstack/oslo.config/blob/3.18.0/requirements.txt

update requirements for stable/newton in https://github.com/openstack/oslo.config/commit/2f2c1839b7423185a6a48e7b3ca3c3274d5ba8f3, maybe we need a new release 3.17.1 to fix this issue.

[ChangBo Guo(gcb) (glongwave)]

Release Fix in https://review.openstack.org/418345

[Thiago Martins (martinx)]

Guys,

Why is taking so long for python-rfc3986 package to leave the Proposed repo, into main?

https://launchpad.net/ubuntu/+source/python-rfc3986/0.2.2-0ubuntu0.16.04.1

It was uploaded on 2016-10-27!

Cheers!
Thiago

[ChangBo Guo(gcb) (glongwave)]

We released oslo.config 3.17.1 for branch stable/newton, that makes sure rfc3986>=0.2.2 to avoid ConfigFileValueError.

[Corey Bryant (corey.bryant)]

This should be fixed in xenial and yakkety now:

 python-rfc3986 | 0.2.2-0ubuntu0.16.04.1 | xenial-updates | source, all
 python-rfc3986 | 0.2.2-0ubuntu0.16.10.1 | yakkety-updates | source, all

Still some backport/testing/promotions needed for the corresponding cloud archives.

[Matthew Thode (prometheanfire)]

Not seeing the issue from a requirements perspective so removing us.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/requirements 1.1.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/requirements ocata-eol release.