nova-api failing with invalid URI: 'http://neutron-server:9696', dns issue

[Steve Noyes]

Not sure if this is really a bug or something I have wrong in my environment.

Trying to run through the kolla-kubernetes setup as described in the Kolla Kubernetes Quickstart Guide. I am using a single host (deployer + the host running the pods).

After I do this step:

for x in nova-api nova-conductor nova-scheduler glance-api \
         glance-registry neutron-server horizon nova-consoleauth \
         nova-novncproxy cinder-api cinder-scheduler; \
do
    kolla-kubernetes resource create pod $x

I see that several containers are not happy:

nova-api-3435170786-qhkg0 0/2 CrashLoopBackOff 6 11m
nova-conductor-0 0/1 CrashLoopBackOff 6 11m
nova-consoleauth-0 0/1 CrashLoopBackOff 6 11m
nova-novncproxy-184904410-bvvf8 2/2 Running 0 11m
nova-scheduler-0 0/1 CrashLoopBackOff 6 11m

Breaking into the nova-api container, there is this error in /var/log/kolla/nova/nova-api.log:

2016-10-17 15:16:10.654 5 INFO nova.wsgi [req-da4caf36-4fc4-4ccf-b73d-dcc416aa17ca - - - - -] metadata listening on 127.0.0.1:8081
2016-10-17 15:16:10.655 5 INFO oslo_service.service [req-da4caf36-4fc4-4ccf-b73d-dcc416aa17ca - - - - -] Starting 1 workers
2016-10-17 15:16:10.702 5 CRITICAL nova [req-da4caf36-4fc4-4ccf-b73d-dcc416aa17ca - - - - -] ConfigFileValueError: Value for option url is
not valid: invalid URI: 'http://neutron-server:9696'
2016-10-17 15:16:10.702 5 ERROR nova Traceback (most recent call last):
2016-10-17 15:16:10.702 5 ERROR nova File "/usr/bin/nova-api", line 10, in <module>
2016-10-17 15:16:10.702 5 ERROR nova sys.exit(main())
2016-10-17 15:16:10.702 5 ERROR nova File "/usr/lib/python2.7/site-packages/nova/cmd/api.py", line 73, in main
2016-10-17 15:16:10.702 5 ERROR nova launcher.wait()
2016-10-17 15:16:10.702 5 ERROR nova File "/usr/lib/python2.7/site-packages/oslo_service/service.py", line 568, in wait
2016-10-17 15:16:10.702 5 ERROR nova self.conf.log_opt_values(LOG, logging.DEBUG)
2016-10-17 15:16:10.702 5 ERROR nova File "/usr/lib/python2.7/site-packages/oslo_config/cfg.py", line 2630, in log_opt_values
2016-10-17 15:16:10.702 5 ERROR nova _sanitize(opt, getattr(group_attr, opt_name)))
2016-10-17 15:16:10.702 5 ERROR nova File "/usr/lib/python2.7/site-packages/oslo_config/cfg.py", line 3061, in __getattr__
2016-10-17 15:16:10.702 5 ERROR nova return self._conf._get(name, self._group)
2016-10-17 15:16:10.702 5 ERROR nova File "/usr/lib/python2.7/site-packages/oslo_config/cfg.py", line 2672, in _get
2016-10-17 15:16:10.702 5 ERROR nova value = self._do_get(name, group, namespace)
2016-10-17 15:16:10.702 5 ERROR nova File "/usr/lib/python2.7/site-packages/oslo_config/cfg.py", line 2715, in _do_get
2016-10-17 15:16:10.702 5 ERROR nova % (opt.name, str(ve)))
2016-10-17 15:16:10.702 5 ERROR nova ConfigFileValueError: Value for option url is not valid: invalid URI: 'http://neutron-server:9696'
2016-10-17 15:16:10.702 5 ERROR nova

Then this error repeats every 10min or so.

What's odd is that uri appears valid on the command line of that container:

()[root@nova-api-3435170786-qhkg0 /]# curl http://neutron-server:9696
{"versions": [{"status": "CURRENT", "id": "v2.0", "links": [{"href": "http://neutron-server:9696/v2.0", "rel": "self"}]}]}curl (http://neutron-server:9696/): response: 200, time: 0.007, size: 122

Any idea what may be causing this? Thanks,

[Steve Noyes (steve-noyes)]

This seems to be something related to dns. When I changed the hostname neutron-server to its ip_address in /etc/kolla/nova-api/nova.conf, the container came up ok.

[huwh (huwh)]

try to update your python2-rfc3986 to 0.3.1 or upper version, and build you docker images about nova.
low version of rfc3986 consider ’-‘ is not right in a URL.

[Steve Noyes (steve-noyes)]

Thanks. That is the problem:

In the container:

>>> rfc3986.__version__
'0.2.0'
>>> rfc3986.is_valid_uri('http://neutron-server')
False

On a different system:

>>> rfc3986.__version__
'0.4.1'
>>> rfc3986.is_valid_uri('http://neutron-server')
True