RHOSP13- R5.0-214- DPDK- heat api intermittently returns http 500 error

[alok kumar]

1(out of 3) of the openstack node heat api was returning 500 error for the requests intermittently.

Below error was seen in heat_api_wsgi_error.log continuosly:

[Tue Aug 21 09:08:38.986148 2018] [:error] [pid 21] [remote 10.1.0.10:72] mod_wsgi (pid=21): Target WSGI script '/var/www/cgi-bin/heat/heat_api' cannot be loaded as Python module.
[Tue Aug 21 09:08:38.986184 2018] [:error] [pid 21] [remote 10.1.0.10:72] mod_wsgi (pid=21): Exception occurred processing WSGI script '/var/www/cgi-bin/heat/heat_api'.
[Tue Aug 21 09:08:38.986212 2018] [:error] [pid 21] [remote 10.1.0.10:72] Traceback (most recent call last):
[Tue Aug 21 09:08:38.986238 2018] [:error] [pid 21] [remote 10.1.0.10:72] File "/var/www/cgi-bin/heat/heat_api", line 54, in <module>
[Tue Aug 21 09:08:38.986284 2018] [:error] [pid 21] [remote 10.1.0.10:72] application = init_application()
[Tue Aug 21 09:08:38.986310 2018] [:error] [pid 21] [remote 10.1.0.10:72] File "/usr/lib/python2.7/site-packages/heat/httpd/heat_api.py", line 36, in init_application
[Tue Aug 21 09:08:38.986355 2018] [:error] [pid 21] [remote 10.1.0.10:72] logging.register_options(cfg.CONF)
[Tue Aug 21 09:08:38.986376 2018] [:error] [pid 21] [remote 10.1.0.10:72] File "/usr/lib/python2.7/site-packages/oslo_log/log.py", line 250, in register_options
[Tue Aug 21 09:08:38.986410 2018] [:error] [pid 21] [remote 10.1.0.10:72] conf.register_cli_opts(_options.common_cli_opts)
[Tue Aug 21 09:08:38.986424 2018] [:error] [pid 21] [remote 10.1.0.10:72] File "/usr/lib/python2.7/site-packages/oslo_config/cfg.py", line 2440, in __inner
[Tue Aug 21 09:08:38.986453 2018] [:error] [pid 21] [remote 10.1.0.10:72] result = f(self, *args, **kwargs)
[Tue Aug 21 09:08:38.986467 2018] [:error] [pid 21] [remote 10.1.0.10:72] File "/usr/lib/python2.7/site-packages/oslo_config/cfg.py", line 2662, in register_cli_opts
[Tue Aug 21 09:08:38.986493 2018] [:error] [pid 21] [remote 10.1.0.10:72] self.register_cli_opt(opt, group, clear_cache=False)
[Tue Aug 21 09:08:38.986505 2018] [:error] [pid 21] [remote 10.1.0.10:72] File "/usr/lib/python2.7/site-packages/oslo_config/cfg.py", line 2444, in __inner
[Tue Aug 21 09:08:38.986531 2018] [:error] [pid 21] [remote 10.1.0.10:72] return f(self, *args, **kwargs)
[Tue Aug 21 09:08:38.986544 2018] [:error] [pid 21] [remote 10.1.0.10:72] File "/usr/lib/python2.7/site-packages/oslo_config/cfg.py", line 2654, in register_cli_opt
[Tue Aug 21 09:08:38.986570 2018] [:error] [pid 21] [remote 10.1.0.10:72] raise ArgsAlreadyParsedError("cannot register CLI option")
[Tue Aug 21 09:08:38.986618 2018] [:error] [pid 21] [remote 10.1.0.10:72] ArgsAlreadyParsedError: arguments already parsed: cannot register CLI option

This was causing all the heat test cases to fail, restarting heat_api container on that node fixed the issue and above error is also not seen.

all heat logs will be copied at /cs-shared/bugs/<bugId>

[alexey-mr (alexey-morlang)]

1. This issue is only for heat-api. Contrail plugin is for heat-engine which is different component. So, it looks like an pure openstack issue.
2. It is sporadical rare issue that has an workaround 'restart docker container for heat-api'.
3. So, for now we close this bug. If it become an often issue then an openstack bug is to be opened and this bug can be re-opened to track the OS bug.