Neutron server container failed to start when enabling dvs or nsxv

[Pierre Hanselmann]

When enabling dvs on neutron, the bootstrap container failed to start on deploy with:

TASK [neutron : Running Neutron bootstrap container] ********************************************************************************************************************************************************************************
fatal: [localhost -> localhost]: FAILED! => {"changed": true, "failed": true, "msg": "Container exited with non-zero return code"}

sudo docker logs bootstrap_neutron
INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json
ERROR:__main__:InvalidConfig: Invalid json file found at /var/lib/kolla/config_files/config.json

Taking a closer look at config.json it seem a comma is missing in this scenario:
https://git.openstack.org/cgit/openstack/kolla-ansible/tree/ansible/roles/neutron/templates/neutron-server.json.j2#n45

[Pierre Hanselmann (pierre-hanselmann)]

Review provided here: https://review.openstack.org/#/c/508527/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/508527
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=c60293f6d6eb7cdf55b38585164861d9acd77a75
Submitter: Zuul
Branch: master

commit c60293f6d6eb7cdf55b38585164861d9acd77a75
Author: Pierre Hanselmann <email address hidden>
Date: Fri Sep 29 16:35:04 2017 +0200

    Fix neutron invalid json file

    Fix a missing comma in config.json when neutron_plugin_agent is set
    to 'vmware_nsxv' or 'vmware_dvs'.

    Closes-Bug: #1720380

    Change-Id: I088f09a426a5435202176c45a77b1fbc19468f97

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/509114
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=ce918a6d9a8356d230bc933b2a4d3968517dd383
Submitter: Jenkins
Branch: stable/pike

commit ce918a6d9a8356d230bc933b2a4d3968517dd383
Author: Pierre Hanselmann <email address hidden>
Date: Fri Sep 29 16:35:04 2017 +0200

    Fix neutron invalid json file

    Fix a missing comma in config.json when neutron_plugin_agent is set
    to 'vmware_nsxv' or 'vmware_dvs'.

    Closes-Bug: 1720380

    Change-Id: I088f09a426a5435202176c45a77b1fbc19468f97
    (cherry picked from commit c60293f6d6eb7cdf55b38585164861d9acd77a75)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla-ansible 6.0.0.0b2 development milestone.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla-ansible 5.0.1 release.

[phonix6 (phonix6-deactivatedaccount-deactivatedaccount)]

Hi,

I am using the latest version today on esxi. I am facing a deployment failure.

TASK [neutron : Running Neutron bootstrap container] ********************************************************************************************************************************************************************************
fatal: [compute01 -> controller01]: FAILED! => {"changed": true, "msg": "Container exited with non-zero return code 2", "rc": 2, "stderr": "+ sudo -E kolla_set_configs\nINFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json\nERROR:__main__:Unexpected error:\nTraceback (most recent call last):\n File \"/usr/local/bin/kolla_set_configs\", line 421, in main\n config = load_config()\n File \"/usr/local/bin/kolla_set_configs\", line 303, in load_config\n config = load_from_file()\n File \"/usr/local/bin/kolla_set_configs\", line 291, in load_from_file\n with open(config_file) as f:\nFileNotFoundError: [Errno 2] No such file or directory: '/var/lib/kolla/config_files/config.json'\n", "stderr_lines": ["+ sudo -E kolla_set_configs", "INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json", "ERROR:__main__:Unexpected error:", "Traceback (most recent call last):", " File \"/usr/local/bin/kolla_set_configs\", line 421, in main", " config = load_config()", " File \"/usr/local/bin/kolla_set_configs\", line 303, in load_config", " config = load_from_file()", " File \"/usr/local/bin/kolla_set_configs\", line 291, in load_from_file", " with open(config_file) as f:", "FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/kolla/config_files/config.json'"], "stdout": "", "stdout_lines": []}

Can you please advise?

Thanks

[Nilima Chavan (nilimac)]

Hello phonix6,

Is this this the complete error? Is it a multinode or single node config?
Could you please share more details?
Also, please let you know the hostname used for the deployment?

PS: Please check if the hostname does not contains any "underscore" (_) in it.

Thanks

[Jamil Ahmed (jamil.redhat)]

Hi,

I am using following environment and getting jason file error when I run deploy command. No special characters in any server name, esxi, vcenter, datacenter and DVS names.

I have tried both binary and source install and getting same issue.

Deployment node OS: Ubuntu 20.04.2 LTS
Inventory: multinode
Kolla version: 12.1.0

Controller node OS: Ubuntu 20.04.2 LTS
Compute node OS: Ubuntu 20.04.2 LTS

VMware Environment:

Configured with DRS and a distributed switch.
vCenter: 6.5 17994927
ESXi: 6.5 8294253
DRS: Fully Automated

Error:

TASK [neutron : Creating Neutron database] *************************************************************************************************************
changed: [10.116.10.64 -> 10.116.10.64]

TASK [neutron : Creating Neutron database user and setting permissions] ********************************************************************************
changed: [10.116.10.64 -> 10.116.10.64]

TASK [neutron : Running Neutron bootstrap container] ***************************************************************************************************
fatal: [10.116.10.64 -> 10.116.10.64]: FAILED! => {"changed": true, "msg": "Container exited with non-zero return code 1", "rc": 1, "stderr": "+ sudo -E kolla_set_configs\nINFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json\nERROR:__main__:InvalidConfig: Invalid json file found at /var/lib/kolla/config_files/config.json\n", "stderr_lines": ["+ sudo -E kolla_set_configs", "INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json", "ERROR:__main__:InvalidConfig: Invalid json file found at /var/lib/kolla/config_files/config.json"], "stdout": "", "stdout_lines": []}

NO MORE HOSTS LEFT *************************************************************************************************************************************

PLAY RECAP *********************************************************************************************************************************************
10.116.10.64 : ok=239 changed=166 unreachable=0 failed=1 skipped=155 rescued=0 ignored=1
10.116.10.65 : ok=51 changed=32 unreachable=0 failed=0 skipped=53 rescued=0 ignored=0
localhost : ok=4 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

[Radosław Piliszek (yoctozepto)]

I see you reported also here: https://bugs.launchpad.net/kolla-ansible/+bug/1939080