Support for os_cacert is missing in ovb

Bug #1983313 reported by chandan kumar
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Triaged
High
Unassigned

Bug Description

https://review.rdoproject.org/r/c/testproject/+/44151 is trying to run ovb on openstack deployed on IBM cloud.

Here is the credentials snippets of clouds.yaml
```
    cacert: /etc/pki/ca-trust/source/anchors/simpleca.crt
    identity_api_version: '3'
    region_name: regionOne
    volume_api_version: '3'
```

We are using ovb-manage role to create OVB stacks and stack creation is failing with following error: [1]
```
stack_status: CREATE_FAILED
2022-08-01 21:29:49.292901 | primary | stack_status_reason: 'Resource CREATE failed: WaitConditionTimeout: resources.baremetal_env.resources.bmc.resources.bmc_wait_condition:
2022-08-01 21:29:49.292908 | primary | 0 of 1 received'
```

After looking at bmc logs [2]
```
[ 57.011793] cloud-init[1223]: with open("/tmp/bmc-cloud-data") as f:
[ 57.012380] cloud-init[1223]: data=json.loads(f.read())
[ 57.012874] cloud-init[1223]: clouds={"clouds": {"host_cloud": data}}
[ 57.013457] cloud-init[1223]: print(yaml.safe_dump(clouds, default_flow_style=False))'
[ 57.050439] cloud-init[1223]: + rm -f /tmp/bmc-cloud-data
[ 57.051624] cloud-init[1223]: + export OS_CLOUD=host_cloud
[ 57.052190] cloud-init[1223]: + OS_CLOUD=host_cloud
[ 57.053014] cloud-init[1223]: ++ command -v python3
[ 57.053521] cloud-init[1223]: ++ command -v python2
[ 57.054169] cloud-init[1223]: + /usr/bin/python2
[ 57.522295] cloud-init[1223]: Fetching private network
[ 57.523405] cloud-init[1223]: Traceback (most recent call last):
[ 57.523967] cloud-init[1223]: File "<stdin>", line 12, in <module>
[ 57.524545] cloud-init[1223]: File "/usr/lib/python2.7/site-packages/openstack/service_description.py", line 80, in __get__
[ 57.525784] cloud-init[1223]: instance._proxies[self.service_type] = self._make_proxy(instance)
[ 57.526572] cloud-init[1223]: File "/usr/lib/python2.7/site-packages/openstack/service_description.py", line 184, in _make_proxy
[ 57.527461] cloud-init[1223]: **version_kwargs
[ 57.527910] cloud-init[1223]: File "/usr/lib/python2.7/site-packages/openstack/config/cloud_region.py", line 495, in get_session_client
[ 57.529439] cloud-init[1223]: network_endpoint = network_adapter.get_endpoint()
[ 57.530124] cloud-init[1223]: File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 271, in get_endpoint
[ 57.531169] cloud-init[1223]: return self.session.get_endpoint(auth or self.auth, **kwargs)
[ 57.531921] cloud-init[1223]: File "/usr/lib/python2.7/site-packages/keystoneauth1/session.py", line 1139, in get_endpoint
[ 57.533438] cloud-init[1223]: return auth.get_endpoint(self, **kwargs)
[ 57.534087] cloud-init[1223]: File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 380, in get_endpoint
[ 57.535502] cloud-init[1223]: allow_version_hack=allow_version_hack, **kwargs)
[ 57.536172] cloud-init[1223]: File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 271, in get_endpoint_data
[ 57.537078] cloud-init[1223]: service_catalog = self.get_access(session).service_catalog
[ 57.537742] cloud-init[1223]: File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 134, in get_access
[ 57.538594] cloud-init[1223]: self.auth_ref = self.get_auth_ref(session)
[ 57.539190] cloud-init[1223]: File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/generic/base.py", line 206, in get_auth_ref
[ 57.540413] cloud-init[1223]: self._plugin = self._do_create_plugin(session)
[ 57.541075] cloud-init[1223]: File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/generic/base.py", line 138, in _do_create_plugin
[ 57.542007] cloud-init[1223]: authenticated=False)
[ 57.542486] cloud-init[1223]: File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 610, in get_discovery
[ 57.543363] cloud-init[1223]: authenticated=authenticated)
[ 57.543888] cloud-init[1223]: File "/usr/lib/python2.7/site-packages/keystoneauth1/discover.py", line 1442, in get_discovery
[ 57.545789] cloud-init[1223]: disc = Discover(session, url, authenticated=authenticated)
[ 57.546525] cloud-init[1223]: File "/usr/lib/python2.7/site-packages/keystoneauth1/discover.py", line 526, in __init__
[ 57.547359] cloud-init[1223]: authenticated=authenticated)
[ 57.547865] cloud-init[1223]: File "/usr/lib/python2.7/site-packages/keystoneauth1/discover.py", line 101, in get_version_data
[ 57.548722] cloud-init[1223]: resp = session.get(url, headers=headers, authenticated=authenticated)
[ 57.549451] cloud-init[1223]: File "/usr/lib/python2.7/site-packages/keystoneauth1/session.py", line 1037, in get
[ 57.550252] cloud-init[1223]: return self.request(url, 'GET', **kwargs)
[ 57.550823] cloud-init[1223]: File "/usr/lib/python2.7/site-packages/keystoneauth1/session.py", line 835, in request
[ 57.551633] cloud-init[1223]: resp = send(**kwargs)
[ 57.552117] cloud-init[1223]: File "/usr/lib/python2.7/site-packages/keystoneauth1/session.py", line 926, in _send_request
[ 57.552946] cloud-init[1223]: resp = self.session.request(method, url, **kwargs)
[ 57.553581] cloud-init[1223]: File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 533, in request
[ 57.554926] cloud-init[1223]: resp = self.send(prep, **send_kwargs)
[ 57.555557] cloud-init[1223]: File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 646, in send
[ 57.556337] cloud-init[1223]: r = adapter.send(request, **kwargs)
[ 57.556925] cloud-init[1223]: File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 416, in send
[ 57.568242] cloud-init[1223]: self.cert_verify(conn, request.url, verify, cert)
[ 57.568962] cloud-init[1223]: File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 228, in cert_verify
[ 57.569829] cloud-init[1223]: "invalid path: {}".format(cert_loc))
[ 57.570398] cloud-init[1223]: IOError: Could not find a suitable TLS CA certificate bundle, invalid path: /etc/pki/ca-trust/source/anchors/ibm-bm2-nodepool.crt
```

After looking at the code:
https://opendev.org/openstack/openstack-virtual-baremetal/src/branch/master/bin/install_openstackbmc.sh#L57
```
conn = openstack.connect(cloud='host_cloud')
print('Fetching private network')
items = conn.network.networks(name='$private_net')
```
It seems that ovb repo lacks the support of os_cacert that's why it is failing.

Links:
[1]. https://logserver.rdoproject.org/51/44151/11/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-master-ibm/0ccf977/job-output.txt

[2]. https://logserver.rdoproject.org/51/44151/11/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-master-ibm/0ccf977/logs/bmc_11_18012-console.log

Tags: alert ci
Revision history for this message
chandan kumar (chkumar246) wrote :

By following the steve suggestion
I used the above suggestion
```
virt-customize -a bmc-template --upload ibm-bm2-nodepool.crt:/etc/pki/ca-trust/source/anchors/ibm-bm2-nodepool.crt --run-command update-ca-trust

openstack --os-cloud ibm-bm2-nodepool image create --disk-format qcow2 --container-format bare --shared --progress --file bmc-template bmc-template-ibm
```
and
in the latest run
```
- job:
    name: periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-master-ibm
    parent: periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-master
    nodeset: tripleo-ovb-centos-9-primary-ibm
    attempts: 1
    vars:
      ovb_manage_stack_mode: 'create'
      registry_login_enabled: false
      quickstart_verbosity: -vv
      create_private_network: true
      key_name: chandankumar-ovb-test
      cloud_name: ibm-bm2-nodepool
      bmc_template_name: bmc-template-ibm
      cloud_settings:
          ibm-bm2-nodepool:
            public_ip_net: hostonly
            undercloud_flavor: nodepool
            baremetal_flavor: m1.large
            bmc_flavor: m1.small
            extra_node_flavor: m1.small
            enable_config_drive: true
            radvd_flavor: m1.small
            dhcp_relay_flavor: m1.small
            enable_baremetal_config_drive: true
            baremetal_image: CentOS-Stream-GenericCloud-9-20211216
            baremetal_image_name: CentOS-Stream-GenericCloud-9-20211216
- project:
    check:
      jobs:
        - periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-master-ibm
```
and
periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-master-ibm https://review.rdoproject.org/zuul/build/48c42e98d96546b3970a335e2ee6f23e : SUCCESS in 1h 54m 33s it passed.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.