Magnum SSL Error

[Aji Muhammad Arya Revaldi]

**Bug Report**
What happened: Magnum k8s cluster failed to create

What expected: Magnum k8s created successfully without error

Deploy has been done without error with files specify below

**Environment**
* Deployer: Ubuntu
* Openstack Node: CentOS
* Fedora CoreOS: Tried 30 and 31 (same result, ERROR)
* Amphora Image: bionic ubuntu
* Kolla-ansible: 9.0.1
* Openstack: Train
* Docker image install type: source
* Using Official images from Docker Hub

inventory file
```
[control]
node[0:2] ansible_user=kolla

[network]
node[0:2] ansible_user=kolla

[compute]
node[0:2] ansible_user=kolla

[monitoring]
node[0:2] ansible_user=kolla

[storage]
node[0:2] ansible_user=kolla
```

globals.yml
cat /etc/kolla/globals.yml | grep ^[^#]
---
kolla_install_type: "source"
openstack_release: "train"
kolla_internal_vip_address: "10.100.100.2"
kolla_external_vip_address: "10.100.100.3"
kolla_enable_tls_external: "yes"
enable_ceph: "yes"
enable_cinder: "yes"
enable_magnum: "yes"
enable_neutron_vpnaas: "yes"
enable_neutron_fwaas: "yes"
enable_neutron_qos: "yes"
enable_neutron_agent_ha: "yes"
enable_neutron_provider_networks: "yes"
enable_neutron_port_forwarding: "yes"
enable_octavia: "yes"
enable_prometheus: "yes"
ceph_pool_pg_num: 30
ceph_pool_pgp_num: 30
glance_backend_ceph: "yes"
glance_backend_file: "no"

/etc/kolla/config/octavia.conf
```
[neutron]
region_name = RegionOne
endpoint_type = internalURL

[nova]
region_name = RegionOne
endpoint_type = internalURL

[glance]
region_name = RegionOne
endpoint_type = internalURL

[controller_worker]
amp_flavor_id = 6f843807-6bb9-43f8-8007-071760e2f2d6
amp_ssh_key_name = key-amp
amp_boot_network_list = <net_id>
amp_secgroup_list = <secgroup_id>
```

/etc/kolla/config/neutron.conf

```
[DEFAULT]
global_physnet_mtu = 1550
```

/etc/kolla/config/neutron/ml2_conf.ini

```
[ml2]
path_mtu = 1550
```

/etc/kolla/config/heat.conf
```
[DEFAULT]
heat_metadata_server_url = http://10.100.100.2:8000
heat_waitcondition_server_url = http://10.100.100.3:8000/v1/waitcondition
server_keystone_endpoint_type = internal

[clients]
insecure = true

[clients_cinder]
endpoint_type = internalURL
insecure = true

[clients_glance]
endpoint_type = internalURL
insecure = true

[clients_heat]
endpoint_type = internalURL
insecure = true
url = http://10.100.100.2:8004/v1/%(tenant_id)s

[clients_keystone]
endpoint_type = internalURL
insecure = true
auth_uri = http://10.100.100.2:5000

[clients_magnum]
endpoint_type = internalURL
insecure = true

[clients_neutron]
endpoint_type = internalURL
insecure = true

[clients_nova]
endpoint_type = internalURL
insecure = true

[clients_octavia]
endpoint_type = internalURL
insecure = true
```

/etc/kolla/config/magnum.conf

```
[cinder]
default_docker_volume_type = docker_volume

[octavia_client]
region_name = RegionOne
endpoint_type = internalURL

[drivers]
verify_ca = false

[magnum_client]
region_name = RegionOne
endpoint_type = internalURL
```

The ERROR LOG:
# /var/kolla/magnum/magnum-conductor.log

2020-02-18 13:19:00.729 31 ERROR magnum.drivers.heat.k8s_fedora_template_def [req-1d7d0e77-9ac0-4b05-80dc-f5423cfbe7fd - - - - -] Failed to load default keystone auth policy: IOError: [Errno 2] No such file or directory: '/etc/magnum/keystone_auth_default_policy.json'

2020-02-18 13:19:02.918 31 WARNING keystoneauth.discover [req-9f7a8989-fe7a-4adb-924d-d32c4d9ba556 - - - - -]Failed to contact the endpoint at https://10.30.30.101:5000 for discovery. Fallback to using that endpoint as the base url.: SSLError: SSL exception connecting to https://10.30.30.101:5000: HTTPSConnectionPool(host='10.30.30.101', port=5000): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))

[Aji Muhammad Arya Revaldi (aryasaurus)]

It shows warning before Failed
2020-02-23 08:57:24.648 6 WARNING magnum.drivers.heat.template_def [req-a3e585bb-1dc8-4730-9073-841660c7cec3 - - - - -] stack does not have output_key api_address
2020-02-23 08:57:24.650 6 WARNING magnum.drivers.heat.template_def [req-a3e585bb-1dc8-4730-9073-841660c7cec3 - - - - -] stack does not have output_key kube_masters
2020-02-23 08:57:25.319 6 WARNING magnum.drivers.heat.template_def [req-a3e585bb-1dc8-4730-9073-841660c7cec3 - - - - -] stack does not have output_key api_address
2020-02-23 08:57:25.321 6 WARNING magnum.drivers.heat.template_def [req-a3e585bb-1dc8-4730-9073-841660c7cec3 - - - - -] stack does not have output_key kube_minions

[Jakob Erpf (jakoberpf)]

I have the same issue with victoria deployed with kolla.

2021-04-21 23:12:18.347 30 ERROR magnum.drivers.heat.k8s_fedora_template_def [req-8a1cc650-825c-4fd6-8831-0dd394bfa873 - - - - -] Failed to load default keystone auth policy: FileNotFoundError: [Errno 2] No such file or directory: '/etc/magnum/keystone_auth_default_policy.json'

How did you resolve this?

[Mark Goddard (mgoddard)]

Jakob, I have seen the keystone_auth_default_policy.json error and it should not prevent magnum from working.