MicroOVN with SSL breaks nbctl on >3 machines

Bug #2024634 reported by Max Asnaashari
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
microovn
Fix Released
High
Frode Nordahl

Bug Description

The recent addition of SSL to MicroOVN seems to cause issues with `ovn-nbctl` when there are more than 3 nodes in the cluster:
```
cat /var/snap/microovn/common/data/ovn.env
# Generated by MicroOVN, DO NOT EDIT.
OVN_INITIAL_NB="10.215.246.196"
OVN_INITIAL_SB="10.215.246.196"
OVN_NB_CONNECT="ssl:10.215.246.196:6641,ssl:10.215.246.251:6641,ssl:10.215.246.85:6641"
OVN_SB_CONNECT="ssl:10.215.246.196:6642,ssl:10.215.246.251:6642,ssl:10.215.246.85:6642"
OVN_LOCAL_IP="10.215.246.196"
```

```
snap run --shell microovn -c "ovn-nbctl --timeout=10 --db ssl:10.215.246.196:6641,ssl:10.215.246.251:6641,ssl:10.215.246.85:6641 -c /etc/ovn/cert_host -p /etc/ovn/key_host -C /etc/ovn/ovn-central.crt --wait=sb --format=csv --no-headings --data=bare --colum=_uuid,name,acl find port_group name=lxd_net2"

(2023-06-22T03:32:33Z|00001|stream_ssl|ERR|SSL_use_certificate_file: error:80000002:system library::No such file or directory
2023-06-22T03:32:33Z|00002|stream_ssl|ERR|SSL_use_PrivateKey_file: error:10080002:BIO routines::system lib
2023-06-22T03:32:33Z|00003|stream_ssl|ERR|Private key must be configured to use SSL
2023-06-22T03:32:33Z|00004|stream_ssl|ERR|Certificate must be configured to use SSL
2023-06-22T03:32:33Z|00005|stream_ssl|ERR|Private key must be configured to use SSL
2023-06-22T03:32:33Z|00006|stream_ssl|ERR|Certificate must be configured to use SSL
2023-06-22T03:32:33Z|00007|stream_ssl|ERR|Private key must be configured to use SSL
2023-06-22T03:32:33Z|00008|stream_ssl|ERR|Certificate must be configured to use SSL
ovn-nbctl: ssl:10.215.246.196:6641,ssl:10.215.246.251:6641,ssl:10.215.246.85:6641: database connection failed (Protocol not available))
```

Likely related is that the new certificates are only placed on the first three machines of the cluster, but additional machines don't receive these certs:
on the first 3 machines:
```
ls -l /var/snap/microovn/common/data/pki/
total 36
-rw-r--r-- 1 root root 749 Jun 22 03:32 cacert.pem
-rw-r--r-- 1 root root 745 Jun 22 03:32 ovn-controller-cert.pem
-rw-r--r-- 1 root root 288 Jun 22 03:32 ovn-controller-privkey.pem
-rw-r--r-- 1 root root 741 Jun 22 03:32 ovn-northd-cert.pem
-rw-r--r-- 1 root root 288 Jun 22 03:32 ovn-northd-privkey.pem
-rw-r--r-- 1 root root 733 Jun 22 03:32 ovnnb-cert.pem
-rw-r--r-- 1 root root 288 Jun 22 03:32 ovnnb-privkey.pem
-rw-r--r-- 1 root root 733 Jun 22 03:32 ovnsb-cert.pem
```

on machines that joined afterward:
```
ls -l /var/snap/microovn/common/data/pki/
total 12
-rw-r--r-- 1 root root 749 Jun 22 03:32 cacert.pem
-rw-r--r-- 1 root root 745 Jun 22 03:32 ovn-controller-cert.pem
-rw-r--r-- 1 root root 288 Jun 22 03:32 ovn-controller-privkey.pem
```

Tags: lxd microcloud
Revision history for this message
Frode Nordahl (fnordahl) wrote :
Changed in microovn:
status: New → In Progress
importance: Undecided → High
Revision history for this message
Martin Kalcok (martin-kalcok) wrote :
Changed in microovn:
status: In Progress → Fix Committed
assignee: nobody → Frode Nordahl (fnordahl)
Frode Nordahl (fnordahl)
Changed in microovn:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.