Member creation specifying TLS_cyphers of TLSv1.3 only on Pool leads to Loadbalancer in Error

[Quentin GROLLEAU]

Hello !

When creating a member in a HTTPS Pool specifying the default tls_ciphers from TLSv1.3 result in the Octavia Loadbalancer in Error and octavia-worker raising an exception.

Steps to reproduce:
1. openstack loadbalancer create --vip-subnet-id 1d357d00-7845-4c74-ba1b-151e043a5596
2. openstack loadbalancer listener create --protocol HTTPS --protocol-port 443 087ad07d-1f33-415a-98e3-9343e1d50822
3. loadbalancer pool create --listener cfbba52d-c2bc-426f-8eba-f166e59a6416 --protocol HTTPS --lb-algorithm ROUND_ROBIN --enable-tls --tls-ciphers TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
4. openstack loadbalancer member create 3323d1b8-46ea-4f19-a308-5e9ca461f9dc --address 10.0.3.43 --protocol-port 80

Expected result:

The loadbalancer ends up in ACTIVE provisioning_status

Actual result:

The member fails to create and it ends up in ERROR

Following octavia-worker logs can be found :

https://paste.opendev.org/show/byludJTK8Y2dKNeIYaev/

Haproxy config before creating the member :
https://paste.opendev.org/show/b61Om3TgR8XOm9w0ofTs/

haproxy.cfg.new-failed :
https://paste.opendev.org/show/bTU1iLdrJ95q5G0PnpnG/

[Michael Johnson (johnsom)]

This issue here is you have selected both TLS 1.2 and TLS 1.3 for the pool, but you have provided ciphers that are valid for TLS 1.3 only. To allow TLS 1.2, there must be at least one TLS 1.2 compatible cipher included in the list.

If you add a valid TLS 1.2 cipher to the list, the member will be created properly.

For example, the "Intermediate" compatibility cipher list for TLS 1.2 and 1.3 is:

Cipher suites (TLS 1.3): TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
Cipher suites (TLS 1.2): ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-CHACHA20-POLY1305

If you only want to use "TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256" as the only valid ciphers, you must only enable TLS 1.3.

This website can be a helpful reference for understanding valid ciphers for each protocol version:
https://wiki.mozilla.org/Security/Server_Side_TLS#Recommended_configurations

[Michael Johnson (johnsom)]

Ok, I am going to re-open this.

When I test the above cipher list while limiting the pool to only use TLS 1.3, I get a similar error state, which should work. There is something wrong here.

Example that fails but should not:

openstack loadbalancer pool create --listener 4c4e6bb7-944f-4de3-bf3d-46e251a3f1af --protocol HTTP --lb-algorithm ROUND_ROBIN --enable-tls --tls-ciphers TLS_AES_256_GCM_SHA384 --tls-version TLSv1.3 --name pool1

openstack loadbalancer member create pool1 --address 10.0.3.43 --protocol-port 80 --name member1

[Michael Johnson (johnsom)]

Confirmed, this is a bug and we are not handling TLS v1.3 cipher suites correctly.

HAPRoxy requires "ciphers" for TLS 1.2 and older, but TLS 1.3 requires the "cipphersuites" keyword.

We are going to need to separate them in the jinja.

[Tom Weininger (tweining)]

Commands I used for reproducing the issue and verifying the fix:

LBNAME='lb-c2'
TLSOPTS="--tls-ciphers ECDHE-RSA-AES256-GCM-SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_CCM_SHA256 --tls-version TLSv1.2 --tls-version TLSv1.3"
openstack loadbalancer create --wait --vip-subnet-id public-subnet --name $LBNAME
openstack loadbalancer listener create --wait --name ls-c --protocol TERMINATED_HTTPS --protocol-port 443 ${TLSOPTS} --default-tls-container=$(openstack secret list | awk '/ tls_secret1 / {print $2}') $LBNAME
openstack loadbalancer pool create --wait --listener ls-c --protocol HTTP --lb-algorithm ROUND_ROBIN --enable-tls ${TLSOPTS} --name pl-c
openstack loadbalancer member create pl-c --wait --address 10.0.0.43 --protocol-port 80 --name mem1-c

ssh -o StrictHostKeyChecking=no -i /etc/octavia/.ssh/octavia_ssh_key cloud-user@$(openstack loadbalancer amphora list -f value -c lb_network_ip) sudo cat /var/lib/octavia/*/haproxy.cfg | grep cipher

Output (with fix):
    bind 172.24.4.71:443 ssl crt-list /var/lib/octavia/certs/c0d38597-d13b-4e39-a1aa-25cf67ededa3/6001b31a-f1f4-4a0c-81e5-19b61542caac.pem ciphers ECDHE-RSA-AES256-GCM-SHA384:TLS_AES_128_CCM_SHA256 ciphersuites TLS_AES_128_CCM_SHA256:TLS_CHACHA20_POLY1305_SHA256 no-sslv3 no-tlsv10 no-tlsv11 alpn h2,http/1.1,http/1.0
    server 1a72edbf-d4d4-47b9-a2b2-15ffc114609d 10.0.0.43:80 weight 1 ssl verify none sni ssl_fc_sni ciphers ECDHE-RSA-AES256-GCM-SHA384:TLS_AES_128_CCM_SHA256 ciphersuites TLS_AES_128_CCM_SHA256:TLS_CHACHA20_POLY1305_SHA256 no-sslv3 no-tlsv10 no-tlsv11 alpn h2,http/1.1,http/1.0