ceph_client role fails verifing keys when deploying behind a proxy

Bug #1815430 reported by panic!
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack-Ansible
Fix Released
Undecided
Jonathan Rosser

Bug Description

When deploying OSA rocky 18.1.2-16 behind a proxy it fails to verify the apt keys for ceph_client packages due to apt-key not supporting using a proxy.

This was fixed in the rocky cycle of OSA for galera and rabbit mq

Patch for galera:

https://review.openstack.org/#/c/625291/

I modfied the ceph client role to use a local provided key and it deploys flawlessly.

here is the patch for ubuntu only

diff -Naur /etc/ansible/roles/ceph_client_org/files/gpg/460F3994 /etc/ansible/roles/ceph_client/files/gpg/460F3994
--- /etc/ansible/roles/ceph_client_org/files/gpg/460F3994 1970-01-01 01:00:00.000000000 +0100
+++ /etc/ansible/roles/ceph_client/files/gpg/460F3994 2019-02-07 14:14:27.804012078 +0100
@@ -0,0 +1,28 @@
+-----BEGIN PGP PUBLIC KEY BLOCK-----
+Version: SKS 1.1.6
+Comment: Hostname: keyserver.ubuntu.com
+
+mQINBFX4hgkBEADLqn6O+UFp+ZuwccNldwvh5PzEwKUPlXKPLjQfXlQRig1flpCHE0HJ5wgG
+lCtYd3Ol9f9+qU24kDNzfbs5bud58BeE7zFaZ4s0JMOMuVm7p8JhsvkUC/Lo/7NFh25e4kgJ
+pjvnwua7c2YrA44ggRb1QT19ueOZLK5wCQ1mR+0GdrcHRCLr7Sdw1d7aLxMT+5nvqfzsmbDu
+llsWOD6RnMdcqhOxZZvpay8OeuK+yb8FVQ4sOIzBFiNi5cNOFFHg+8dZQoDrK3BpwNxYdGHs
+YIwU9u6DWWqXybBnB9jd2pve9PlzQUbOeHEa4Z+jPqxY829f4ldaql7ig8e6BaInTfs2wPnH
+J+606g2UH86QUmrVAjVzlLCmnqoGymoAPGA4ObHu9X3kO8viMBId9FzooVqR8a9En7ZE0Dm9
+O7puzXR7A1f5sHozJdYHnr32I+B8iOixhDUtxIY4GA8biGATNaPd8XR2Ca1hPuZRVuIiGG9H
+DqUEtXhVfY5qjTjaThIVKtYgEkWMT+Wet3DPPiWT3ftNOE907e6EWEBCHgsEuuZnAbku1GgD
+LBH4/a/yo9bNvGZKRaTUM/1TXhM5XgVKjd07B4cChgKypAVHvef3HKfCG2U/DkyALjteHt/V
+807MtSlQyYaXUTGtDCrQPSlMK5TjmqUnDwy6Qdq8dtWN3DtBWQARAQABtCpDZXBoLmNvbSAo
+cmVsZWFzZSBrZXkpIDxzZWN1cml0eUBjZXBoLmNvbT6JAjgEEwECACIFAlX4hgkCGwMGCwkI
+BwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEOhKwsBGDzmUXdIQAI8YPcZMBWdv489q8CzxlfRI
+RZ3Gv/G/8CH+EOExcmkVZ89mVHngCdAPDOYCl8twWXC1lwJuLDBtkUOHXNuR5+Jcl5zFOUyl
+dq1Hv8u03vjnGT7lLJkJoqpGl9QD8nBqRvBU7EM+CU7kP8+09b+088pULil+8x46PwgXkvOQ
+wfVKSOr740Q4J4nm/nUOyTNtToYntmt2fAVWDTIuyPpAqA6jcqSOC7Xoz9cYxkVWnYMLBUyS
+XmSS0uxl3p+wK0lMG0my/gb+alke5PAQjcE5dtXYzCn+8Lj0uSfCk8Gy0ZOK2oiUjaCGYN6D
+u72qDRFBnR3jaoFqi03bGBIMnglGuAPyBZiI7LJgzuT9xumjKTJW3kN4YJxMNYu1FzmIyFZp
+yvZ7930vB2UpCOiIaRdZiX4Z6ZN2frD3a/vBxBNqiNh/BO+Dex+PDfI4TqwF8zlcjt4XZ2te
+Q8nNMR/D8oiYTUW8hwR4laEmDy7ASxe0p5aijmUApWq5UTsF+s/QbwugccU0iR5orksM5u9M
+ZH4J/mFGKzOltfGXNLYI6D5Mtwrnyi0BsF5eY0u6vkdivtdqrq2DXY+ftuqLOQ7b+t1Rctbc
+MHGPptlxFuN9ufP5TiTWSpfqDwmHCLsTk2vFiMwcHdLpQ1IH8ORVRgPPsiBnBOJ/kIiXG2Sx
+PUTjjEGOVgeA
+=/Tod
+-----END PGP PUBLIC KEY BLOCK-----
diff -Naur /etc/ansible/roles/ceph_client_org/tasks/ceph_preinstall_apt.yml /etc/ansible/roles/ceph_client/tasks/ceph_preinstall_apt.yml
--- /etc/ansible/roles/ceph_client_org/tasks/ceph_preinstall_apt.yml 2019-02-10 22:40:28.229603473 +0100
+++ /etc/ansible/roles/ceph_client/tasks/ceph_preinstall_apt.yml 2019-02-07 14:29:45.900793012 +0100
@@ -22,38 +22,26 @@
   when:
     - ceph_pkg_source == 'ceph'

-- name: Add ceph apt-keys
- block:
- - name: Add keys (primary keyserver)
- apt_key:
- id: "{{ item.hash_id }}"
- keyserver: "{{ item.keyserver | default(omit) }}"
- data: "{{ item.data | default(omit) }}"
- url: "{{ item.url | default(omit) }}"
- state: "present"
- register: add_keys
- until: add_keys is success
- retries: 5
- delay: 2
- with_items: "{{ ceph_gpg_keys }}"
- when:
- - ceph_pkg_source == 'ceph'
+- name: If a keyfile is provided, copy the gpg keyfile to the key location
+ copy:
+ src: "gpg/{{ item.id }}"
+ dest: "{{ item.file }}"
+ mode: '0644'
+ with_items: "{{ ceph_gpg_keys | selectattr('file','defined') | list }}"
+ when:
+ - ceph_pkg_source == 'ceph'

- rescue:
- - name: Add keys (fallback keyserver)
- apt_key:
- id: "{{ item.hash_id }}"
- keyserver: "{{ item.fallback_keyserver | default(omit) }}"
- url: "{{ item.fallback_url | default(omit) }}"
- state: "present"
- register: add_keys_fallback
- until: add_keys_fallback is success
- retries: 5
- delay: 2
- with_items: "{{ ceph_gpg_keys }}"
- when:
- - ceph_pkg_source == 'ceph'
- - item.fallback_keyserver is defined or item.fallback_url is defined
+- name: Install gpg keys
+ apt_key: "{{ key }}"
+ with_items: "{{ ceph_gpg_keys }}"
+ loop_control:
+ loop_var: key
+ register: _add_apt_keys
+ until: _add_apt_keys is success
+ retries: 5
+ delay: 2
+ when:
+ - ceph_pkg_source == 'ceph'

 - name: add ubuntu cloud archive key package
   apt:
diff -Naur /etc/ansible/roles/ceph_client_org/vars/ubuntu.yml /etc/ansible/roles/ceph_client/vars/ubuntu.yml
--- /etc/ansible/roles/ceph_client_org/vars/ubuntu.yml 2019-02-10 22:40:28.229603473 +0100
+++ /etc/ansible/roles/ceph_client/vars/ubuntu.yml 2019-02-07 14:20:30.395581238 +0100
@@ -19,10 +19,8 @@

 # Ceph GPG Keys
 ceph_gpg_keys:
- - key_name: 'ceph'
- keyserver: 'hkp://keyserver.ubuntu.com:80'
- fallback_keyserver: 'hkp://p80.pool.sks-keyservers.net:80'
- hash_id: '0xe84ac2c0460f3994'
+ - id: 460F3994
+ file: /etc/ssl/ceph-key

 # The apt-key command won't del a key when you give it the hash_id, so we have
 # to use the short key ID here instead.

Changed in openstack-ansible:
assignee: nobody → Jonathan Rosser (jrosser)
Changed in openstack-ansible:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to openstack-ansible-ceph_client (master)

Fix proposed to branch: master
Review: https://review.openstack.org/636711

Changed in openstack-ansible:
assignee: Jonathan Rosser (jrosser) → Stuart Grace (stuartgrace)
Changed in openstack-ansible:
assignee: Stuart Grace (stuartgrace) → Jonathan Rosser (jrosser)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to openstack-ansible-ceph_client (master)

Reviewed: https://review.openstack.org/636711
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible-ceph_client/commit/?id=58be4bd5e31475c9a3f897de04f226880a45f7e1
Submitter: Zuul
Branch: master

commit 58be4bd5e31475c9a3f897de04f226880a45f7e1
Author: Stuart Grace <email address hidden>
Date: Wed Feb 13 18:27:39 2019 +0000

    Use in-repo GPG keys

    We make remote network hits to get the GPG keys which are quite
    unreliable, and apt_key does not support using a proxy properly [1]
    so this change installs them from files inside the role.

    The implementation here is derived from that which was done in the
    galera_server role in I7ac1a5e3a05aa3d0b4fae86c4a325ef147a9a528.

    [1] https://github.com/ansible/ansible/issues/31691

    Change-Id: Id040de19dbefc820851928c9a3589f20a6b4bd61
    Closes-Bug: #1815430

Changed in openstack-ansible:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to openstack-ansible-ceph_client (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.openstack.org/638825

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to openstack-ansible-ceph_client (stable/rocky)

Reviewed: https://review.openstack.org/638825
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible-ceph_client/commit/?id=fec4dcc5d504013eda40297bcf9b0f1d8a69d0da
Submitter: Zuul
Branch: stable/rocky

commit fec4dcc5d504013eda40297bcf9b0f1d8a69d0da
Author: Stuart Grace <email address hidden>
Date: Wed Feb 13 18:27:39 2019 +0000

    Use in-repo GPG keys

    We make remote network hits to get the GPG keys which are quite
    unreliable, and apt_key does not support using a proxy properly [1]
    so this change installs them from files inside the role.

    The implementation here is derived from that which was done in the
    galera_server role in I7ac1a5e3a05aa3d0b4fae86c4a325ef147a9a528.

    [1] https://github.com/ansible/ansible/issues/31691

    Change-Id: Id040de19dbefc820851928c9a3589f20a6b4bd61
    Closes-Bug: #1815430
    (cherry picked from commit 58be4bd5e31475c9a3f897de04f226880a45f7e1)

tags: added: in-stable-rocky
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/openstack-ansible-ceph_client rocky-eol

This issue was fixed in the openstack/openstack-ansible-ceph_client rocky-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/openstack-ansible-ceph_client stein-eol

This issue was fixed in the openstack/openstack-ansible-ceph_client stein-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/openstack-ansible-ceph_client train-eol

This issue was fixed in the openstack/openstack-ansible-ceph_client train-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/openstack-ansible-ceph_client ussuri-eol

This issue was fixed in the openstack/openstack-ansible-ceph_client ussuri-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/openstack-ansible-ceph_client yoga-eom

This issue was fixed in the openstack/openstack-ansible-ceph_client yoga-eom release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/openstack-ansible-ceph_client victoria-eom

This issue was fixed in the openstack/openstack-ansible-ceph_client victoria-eom release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/openstack-ansible-ceph_client wallaby-eom

This issue was fixed in the openstack/openstack-ansible-ceph_client wallaby-eom release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/openstack-ansible-ceph_client xena-eom

This issue was fixed in the openstack/openstack-ansible-ceph_client xena-eom release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.