vault-charm

Invalid certificate generated by Vault causes apache2 service to fail

Bug #1813982 reported by Vladimir Grevtsev on 2019-01-30

This bug affects 1 person

	Status	Importance	Assigned to	Milestone
Gnocchi Charm	Fix Released	High	David Ames	Gnocchi Charm 19.04
OpenStack AODH Charm	Fix Released	High	David Ames	OpenStack AODH Charm 19.04
OpenStack Barbican Charm	Fix Released	High	David Ames	OpenStack Barbican Charm 19.04
OpenStack Cinder Charm	Fix Released	High	Frode Nordahl
OpenStack Dashboard Charm	Fix Released	High	Frode Nordahl
OpenStack Designate Charm	Fix Released	High	David Ames	OpenStack Designate Charm 19.04
OpenStack Glance Charm	Fix Released	High	Frode Nordahl
OpenStack Heat Charm	Fix Released	High	Frode Nordahl
OpenStack Keystone Charm	Fix Released	High	Frode Nordahl
OpenStack Manila Charm	Fix Released	High	David Ames	OpenStack Manila Charm 19.04
OpenStack Neutron API Charm	Fix Released	High	Frode Nordahl
OpenStack Nova Cell Controller Charm	Fix Released	High	David Ames	OpenStack Nova Cell Controller Charm 19.07
OpenStack Nova Cloud Controller Charm	Fix Released	High	Frode Nordahl
OpenStack Octavia Charm	Fix Released	High	David Ames	OpenStack Octavia Charm 19.04
OpenStack RabbitMQ Server Charm	Fix Released	High	Frode Nordahl
OpenStack Swift Proxy Charm	Fix Released	High	Frode Nordahl
charms.openstack	Fix Released	High	David Ames
vault-charm	Invalid	Undecided	Unassigned

Bug Description

Charm: cs:vault-9

Problem statement: After endpoint unit (ex: openstack-dashboard) is establishing "certificates" relation with Vault, it will receive a malformed certificate [2], so apache2 will never come back up.

Part of cert chain that causes apache2 to fail:

4jxDjOfUx5b0UgmXWBXZIcwhQmPMe+O+MfB24t00Ea1XfO7jjvGfNTvfOBSRMoN3
Ay4gY9knZspq2dxJO8NsZW08BYc7XXRG8ta4B1ll+mSVE5M8yOuT0w6iCf0brVSb
Ow==
-----END CERTIFICATE----------BEGIN CERTIFICATE-----
MIID3zCCAsegAwIBAgIJAICQx74GXB7+MA0GCSqGSIb3DQEBCwUAMIGYMQswCQYD
VQQGEwJOTDETMBEGA1UECAwKU29tZS1TdGF0ZTESMBAGA1UEBwwJQW1zdGVyZGFt
MREwDwYDVQQKDAhJbm1hcnNhdDEgMB4GA1UEAwwXYW0tc3YtY2EwMS5uZXRvcHMu

If line break between END and BEGIN blocks is added manually - Apache service is coming back online.

ubuntu@juju-da7df6-6-lxd-8:~$ sudo service apache2 status
● apache2.service - The Apache HTTP Server
   Loaded: loaded (/lib/systemd/system/apache2.service; enabled; vendor preset: enabled)
  Drop-In: /lib/systemd/system/apache2.service.d
           └─apache2-systemd.conf
   Active: failed (Result: exit-code) since Wed 2019-01-30 16:35:08 UTC; 3h 25min ago
Main PID: 1631200 (code=exited, status=1/FAILURE)

Jan 30 16:33:55 juju-da7df6-6-lxd-8 systemd[1]: Reloading The Apache HTTP Server.
Jan 30 16:33:55 juju-da7df6-6-lxd-8 systemd[1]: Reloaded The Apache HTTP Server.
Jan 30 16:34:08 juju-da7df6-6-lxd-8 systemd[1]: Reloading The Apache HTTP Server.
Jan 30 16:34:08 juju-da7df6-6-lxd-8 systemd[1]: Reloaded The Apache HTTP Server.
Jan 30 16:34:33 juju-da7df6-6-lxd-8 systemd[1]: Reloading The Apache HTTP Server.
Jan 30 16:34:33 juju-da7df6-6-lxd-8 systemd[1]: Reloaded The Apache HTTP Server.
Jan 30 16:35:08 juju-da7df6-6-lxd-8 systemd[1]: Reloading The Apache HTTP Server.
Jan 30 16:35:08 juju-da7df6-6-lxd-8 systemd[1]: Reloaded The Apache HTTP Server.
Jan 30 16:35:08 juju-da7df6-6-lxd-8 systemd[1]: apache2.service: Main process exited, code=exited, status=1/FAILURE
Jan 30 16:35:08 juju-da7df6-6-lxd-8 systemd[1]: apache2.service: Failed with result 'exit-code'.

$ tail -n10 /var/log/apache2/error.log
[Wed Jan 30 16:34:08.211127 2019] [mpm_event:notice] [pid 1631200:tid 140067706403776] AH00489: Apache/2.4.29 (Ubuntu) OpenSSL/1.1.0g mod_wsgi/4.5.17 Python/2.7 configured -- resuming normal operations
[Wed Jan 30 16:34:08.211165 2019] [core:notice] [pid 1631200:tid 140067706403776] AH00094: Command line: '/usr/sbin/apache2'
[Wed Jan 30 16:34:33.812752 2019] [mpm_event:notice] [pid 1631200:tid 140067706403776] AH00493: SIGUSR1 received. Doing graceful restart
[Wed Jan 30 16:34:33.877196 2019] [mpm_event:notice] [pid 1631200:tid 140067706403776] AH00489: Apache/2.4.29 (Ubuntu) OpenSSL/1.1.0g mod_wsgi/4.5.17 Python/2.7 configured -- resuming normal operations
[Wed Jan 30 16:34:33.877234 2019] [core:notice] [pid 1631200:tid 140067706403776] AH00094: Command line: '/usr/sbin/apache2'
[Wed Jan 30 16:35:08.489187 2019] [mpm_event:notice] [pid 1631200:tid 140067706403776] AH00493: SIGUSR1 received. Doing graceful restart
[Wed Jan 30 16:35:08.551041 2019] [ssl:emerg] [pid 1631200:tid 140067706403776] AH02562: Failed to configure certificate juju-da7df6-6-lxd-8.maas:433:0 (with chain), check /etc/apache2/ssl/horizon/cert_dashboard
[Wed Jan 30 16:35:08.551080 2019] [ssl:emerg] [pid 1631200:tid 140067706403776] SSL Library Error: error:0906D066:PEM routines:PEM_read_bio:bad end line
[Wed Jan 30 16:35:08.551106 2019] [ssl:emerg] [pid 1631200:tid 140067706403776] SSL Library Error: error:140DC009:SSL routines:use_certificate_chain_file:PEM lib
[Wed Jan 30 16:35:08.551125 2019] [:emerg] [pid 1631200:tid 140067706403776] AH00020: Configuration Failed, exiting

[1] https://docs.openstack.org/project-deploy-guide/charm-deployment-guide/latest/app-certificate-management.html
[2] https://pastebin.canonical.com/p/BbFWsMBHBZ/

Tags:

Revision history for this message

Vladimir Grevtsev (vlgrevtsev) wrote on 2019-01-30:

Subscribed ~field-high as this may impact ongoing customer delivery and handover.

tags:

added: cpe-onsite field-high

Revision history for this message

Vladimir Grevtsev (vlgrevtsev) wrote on 2019-01-30:

Some kind of workaround:

for service in nova-cloud-controller keystone heat cinder ceph-radosgw designate glance neutron-api gnocchi aodh openstack-dashboard; do

juju run --application $service 'for x in $(sudo grep -rIl "END CERTIFICATE----------BEGIN CERTIFICATE" /etc/apache2/ssl); do sed -i "s/END CERTIFICATE----------BEGIN CERTIFICATE/END CERTIFICATE-----\n-----BEGIN CERTIFICATE/g" $x; done; service apache2 restart'

done

Revision history for this message

Frode Nordahl (fnordahl) wrote on 2019-01-31:

This is fixed in charm-helpers here https://github.com/juju/charm-helpers/commit/cb2c2282f900a6e198b3578610c6c5375ea83292

The master branch of charm-openstack-dashboard got the charm-helpers synced in here: https://git.openstack.org/cgit/openstack/charm-openstack-dashboard/commit/?id=40b511a95b4010c57c64cee8ac66163c379b46e2

Changed in vault-charm:
status:	New → Confirmed
importance:	Undecided → High
assignee:	nobody → Frode Nordahl (fnordahl)
status:	Confirmed → Fix Committed
status:	Fix Committed → Invalid
Changed in charm-openstack-dashboard:
status:	New → Fix Committed
importance:	Undecided → High
assignee:	nobody → Frode Nordahl (fnordahl)
Changed in vault-charm:
importance:	High → Undecided
assignee:	Frode Nordahl (fnordahl) → nobody

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-01-31: Fix proposed to charm-openstack-dashboard (stable/18.11)

Fix proposed to branch: stable/18.11
Review: https://review.openstack.org/634107

Frode Nordahl (fnordahl) on 2019-01-31

Changed in charm-cinder:
status:	New → Fix Committed
importance:	Undecided → High
assignee:	nobody → Frode Nordahl (fnordahl)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-01-31: Fix proposed to charm-cinder (stable/18.11)

Fix proposed to branch: stable/18.11
Review: https://review.openstack.org/634124

Frode Nordahl (fnordahl) on 2019-01-31

Changed in charm-glance:
status:	New → Fix Committed
importance:	Undecided → High
assignee:	nobody → Frode Nordahl (fnordahl)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-01-31: Fix proposed to charm-glance (stable/18.11)

Fix proposed to branch: stable/18.11
Review: https://review.openstack.org/634125

Frode Nordahl (fnordahl) on 2019-01-31

Changed in charm-heat:
status:	New → Fix Committed
importance:	Undecided → High
assignee:	nobody → Frode Nordahl (fnordahl)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-01-31: Fix proposed to charm-heat (stable/18.11)

Fix proposed to branch: stable/18.11
Review: https://review.openstack.org/634127

Frode Nordahl (fnordahl) on 2019-01-31

Changed in charm-neutron-api:
status:	New → Fix Committed
importance:	Undecided → High
assignee:	nobody → Frode Nordahl (fnordahl)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-01-31: Fix proposed to charm-neutron-api (stable/18.11)

Fix proposed to branch: stable/18.11
Review: https://review.openstack.org/634128

Frode Nordahl (fnordahl) on 2019-01-31

Changed in charm-nova-cloud-controller:
status:	New → Fix Committed
importance:	Undecided → High
assignee:	nobody → Frode Nordahl (fnordahl)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-01-31: Fix proposed to charm-nova-cloud-controller (stable/18.11)

Fix proposed to branch: stable/18.11
Review: https://review.openstack.org/634130

Frode Nordahl (fnordahl) on 2019-01-31

Changed in charm-rabbitmq-server:
status:	New → Fix Committed
importance:	Undecided → High
assignee:	nobody → Frode Nordahl (fnordahl)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-01-31: Fix proposed to charm-rabbitmq-server (stable/18.11)

#10

Fix proposed to branch: stable/18.11
Review: https://review.openstack.org/634131

Frode Nordahl (fnordahl) on 2019-01-31

Changed in charm-swift-proxy:
status:	New → Fix Committed
importance:	Undecided → High
assignee:	nobody → Frode Nordahl (fnordahl)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-01-31: Fix proposed to charm-swift-proxy (stable/18.11)

#11

Fix proposed to branch: stable/18.11
Review: https://review.openstack.org/634132

Revision history for this message

Frode Nordahl (fnordahl) wrote on 2019-01-31:

#12

FTR; all reactive charms already contain the fix after rebuild commits landed January 24th

Revision history for this message

Frode Nordahl (fnordahl) wrote on 2019-01-31:

#13

Having said that the stable reactive charms may still need a rebuild for it to be marked as "Fix Released"

Frode Nordahl (fnordahl) on 2019-01-31

Changed in charm-aodh:
assignee:	nobody → Frode Nordahl (fnordahl)
importance:	Undecided → High
status:	New → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-01-31: Fix proposed to charm-aodh (stable/18.11)

#14

Fix proposed to branch: stable/18.11
Review: https://review.openstack.org/634134

Frode Nordahl (fnordahl) on 2019-01-31

Changed in charm-barbican:
assignee:	nobody → Frode Nordahl (fnordahl)
importance:	Undecided → High
status:	New → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-01-31: Fix proposed to charm-barbican (stable/18.11)

#15

Fix proposed to branch: stable/18.11
Review: https://review.openstack.org/634135

Frode Nordahl (fnordahl) on 2019-01-31

Changed in charm-designate:
assignee:	nobody → Frode Nordahl (fnordahl)
importance:	Undecided → High
status:	New → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-01-31: Fix proposed to charm-designate (stable/18.11)

#16

Fix proposed to branch: stable/18.11
Review: https://review.openstack.org/634136

Frode Nordahl (fnordahl) on 2019-01-31

Changed in charm-gnocchi:
assignee:	nobody → Frode Nordahl (fnordahl)
importance:	Undecided → High
status:	New → Confirmed
Changed in charm-manila:
assignee:	nobody → Frode Nordahl (fnordahl)
importance:	Undecided → High
status:	New → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-01-31: Fix proposed to charm-manila (stable/18.11)

#17

Fix proposed to branch: stable/18.11
Review: https://review.openstack.org/634138

Frode Nordahl (fnordahl) on 2019-01-31

Changed in charm-nova-cell-controller:
assignee:	nobody → Frode Nordahl (fnordahl)
importance:	Undecided → High
status:	New → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-01-31: Fix proposed to charm-nova-cell-controller (stable/18.11)

#18

Fix proposed to branch: stable/18.11
Review: https://review.openstack.org/634141

Frode Nordahl (fnordahl) on 2019-01-31

Changed in charm-octavia:
assignee:	nobody → Frode Nordahl (fnordahl)
importance:	Undecided → High
status:	New → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-01-31: Fix proposed to charm-octavia (stable/18.11)

#19

Fix proposed to branch: stable/18.11
Review: https://review.openstack.org/634142

Frode Nordahl (fnordahl) on 2019-01-31

Changed in charm-keystone:
importance:	Undecided → High
status:	New → Fix Released
assignee:	nobody → Frode Nordahl (fnordahl)

OpenStack Infra (hudson-openstack) on 2019-01-31

Changed in charm-gnocchi:
status:	Confirmed → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-01-31: Fix proposed to charm-gnocchi (stable/18.11)

#20

Fix proposed to branch: stable/18.11
Review: https://review.openstack.org/634259

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-01-31: Fix merged to charm-barbican (stable/18.11)

#21

Reviewed: https://review.openstack.org/634135
Committed: https://git.openstack.org/cgit/openstack/charm-barbican/commit/?id=23ce1be4514b40294bf39b75c4f4c084cbdc1ff8
Submitter: Zuul
Branch: stable/18.11

commit 23ce1be4514b40294bf39b75c4f4c084cbdc1ff8
Author: Corey Bryant <email address hidden>
Date: Thu Jan 17 17:01:43 2019 +0000

Rebuild for certificate line separation issue

    Closes-Bug: #1813982
    Change-Id: Ie26a4945f705c448e160b6514b06338f9d6ac3ad
    (cherry picked from commit b7ba9448b80bbf31fd9df920f4107e3f900d7859)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-01-31: Fix merged to charm-openstack-dashboard (stable/18.11)

#22

Reviewed: https://review.openstack.org/634107
Committed: https://git.openstack.org/cgit/openstack/charm-openstack-dashboard/commit/?id=9cc0c77d568e7e17bf4da1e05be9b68b7014ed24
Submitter: Zuul
Branch: stable/18.11

commit 9cc0c77d568e7e17bf4da1e05be9b68b7014ed24
Author: Frode Nordahl <email address hidden>
Date: Thu Jan 31 07:57:35 2019 +0100

c-h: Separate certificates with lineseparator in bundles

Cherry-pick specific fix for charm-helpers already synced
into master.

Change-Id: I9563f86ca97b93ef96295a34da2a77eb8c9520a7
Closes-Bug: #1813982

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-01-31: Fix merged to charm-glance (stable/18.11)

#23

Reviewed: https://review.openstack.org/634125
Committed: https://git.openstack.org/cgit/openstack/charm-glance/commit/?id=76cb969967330df3249304398a2ccd98dfc67c54
Submitter: Zuul
Branch: stable/18.11

commit 76cb969967330df3249304398a2ccd98dfc67c54
Author: Frode Nordahl <email address hidden>
Date: Thu Jan 31 09:18:25 2019 +0100

c-h: Separate certificates with lineseparator in bundles

Cherry-pick specific fix for charm-helpers already synced
into master.

Change-Id: If8bd06c867fbe9e920826b9268a0d388e987b298
Closes-Bug: #1813982

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-01-31: Fix merged to charm-neutron-api (stable/18.11)

#24

Reviewed: https://review.openstack.org/634128
Committed: https://git.openstack.org/cgit/openstack/charm-neutron-api/commit/?id=23cdf769cd457ba8ef1b57bae660374159773524
Submitter: Zuul
Branch: stable/18.11

commit 23cdf769cd457ba8ef1b57bae660374159773524
Author: Frode Nordahl <email address hidden>
Date: Thu Jan 31 09:22:05 2019 +0100

c-h: Separate certificates with lineseparator in bundles

Cherry-pick specific fix for charm-helpers already synced
into master.

Change-Id: Ia6fd02769b48d9fa6ab7e20352352b5ef78f016c
Closes-Bug: #1813982

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-01-31: Fix merged to charm-cinder (stable/18.11)

#25

Reviewed: https://review.openstack.org/634124
Committed: https://git.openstack.org/cgit/openstack/charm-cinder/commit/?id=0e3181d09f9c80c3b1a3f68e85cc2f3571ae13c1
Submitter: Zuul
Branch: stable/18.11

commit 0e3181d09f9c80c3b1a3f68e85cc2f3571ae13c1
Author: Frode Nordahl <email address hidden>
Date: Thu Jan 31 09:16:05 2019 +0100

c-h: Separate certificates with lineseparator in bundles

Cherry-pick specific fix for charm-helpers already synced
into master.

Change-Id: If2d24f9ef8574a7ad202ee62073568a94b0960bc
Closes-Bug: #1813982

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-01-31: Fix merged to charm-nova-cloud-controller (stable/18.11)

#26

Reviewed: https://review.openstack.org/634130
Committed: https://git.openstack.org/cgit/openstack/charm-nova-cloud-controller/commit/?id=d76a3f4a01a75f2375b71936af306b2512b154a0
Submitter: Zuul
Branch: stable/18.11

commit d76a3f4a01a75f2375b71936af306b2512b154a0
Author: Frode Nordahl <email address hidden>
Date: Thu Jan 31 09:23:44 2019 +0100

c-h: Separate certificates with lineseparator in bundles

Cherry-pick specific fix for charm-helpers already synced
into master.

Change-Id: Ib098462c3ef38022a5a215f4cfe49ccc0a2993ab
Closes-Bug: #1813982

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-01-31: Fix merged to charm-nova-cell-controller (stable/18.11)

#27

Reviewed: https://review.openstack.org/634141
Committed: https://git.openstack.org/cgit/openstack/charm-nova-cell-controller/commit/?id=3da05863c0bc4c80d3db454b6f9dc858897f3e42
Submitter: Zuul
Branch: stable/18.11

commit 3da05863c0bc4c80d3db454b6f9dc858897f3e42
Author: Corey Bryant <email address hidden>
Date: Thu Jan 17 17:03:44 2019 +0000

Rebuild for certificate line separation issue

    Closes-Bug: #1813982
    Change-Id: Id0a27b4271a6435c00207f703c31c565da84d977
    (cherry picked from commit 1f10bb491699e7af61b15f2cc7b08ab191e7b8bb)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-01-31: Fix merged to charm-swift-proxy (stable/18.11)

#28

Reviewed: https://review.openstack.org/634132
Committed: https://git.openstack.org/cgit/openstack/charm-swift-proxy/commit/?id=0cdf8c2518259178b33f80a2874015ae97e63a62
Submitter: Zuul
Branch: stable/18.11

commit 0cdf8c2518259178b33f80a2874015ae97e63a62
Author: Frode Nordahl <email address hidden>
Date: Thu Jan 31 09:26:11 2019 +0100

c-h: Separate certificates with lineseparator in bundles

Cherry-pick specific fix for charm-helpers already synced
into master.

Change-Id: I1ed11fc3f7347d1722dbce8316644cf380011798
Closes-Bug: #1813982

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-01-31: Fix merged to charm-heat (stable/18.11)

#29

Reviewed: https://review.openstack.org/634127
Committed: https://git.openstack.org/cgit/openstack/charm-heat/commit/?id=c25012e6232c16c18a07676c662c3b72943efe17
Submitter: Zuul
Branch: stable/18.11

commit c25012e6232c16c18a07676c662c3b72943efe17
Author: Frode Nordahl <email address hidden>
Date: Thu Jan 31 09:20:10 2019 +0100

c-h: Separate certificates with lineseparator in bundles

Cherry-pick specific fix for charm-helpers already synced
into master.

Change-Id: Ib0014fcc1a3a6a6820ee46d862a3b88c85388d1c
Closes-Bug: #1813982

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-01-31: Fix merged to charm-octavia (stable/18.11)

#30

Reviewed: https://review.openstack.org/634142
Committed: https://git.openstack.org/cgit/openstack/charm-octavia/commit/?id=168bf9219f83e6b40ef0cf0a08fb142a5ae00ad3
Submitter: Zuul
Branch: stable/18.11

commit 168bf9219f83e6b40ef0cf0a08fb142a5ae00ad3
Author: Corey Bryant <email address hidden>
Date: Thu Jan 17 17:03:53 2019 +0000

Rebuild for certificate line separation issue

    Closes-Bug: #1813982
    Change-Id: Id046f8b490bb1ecc4033985b9919a3319abe1313
    (cherry picked from commit 09d9d93f3b7353560c89f11ee751a4b428399df0)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-01-31: Fix merged to charm-manila (stable/18.11)

#31

Reviewed: https://review.openstack.org/634138
Committed: https://git.openstack.org/cgit/openstack/charm-manila/commit/?id=7a00cf39ab017ced583dff3c24ef9569ce1eccb3
Submitter: Zuul
Branch: stable/18.11

commit 7a00cf39ab017ced583dff3c24ef9569ce1eccb3
Author: Corey Bryant <email address hidden>
Date: Thu Jan 17 17:03:08 2019 +0000

Rebuild for certificate line separation issue

    Closes-Bug: #1813982
    Change-Id: I50f4bcf7a3fd3c7f9caee28c7214294f026d6750
    (cherry picked from commit c91ae4d2f92cdd9c196ed408b011b972e6ae4c24)

Frode Nordahl (fnordahl) on 2019-02-01

Changed in charm-barbican:
status:	Fix Committed → Fix Released
Changed in charm-manila:
status:	Fix Committed → Fix Released
Changed in charm-nova-cell-controller:
status:	Fix Committed → Fix Released
Changed in charm-octavia:
status:	Fix Committed → Fix Released
Changed in charm-cinder:
status:	Fix Committed → Fix Released
Changed in charm-glance:
status:	Fix Committed → Fix Released
Changed in charm-heat:
status:	Fix Committed → Fix Released
Changed in charm-neutron-api:
status:	Fix Committed → Fix Released
Changed in charm-nova-cloud-controller:
status:	Fix Committed → Fix Released
Changed in charm-openstack-dashboard:
status:	Fix Committed → Fix Released
Changed in charm-swift-proxy:
status:	Fix Committed → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-01: Fix merged to charm-rabbitmq-server (stable/18.11)

#32

Reviewed: https://review.openstack.org/634131
Committed: https://git.openstack.org/cgit/openstack/charm-rabbitmq-server/commit/?id=4365f896ddf5fc54df335059fc7e67a41b040438
Submitter: Zuul
Branch: stable/18.11

commit 4365f896ddf5fc54df335059fc7e67a41b040438
Author: Frode Nordahl <email address hidden>
Date: Thu Jan 31 09:24:52 2019 +0100

c-h: Separate certificates with lineseparator in bundles

Cherry-pick specific fix for charm-helpers already synced
into master.

Change-Id: I3196e7a4ea87a823a2cb487975a92110a98405b2
Closes-Bug: #1813982

Frode Nordahl (fnordahl) on 2019-02-01

Changed in charm-rabbitmq-server:
status:	Fix Committed → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-01: Fix merged to charm-aodh (stable/18.11)

#33

Reviewed: https://review.openstack.org/634134
Committed: https://git.openstack.org/cgit/openstack/charm-aodh/commit/?id=1dff4ab5ead1892b5271f99ea458d13a1f422984
Submitter: Zuul
Branch: stable/18.11

commit 1dff4ab5ead1892b5271f99ea458d13a1f422984
Author: Corey Bryant <email address hidden>
Date: Thu Jan 17 17:01:26 2019 +0000

Rebuild for certificate line separation issue

    Closes-Bug: #1813982
    Change-Id: Ie159226495e61c131d02a41f8cc235eb1086b44a
    (cherry picked from commit 758654ac8eb3cefe86091008ed368c2c37099a55)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-01: Fix merged to charm-gnocchi (master)

#34

Reviewed: https://review.openstack.org/631570
Committed: https://git.openstack.org/cgit/openstack/charm-gnocchi/commit/?id=329beab23915578b8ce438f8d0334c28ef9bdab5
Submitter: Zuul
Branch: master

commit 329beab23915578b8ce438f8d0334c28ef9bdab5
Author: Corey Bryant <email address hidden>
Date: Thu Jan 17 17:02:46 2019 +0000

Rebuild for sync charm-helpers

Run ceilometer upgrade action in functional tests.

Switch to using proposed packages pending LP: #1813582

    Closes-Bug: #1813982
    Related-Bug: #1813582
    Change-Id: Ie4bce16994fee7a3c4a546e0d7450dd44e92373d

Changed in charm-gnocchi:
status:	In Progress → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-01: Fix merged to charm-gnocchi (stable/18.11)

#35

Reviewed: https://review.openstack.org/634259
Committed: https://git.openstack.org/cgit/openstack/charm-gnocchi/commit/?id=b9173f1587611f6f0bb998751ee38b72d048aef3
Submitter: Zuul
Branch: stable/18.11

commit b9173f1587611f6f0bb998751ee38b72d048aef3
Author: Corey Bryant <email address hidden>
Date: Thu Jan 17 17:02:46 2019 +0000

Rebuild for sync charm-helpers

Run ceilometer upgrade action in functional tests.

Switch to using proposed packages pending LP: #1813582

    Closes-Bug: #1813982
    Related-Bug: #1813582
    Change-Id: Ie4bce16994fee7a3c4a546e0d7450dd44e92373d
    (cherry picked from commit 329beab23915578b8ce438f8d0334c28ef9bdab5)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-01: Fix merged to charm-designate (stable/18.11)

#36

Reviewed: https://review.openstack.org/634136
Committed: https://git.openstack.org/cgit/openstack/charm-designate/commit/?id=a09d89f5f459ba7f99017f7442b268696164eeb9
Submitter: Zuul
Branch: stable/18.11

commit a09d89f5f459ba7f99017f7442b268696164eeb9
Author: Corey Bryant <email address hidden>
Date: Thu Jan 17 17:02:25 2019 +0000

Rebuild for certificate line separation issue

    Closes-Bug: #1813982
    Change-Id: I2936577d8c11cd3f7d1b1ce84696f6afa91f64df
    (cherry picked from commit a5bfa3978ad003cb51ffbd446d7ee2006942c5ac)

Revision history for this message

Vladimir Grevtsev (vlgrevtsev) wrote on 2019-02-01:

#37

Almost all charms are fine except of aodh and gnocchi - they are still affected.

Charm versions:

$ juju status | egrep 'gnocchi|aodh' | egrep -v 'telegraf|hacluster'
aodh 6.0.1 blocked 3 aodh jujucharms 21 ubuntu
gnocchi 4.2.5 active 3 gnocchi jujucharms 16 ubuntu
aodh/0 blocked idle 3/lxd/0 10.220.41.25 8042/tcp Services not running that should be: apache2
aodh/1 blocked idle 5/lxd/0 10.220.41.16 8042/tcp Services not running that should be: apache2
aodh/2* blocked idle 6/lxd/0 10.220.41.3 8042/tcp Services not running that should be: apache2
gnocchi/0 active idle 3/lxd/4 10.220.41.34 8041/tcp Unit is ready
gnocchi/1* active idle 5/lxd/3 10.220.41.15 8041/tcp Unit is ready
gnocchi/2 active idle 6/lxd/4 10.220.41.23 8041/tcp Unit is ready

(gnocchi was in failed state too, I recovered it manually by fixing certificates).

Revision history for this message

David Ames (thedac) wrote on 2019-02-01:

#38

The reactive OpenStack charms do not use charmhelpers.contrib.openstack.cert_utils.install_certs when related to the certificates relation with vault.

The same bug exists in charms.openstack.charms.classes.OpenStackCharm.get_certs_and_keys when certificates_interface is in use.

PR on the way.

Once the PR lands this will require a rebuild of reactive charms again.

Changed in charms.openstack:
importance:	Undecided → High
status:	New → Triaged
assignee:	nobody → David Ames (thedac)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-01: Fix proposed to charms.openstack (master)

#39

Fix proposed to branch: master
Review: https://review.openstack.org/634510

Changed in charms.openstack:
status:	Triaged → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-02: Fix merged to charms.openstack (master)

#40

Reviewed: https://review.openstack.org/634510
Committed: https://git.openstack.org/cgit/openstack/charms.openstack/commit/?id=982ae33054e57c8f64095ebbfbb187a181db13e6
Submitter: Zuul
Branch: master

commit 982ae33054e57c8f64095ebbfbb187a181db13e6
Author: David Ames <email address hidden>
Date: Fri Feb 1 15:05:32 2019 -0800

Fix certificate chain concatenation

    The reactive OpenStack charms do not use
    charmhelpers.contrib.openstack.cert_utils.install_certs when related to
    the certificates relation with vault.

    The same bug exists in
    charms.openstack.charms.classes.OpenStackCharm.get_certs_and_keys when
    the certificates_interface is in use.

This change applies the same fix.

Change-Id: I530d1efd2f847ec282a6b9cc7c4bfcf7b317b42d
Closes-Bug: #1813982

Changed in charms.openstack:
status:	In Progress → Fix Released

David Ames (thedac) on 2019-02-02

Changed in charm-gnocchi:
assignee:	Frode Nordahl (fnordahl) → David Ames (thedac)
status:	Fix Committed → Triaged
Changed in charm-aodh:
assignee:	Frode Nordahl (fnordahl) → David Ames (thedac)
status:	Fix Committed → Triaged
Changed in charm-barbican:
assignee:	Frode Nordahl (fnordahl) → David Ames (thedac)
status:	Fix Released → Triaged
Changed in charm-designate:
assignee:	Frode Nordahl (fnordahl) → David Ames (thedac)
status:	Fix Committed → Triaged
Changed in charm-manila:
assignee:	Frode Nordahl (fnordahl) → David Ames (thedac)
status:	Fix Released → Triaged
Changed in charm-nova-cell-controller:
assignee:	Frode Nordahl (fnordahl) → David Ames (thedac)
status:	Fix Released → Triaged
Changed in charm-octavia:
assignee:	Frode Nordahl (fnordahl) → David Ames (thedac)
status:	Fix Released → Triaged

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-02: Fix proposed to charm-gnocchi (master)

#41

Fix proposed to branch: master
Review: https://review.openstack.org/634522

Changed in charm-gnocchi:
status:	Triaged → In Progress
Changed in charm-aodh:
status:	Triaged → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-02: Fix proposed to charm-aodh (master)

#42

Fix proposed to branch: master
Review: https://review.openstack.org/634523

Changed in charm-barbican:
status:	Triaged → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-02: Fix proposed to charm-barbican (master)

#43

Fix proposed to branch: master
Review: https://review.openstack.org/634524

Changed in charm-designate:
status:	Triaged → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-02: Fix proposed to charm-designate (master)

#44

Fix proposed to branch: master
Review: https://review.openstack.org/634525

Changed in charm-manila:
status:	Triaged → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-02: Fix proposed to charm-manila (master)

#45

Fix proposed to branch: master
Review: https://review.openstack.org/634526

Changed in charm-nova-cell-controller:
status:	Triaged → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-02: Fix proposed to charm-nova-cell-controller (master)

#46

Fix proposed to branch: master
Review: https://review.openstack.org/634527

Changed in charm-octavia:
status:	Triaged → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-02: Fix proposed to charm-octavia (master)

#47

Fix proposed to branch: master
Review: https://review.openstack.org/634528

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-02: Fix merged to charm-nova-cell-controller (master)

#48

Reviewed: https://review.openstack.org/634527
Committed: https://git.openstack.org/cgit/openstack/charm-nova-cell-controller/commit/?id=5d09b43119c268f40e748f993de350f8105d887e
Submitter: Zuul
Branch: master

commit 5d09b43119c268f40e748f993de350f8105d887e
Author: David Ames <email address hidden>
Date: Fri Feb 1 16:13:27 2019 -0800

Rebuild for sync charms.openstck

Change-Id: Ic838ffb5e042cc93ef211d17dbca6acc2e633d73
Closes-Bug: #1813982

Changed in charm-nova-cell-controller:
status:	In Progress → Fix Committed
Changed in charm-designate:
status:	In Progress → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-02: Fix merged to charm-designate (master)

#49

Reviewed: https://review.openstack.org/634525
Committed: https://git.openstack.org/cgit/openstack/charm-designate/commit/?id=b6f6dd7e3c28bd9f9a2dde49557486cca1a1cef6
Submitter: Zuul
Branch: master

commit b6f6dd7e3c28bd9f9a2dde49557486cca1a1cef6
Author: David Ames <email address hidden>
Date: Fri Feb 1 16:13:13 2019 -0800

Rebuild for sync charms.openstck

Change-Id: I43904f57e4b9e6dc9218fc45755c7c3072403483
Closes-Bug: #1813982

Changed in charm-barbican:
status:	In Progress → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-02: Fix merged to charm-barbican (master)

#50

Reviewed: https://review.openstack.org/634524
Committed: https://git.openstack.org/cgit/openstack/charm-barbican/commit/?id=2cdcea8d1589bf7a42e648fcd26bfcc42f169833
Submitter: Zuul
Branch: master

commit 2cdcea8d1589bf7a42e648fcd26bfcc42f169833
Author: David Ames <email address hidden>
Date: Fri Feb 1 16:13:06 2019 -0800

Rebuild for sync charms.openstck

Change-Id: Ib61d3a0087d8441af9e4a34056e548e5b66572ba
Closes-Bug: #1813982

Changed in charm-manila:
status:	In Progress → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-02: Fix merged to charm-manila (master)

#51

Reviewed: https://review.openstack.org/634526
Committed: https://git.openstack.org/cgit/openstack/charm-manila/commit/?id=5fc15ee7a4afd304e130273375a24dab43131ac6
Submitter: Zuul
Branch: master

commit 5fc15ee7a4afd304e130273375a24dab43131ac6
Author: David Ames <email address hidden>
Date: Fri Feb 1 16:13:20 2019 -0800

Rebuild for sync charms.openstck

Change-Id: Ib20fb9182a411db5deaa81d448b315e867be7e67
Closes-Bug: #1813982

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-02: Fix merged to charm-octavia (master)

#52

Reviewed: https://review.openstack.org/634528
Committed: https://git.openstack.org/cgit/openstack/charm-octavia/commit/?id=920c630a9e50c43685004e07eb0358df4644c3b6
Submitter: Zuul
Branch: master

commit 920c630a9e50c43685004e07eb0358df4644c3b6
Author: David Ames <email address hidden>
Date: Fri Feb 1 16:13:35 2019 -0800

Rebuild for sync charms.openstck

Change-Id: Ie4a122adfef06ad3f3ba0c91b1e3d3d686d2642d
Closes-Bug: #1813982

Changed in charm-octavia:
status:	In Progress → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-02: Fix proposed to charm-nova-cell-controller (stable/18.11)

#53

Fix proposed to branch: stable/18.11
Review: https://review.openstack.org/634566

Changed in charm-gnocchi:
status:	In Progress → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-02: Fix merged to charm-gnocchi (master)

#54

Reviewed: https://review.openstack.org/634522
Committed: https://git.openstack.org/cgit/openstack/charm-gnocchi/commit/?id=7f1ea17ec4bc2f2448008a43a462b927b3fb2e43
Submitter: Zuul
Branch: master

commit 7f1ea17ec4bc2f2448008a43a462b927b3fb2e43
Author: David Ames <email address hidden>
Date: Fri Feb 1 16:12:51 2019 -0800

Rebuild for sync charms.openstck

Change-Id: Iecd65ae70b7a2672306a33801836f365a8dfc6d1
Closes-Bug: #1813982

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-02: Fix proposed to charm-designate (stable/18.11)

#55

Fix proposed to branch: stable/18.11
Review: https://review.openstack.org/634567

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-02: Fix proposed to charm-barbican (stable/18.11)

#56

Fix proposed to branch: stable/18.11
Review: https://review.openstack.org/634568

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-02: Fix proposed to charm-manila (stable/18.11)

#57

Fix proposed to branch: stable/18.11
Review: https://review.openstack.org/634569

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-02: Fix proposed to charm-octavia (stable/18.11)

#58

Fix proposed to branch: stable/18.11
Review: https://review.openstack.org/634570

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-02: Fix proposed to charm-gnocchi (stable/18.11)

#59

Fix proposed to branch: stable/18.11
Review: https://review.openstack.org/634571

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-04: Fix merged to charm-aodh (master)

#60

Reviewed: https://review.openstack.org/634523
Committed: https://git.openstack.org/cgit/openstack/charm-aodh/commit/?id=ee68a502a36fdc25fe95faa4b003908c5a9db325
Submitter: Zuul
Branch: master

commit ee68a502a36fdc25fe95faa4b003908c5a9db325
Author: David Ames <email address hidden>
Date: Fri Feb 1 16:12:59 2019 -0800

Rebuild for sync charms.openstck

Additionally adds initialization for ceilometer
when requiring action be taken to unblock.

Change-Id: Iac55ca7323c201be8b6a9299946d196b2eb57572
Closes-Bug: #1813982

Changed in charm-aodh:
status:	In Progress → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-04: Fix proposed to charm-aodh (stable/18.11)

#61

Fix proposed to branch: stable/18.11
Review: https://review.openstack.org/634696

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-04: Fix merged to charm-barbican (stable/18.11)

#62

Reviewed: https://review.openstack.org/634568
Committed: https://git.openstack.org/cgit/openstack/charm-barbican/commit/?id=d75350838a2c17bf9f094cda39f0ed6ccf6cc1ac
Submitter: Zuul
Branch: stable/18.11

commit d75350838a2c17bf9f094cda39f0ed6ccf6cc1ac
Author: David Ames <email address hidden>
Date: Fri Feb 1 16:13:06 2019 -0800

Rebuild for sync charms.openstck

    Change-Id: Ib61d3a0087d8441af9e4a34056e548e5b66572ba
    Closes-Bug: #1813982
    (cherry picked from commit 2cdcea8d1589bf7a42e648fcd26bfcc42f169833)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-04: Fix merged to charm-octavia (stable/18.11)

#63

Reviewed: https://review.openstack.org/634570
Committed: https://git.openstack.org/cgit/openstack/charm-octavia/commit/?id=9e47becae622a17fdc09dd173fe9bf7e9777184b
Submitter: Zuul
Branch: stable/18.11

commit 9e47becae622a17fdc09dd173fe9bf7e9777184b
Author: David Ames <email address hidden>
Date: Fri Feb 1 16:13:35 2019 -0800

Rebuild for sync charms.openstck

    Change-Id: Ie4a122adfef06ad3f3ba0c91b1e3d3d686d2642d
    Closes-Bug: #1813982
    (cherry picked from commit 920c630a9e50c43685004e07eb0358df4644c3b6)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-04: Fix merged to charm-nova-cell-controller (stable/18.11)

#64

Reviewed: https://review.openstack.org/634566
Committed: https://git.openstack.org/cgit/openstack/charm-nova-cell-controller/commit/?id=75744256afb7a7b4343bc432b3e87dd3802ea816
Submitter: Zuul
Branch: stable/18.11

commit 75744256afb7a7b4343bc432b3e87dd3802ea816
Author: David Ames <email address hidden>
Date: Fri Feb 1 16:13:27 2019 -0800

Rebuild for sync charms.openstck

    Change-Id: Ic838ffb5e042cc93ef211d17dbca6acc2e633d73
    Closes-Bug: #1813982
    (cherry picked from commit 5d09b43119c268f40e748f993de350f8105d887e)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-04: Fix merged to charm-manila (stable/18.11)

#65

Reviewed: https://review.openstack.org/634569
Committed: https://git.openstack.org/cgit/openstack/charm-manila/commit/?id=b10f8995c196fd4ef2ab987a933738b5a594c5fb
Submitter: Zuul
Branch: stable/18.11

commit b10f8995c196fd4ef2ab987a933738b5a594c5fb
Author: David Ames <email address hidden>
Date: Fri Feb 1 16:13:20 2019 -0800

Rebuild for sync charms.openstck

    Change-Id: Ib20fb9182a411db5deaa81d448b315e867be7e67
    Closes-Bug: #1813982
    (cherry picked from commit 5fc15ee7a4afd304e130273375a24dab43131ac6)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-04: Fix merged to charm-designate (stable/18.11)

#66

Reviewed: https://review.openstack.org/634567
Committed: https://git.openstack.org/cgit/openstack/charm-designate/commit/?id=7169a56ce10d2ffe7c28b4dc653aa441717eda7a
Submitter: Zuul
Branch: stable/18.11

commit 7169a56ce10d2ffe7c28b4dc653aa441717eda7a
Author: David Ames <email address hidden>
Date: Fri Feb 1 16:13:13 2019 -0800

Rebuild for sync charms.openstck

    Change-Id: I43904f57e4b9e6dc9218fc45755c7c3072403483
    Closes-Bug: #1813982
    (cherry picked from commit b6f6dd7e3c28bd9f9a2dde49557486cca1a1cef6)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-02-04: Fix merged to charm-aodh (stable/18.11)

#67

Reviewed: https://review.openstack.org/634696
Committed: https://git.openstack.org/cgit/openstack/charm-aodh/commit/?id=f3e92848b4da5fc1ed22a33f0755b208dfc9e491
Submitter: Zuul
Branch: stable/18.11

commit f3e92848b4da5fc1ed22a33f0755b208dfc9e491
Author: David Ames <email address hidden>
Date: Fri Feb 1 16:12:59 2019 -0800

Rebuild for sync charms.openstck

    Change-Id: Iac55ca7323c201be8b6a9299946d196b2eb57572
    Closes-Bug: #1813982
    (cherry picked from commit ee68a502a36fdc25fe95faa4b003908c5a9db325)
    (note that only the update to rebuild was cherry-picked)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-03-12: Change abandoned on charm-gnocchi (stable/18.11)

#68

Change abandoned by Frode Nordahl (<email address hidden>) on branch: stable/18.11
Review: https://review.openstack.org/634571
Reason: Superseded by rebuild done in https://review.openstack.org/#/c/640073/

James Page (james-page) on 2019-04-17

Changed in charm-aodh:
milestone:	none → 19.04
Changed in charm-barbican:
milestone:	none → 19.04
Changed in charm-designate:
milestone:	none → 19.04
Changed in charm-gnocchi:
milestone:	none → 19.04
Changed in charm-manila:
milestone:	none → 19.04

James Page (james-page) on 2019-04-17

Changed in charm-octavia:
milestone:	none → 19.04

David Ames (thedac) on 2019-04-17

Changed in charm-aodh:
status:	Fix Committed → Fix Released
Changed in charm-barbican:
status:	Fix Committed → Fix Released
Changed in charm-designate:
status:	Fix Committed → Fix Released
Changed in charm-gnocchi:
status:	Fix Committed → Fix Released
Changed in charm-manila:
status:	Fix Committed → Fix Released
Changed in charm-octavia:
status:	Fix Committed → Fix Released

James Page (james-page) on 2019-08-07

Changed in charm-nova-cell-controller:
milestone:	none → 19.07

David Ames (thedac) on 2019-08-12

Changed in charm-nova-cell-controller:
status:	Fix Committed → Fix Released

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.