Activity log for bug #1673569

Date Who What changed Old value New value Message
2017-03-16 18:41:24 Matt Riedemann bug added bug
2017-03-16 18:42:05 Matt Riedemann bug task added oslo.messaging
2017-03-16 18:48:28 Jeremy Stanley bug task added ossa
2017-03-16 18:48:52 Jeremy Stanley description Noticed here: http://logs.openstack.org/08/445308/3/check/gate-tempest-dsvm-py35-ubuntu-xenial/7bf0d72/logs/screen-n-api.txt.gz#_2017-03-16_05_31_09_399 I noticed this while investigating public nova bug 1673375, but it looks like that bug is caused by a ValueError coming from the oslo.messaging notification code, related to a circular reference in the json blob: 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging Traceback (most recent call last): 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/oslo_messaging/notify/messaging.py", line 70, in notify 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging retry=retry) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/oslo_messaging/transport.py", line 104, in _send_notification 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging retry=retry) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 509, in send_notification 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging envelope=(version == 2.0), notify=True, retry=retry) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 457, in _send 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging msg = rpc_common.serialize_msg(msg) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/oslo_messaging/_drivers/common.py", line 293, in serialize_msg 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging _MESSAGE_KEY: jsonutils.dumps(raw_msg)} 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/oslo_serialization/jsonutils.py", line 190, in dumps 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging return json.dumps(obj, default=default, **kwargs) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/lib/python3.5/json/__init__.py", line 237, in dumps 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging **kw).encode(obj) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/lib/python3.5/json/encoder.py", line 198, in encode 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging chunks = self.iterencode(o, _one_shot=True) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/lib/python3.5/json/encoder.py", line 256, in iterencode 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging return _iterencode(o, 0) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging ValueError: Circular reference detected 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging The security issue here is that the notification payload that's logged has all kinds of auth secrets in it, like tokens and passwords. From logstash it looks like this is only hitting master (pike) right now. This issue is being treated as a potential security risk under embargo. Please do not make any public mention of embargoed (private) security vulnerabilities before their coordinated publication by the OpenStack Vulnerability Management Team in the form of an official OpenStack Security Advisory. This includes discussion of the bug or associated fixes in public forums such as mailing lists, code review systems and bug trackers. Please also avoid private disclosure to other individuals not already approved for access to this information, and provide this same reminder to those who are made aware of the issue prior to publication. All discussion should remain confined to this private bug report, and any proposed fixes should be added to the bug as attachments. Noticed here: http://logs.openstack.org/08/445308/3/check/gate-tempest-dsvm-py35-ubuntu-xenial/7bf0d72/logs/screen-n-api.txt.gz#_2017-03-16_05_31_09_399 I noticed this while investigating public nova bug 1673375, but it looks like that bug is caused by a ValueError coming from the oslo.messaging notification code, related to a circular reference in the json blob: 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging Traceback (most recent call last): 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/oslo_messaging/notify/messaging.py", line 70, in notify 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging retry=retry) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/oslo_messaging/transport.py", line 104, in _send_notification 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging retry=retry) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 509, in send_notification 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging envelope=(version == 2.0), notify=True, retry=retry) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 457, in _send 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging msg = rpc_common.serialize_msg(msg) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/oslo_messaging/_drivers/common.py", line 293, in serialize_msg 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging _MESSAGE_KEY: jsonutils.dumps(raw_msg)} 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/oslo_serialization/jsonutils.py", line 190, in dumps 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging return json.dumps(obj, default=default, **kwargs) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/lib/python3.5/json/__init__.py", line 237, in dumps 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging **kw).encode(obj) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/lib/python3.5/json/encoder.py", line 198, in encode 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging chunks = self.iterencode(o, _one_shot=True) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/lib/python3.5/json/encoder.py", line 256, in iterencode 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging return _iterencode(o, 0) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging ValueError: Circular reference detected 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging The security issue here is that the notification payload that's logged has all kinds of auth secrets in it, like tokens and passwords. From logstash it looks like this is only hitting master (pike) right now.
2017-03-16 18:49:18 Jeremy Stanley bug added subscriber Oslo Core security contacts
2017-03-16 18:49:35 Jeremy Stanley bug added subscriber Nova Core security contacts
2017-03-16 18:52:16 Jeremy Stanley ossa: status New Incomplete
2017-03-20 19:04:04 Jeremy Stanley ossa: status Incomplete Confirmed
2017-03-21 15:05:14 Jeremy Stanley information type Private Security Public Security
2017-03-21 15:09:20 Matt Riedemann nominated for series nova/mitaka
2017-03-21 15:09:20 Matt Riedemann bug task added nova/mitaka
2017-03-21 15:09:20 Matt Riedemann nominated for series nova/newton
2017-03-21 15:09:20 Matt Riedemann bug task added nova/newton
2017-03-21 15:09:20 Matt Riedemann nominated for series nova/ocata
2017-03-21 15:09:20 Matt Riedemann bug task added nova/ocata
2017-03-21 15:09:32 Matt Riedemann nova: assignee Balazs Gibizer (balazs-gibizer)
2017-03-21 15:09:40 Matt Riedemann nova: status Confirmed Fix Released
2017-03-21 15:10:08 Matt Riedemann bug task deleted oslo.messaging
2017-03-21 15:10:11 Matt Riedemann nova/ocata: importance Undecided High
2017-03-21 15:10:15 Matt Riedemann nova/newton: status New Fix Released
2017-03-21 15:10:20 Matt Riedemann nova/mitaka: assignee Balazs Gibizer (balazs-gibizer)
2017-03-21 15:10:27 Matt Riedemann nova/newton: assignee Balazs Gibizer (balazs-gibizer)
2017-03-21 15:10:34 Matt Riedemann nova/ocata: assignee Balazs Gibizer (balazs-gibizer)
2017-03-21 15:10:39 Matt Riedemann nova/mitaka: importance Undecided High
2017-03-21 15:10:44 Matt Riedemann nova/newton: importance Undecided High
2017-03-21 15:10:51 Matt Riedemann nova/mitaka: status New Fix Released
2017-03-21 15:11:04 Matt Riedemann nova/ocata: status New Fix Released
2017-03-21 15:13:22 Tristan Cacqueray description This issue is being treated as a potential security risk under embargo. Please do not make any public mention of embargoed (private) security vulnerabilities before their coordinated publication by the OpenStack Vulnerability Management Team in the form of an official OpenStack Security Advisory. This includes discussion of the bug or associated fixes in public forums such as mailing lists, code review systems and bug trackers. Please also avoid private disclosure to other individuals not already approved for access to this information, and provide this same reminder to those who are made aware of the issue prior to publication. All discussion should remain confined to this private bug report, and any proposed fixes should be added to the bug as attachments. Noticed here: http://logs.openstack.org/08/445308/3/check/gate-tempest-dsvm-py35-ubuntu-xenial/7bf0d72/logs/screen-n-api.txt.gz#_2017-03-16_05_31_09_399 I noticed this while investigating public nova bug 1673375, but it looks like that bug is caused by a ValueError coming from the oslo.messaging notification code, related to a circular reference in the json blob: 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging Traceback (most recent call last): 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/oslo_messaging/notify/messaging.py", line 70, in notify 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging retry=retry) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/oslo_messaging/transport.py", line 104, in _send_notification 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging retry=retry) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 509, in send_notification 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging envelope=(version == 2.0), notify=True, retry=retry) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 457, in _send 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging msg = rpc_common.serialize_msg(msg) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/oslo_messaging/_drivers/common.py", line 293, in serialize_msg 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging _MESSAGE_KEY: jsonutils.dumps(raw_msg)} 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/oslo_serialization/jsonutils.py", line 190, in dumps 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging return json.dumps(obj, default=default, **kwargs) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/lib/python3.5/json/__init__.py", line 237, in dumps 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging **kw).encode(obj) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/lib/python3.5/json/encoder.py", line 198, in encode 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging chunks = self.iterencode(o, _one_shot=True) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/lib/python3.5/json/encoder.py", line 256, in iterencode 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging return _iterencode(o, 0) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging ValueError: Circular reference detected 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging The security issue here is that the notification payload that's logged has all kinds of auth secrets in it, like tokens and passwords. From logstash it looks like this is only hitting master (pike) right now. Noticed here: http://logs.openstack.org/08/445308/3/check/gate-tempest-dsvm-py35-ubuntu-xenial/7bf0d72/logs/screen-n-api.txt.gz#_2017-03-16_05_31_09_399 I noticed this while investigating public nova bug 1673375, but it looks like that bug is caused by a ValueError coming from the oslo.messaging notification code, related to a circular reference in the json blob: 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging Traceback (most recent call last): 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/oslo_messaging/notify/messaging.py", line 70, in notify 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging retry=retry) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/oslo_messaging/transport.py", line 104, in _send_notification 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging retry=retry) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 509, in send_notification 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging envelope=(version == 2.0), notify=True, retry=retry) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 457, in _send 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging msg = rpc_common.serialize_msg(msg) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/oslo_messaging/_drivers/common.py", line 293, in serialize_msg 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging _MESSAGE_KEY: jsonutils.dumps(raw_msg)} 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/oslo_serialization/jsonutils.py", line 190, in dumps 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging return json.dumps(obj, default=default, **kwargs) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/lib/python3.5/json/__init__.py", line 237, in dumps 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging **kw).encode(obj) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/lib/python3.5/json/encoder.py", line 198, in encode 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging chunks = self.iterencode(o, _one_shot=True) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/lib/python3.5/json/encoder.py", line 256, in iterencode 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging return _iterencode(o, 0) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging ValueError: Circular reference detected 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging The security issue here is that the notification payload that's logged has all kinds of auth secrets in it, like tokens and passwords. From logstash it looks like this is only hitting master (pike) right now.
2017-03-21 17:10:11 Jeremy Stanley ossa: status Confirmed In Progress
2017-03-21 17:33:14 Jeremy Stanley tags security
2017-03-21 20:05:11 Jeremy Stanley summary Failed notification payload is dumped in logs with auth secrets Failed notification payload is dumped in logs with auth secrets (2017-7214)
2017-03-21 20:05:25 Jeremy Stanley summary Failed notification payload is dumped in logs with auth secrets (2017-7214) Failed notification payload is dumped in logs with auth secrets (CVE-2017-7214)
2017-03-21 23:28:18 OpenStack Infra cve linked 2017-7214
2017-03-22 01:06:25 OpenStack Infra tags in-stable-ocata
2017-03-22 01:06:48 OpenStack Infra tags in-stable-ocata in-stable-newton in-stable-ocata
2017-03-22 01:07:09 OpenStack Infra tags in-stable-newton in-stable-ocata in-stable-mitaka in-stable-newton in-stable-ocata
2017-03-22 20:48:03 Jeremy Stanley ossa: status In Progress Fix Committed
2017-03-23 01:36:32 OpenStack Infra ossa: status Fix Committed Fix Released
2017-03-23 14:34:56 Jeremy Stanley summary Failed notification payload is dumped in logs with auth secrets (CVE-2017-7214) [OSSA-2017-002] Failed notification payload is dumped in logs with auth secrets (CVE-2017-7214)
2017-03-23 14:43:25 Jeremy Stanley ossa: importance Undecided Medium
2017-03-23 14:43:33 Jeremy Stanley ossa: assignee Jeremy Stanley (fungi)
2017-05-15 12:30:37 Corey Bryant bug task added cloud-archive
2017-05-15 12:31:03 Corey Bryant nominated for series cloud-archive/mitaka
2017-05-15 12:31:03 Corey Bryant bug task added cloud-archive/mitaka
2017-05-15 12:31:03 Corey Bryant nominated for series cloud-archive/ocata
2017-05-15 12:31:03 Corey Bryant bug task added cloud-archive/ocata
2017-05-15 12:31:03 Corey Bryant nominated for series cloud-archive/newton
2017-05-15 12:31:03 Corey Bryant bug task added cloud-archive/newton
2017-05-15 12:31:28 Corey Bryant bug task added nova (Ubuntu)
2017-05-15 12:31:43 Corey Bryant nominated for series Ubuntu Xenial
2017-05-15 12:31:43 Corey Bryant bug task added nova (Ubuntu Xenial)
2017-05-15 12:31:43 Corey Bryant nominated for series Ubuntu Yakkety
2017-05-15 12:31:43 Corey Bryant bug task added nova (Ubuntu Yakkety)
2017-05-15 12:31:43 Corey Bryant nominated for series Ubuntu Artful
2017-05-15 12:31:43 Corey Bryant bug task added nova (Ubuntu Artful)
2017-05-15 12:31:43 Corey Bryant nominated for series Ubuntu Zesty
2017-05-15 12:31:43 Corey Bryant bug task added nova (Ubuntu Zesty)
2017-05-15 12:52:17 Corey Bryant nova (Ubuntu Artful): status New Fix Released
2017-05-15 13:34:34 Corey Bryant cloud-archive/ocata: status New Fix Released
2017-05-15 13:34:47 Corey Bryant nova (Ubuntu Zesty): status New Fix Released
2017-05-15 17:16:20 Brian Murray nova (Ubuntu Yakkety): status New Fix Committed
2017-05-15 17:16:24 Brian Murray bug added subscriber Ubuntu Stable Release Updates Team
2017-05-15 17:16:31 Brian Murray bug added subscriber SRU Verification
2017-05-15 17:16:37 Brian Murray tags in-stable-mitaka in-stable-newton in-stable-ocata in-stable-mitaka in-stable-newton in-stable-ocata verification-needed
2017-05-16 14:26:17 James Page cloud-archive/newton: status New Fix Committed
2017-05-16 14:26:19 James Page tags in-stable-mitaka in-stable-newton in-stable-ocata verification-needed in-stable-mitaka in-stable-newton in-stable-ocata verification-needed verification-newton-needed
2017-05-30 17:29:45 Corey Bryant tags in-stable-mitaka in-stable-newton in-stable-ocata verification-needed verification-newton-needed in-stable-mitaka in-stable-newton in-stable-ocata verification-newton-done verificationdone
2017-05-30 17:30:09 Corey Bryant tags in-stable-mitaka in-stable-newton in-stable-ocata verification-newton-done verificationdone in-stable-mitaka in-stable-newton in-stable-ocata verification-done verification-newton-done
2017-06-01 18:28:19 Launchpad Janitor nova (Ubuntu Yakkety): status Fix Committed Fix Released
2017-06-01 18:28:31 Brian Murray removed subscriber Ubuntu Stable Release Updates Team
2017-06-05 09:47:21 James Page cloud-archive/newton: status Fix Committed Fix Released