Improper handling of ScaleIO backend credentials

[Eric Harney]

The ScaleIO driver uses the backend storage login and password for authentication for connections to the volume as well as the management API.

https://git.openstack.org/cgit/openstack/cinder/tree/cinder/volume/drivers/dell_emc/scaleio/driver.py?h=13.0.4#n176

https://git.openstack.org/cgit/openstack/cinder/tree/cinder/volume/drivers/dell_emc/scaleio/driver.py?h=13.0.4#n229

This has a few serious implications:
a) A user can create a volume, retrieve the username/password from that volume, and use it to connect to a different tenant's volume. Most drivers create per-volume credentials.

b) A user can create a volume, retrieve the username/password from that volume, and use it to connect to the ScaleIO management API and presumably do lots of things they shouldn't be allowed to. Most drivers create credentials for volumes that are independent of the management credentials.

c) If the password is changed on the backend ScaleIO volumes that are currently being used stop working, because Nova stores the old password in its block_device_mapping table. (Not a security problem other than the fact that it prevents rotation of passwords, but definitely a bug.)

Parts of these issues are separately being looked at in bug 1736773, (which generally advises that in some clouds, only Nova should be able to see connection info, not end users) but the situation there is worse for the ScaleIO driver because most drivers only put usernames/passwords in connection_info that are usable for a single volume, not for the storage backend itself.

[Jeremy Stanley (fungi)]

Since this report concerns a possible security risk, an incomplete security advisory task has been added while the core security reviewers for the affected project or projects confirm the bug and discuss the scope of any vulnerability along with potential solutions.

[Jeremy Stanley (fungi)]

In keeping with recent OpenStack vulnerability management policy changes, no report should remain under private embargo for more than 90 days. Because this report predates the change in policy, the deadline for public disclosure is being set to 90 days from today. If the report is not resolved within the next 90 days, it will revert to our public workflow as of 2020-05-27. Please see http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012721.html for further details.

[Jeremy Stanley (fungi)]

It doesn't look like this report has seen any activity since my update two months ago, so consider this a friendly reminder:

The embargo for this report is due to expire one month from today, on May 27, and will be switched public on or shortly after that day if it is not already resolved sooner.

Thanks!

[Sean McGinnis (sean-mcginnis)]

Checking with the team, it sounds like this may be a limitation with how this storage works. Whether the product can be changed to address this has yet to be determined. The "fix" for this may end up needing to be a documentation update alerting operators that this is a concern they need to be aware of.

[Jeremy Stanley (fungi)]

It sounds like this could wind up being a class B1 report (can only be fixed in master) like related bug 1736773, or B2 (vulnerability without a complete fix yet, security note for all versions):

https://security.openstack.org/vmt-process.html#incident-report-taxonomy

If we can get some confirmation that there's unlikely to be any backportable fix on the way, then we may as well switch this to public ahead of the embargo expiration rather than delaying further. Thanks for checking!

[Ivan Pchelintsev (pcheli)]

VxFlex OS does not support multi-tenancy or per-volume authentication yet. We suggest to eliminate passing credentials from Cinder to Nova and storing them in “block_device_mapping” so that users won’t see any. Instead a separate file will be placed on each host. It will contain the same backend credentials parameters from similar sections in cinder.conf, e.g.

[vxflexos-1]
san_password = password1

[vxflexos-2]
san_password = password2
replication_san_password = replication_password2

Whenever Nova tries to connect a block device it will get the storage credentials from the file.

[Jeremy Stanley (fungi)]

If I'm reading correctly, the proposed fix will require configuration changes for existing deployments, so we'll likely want a security note (OSSN) for this but we would not produce an advisory (OSSA).

[Brian Rosmaita (brian-rosmaita)]

@Jeremy and other VMT people: Now that it looks like we're closing in on a solution, how can we handle this? It would be good if we could give operators a heads up before the OSSN is published, because until they both have os-brick patched on all computes and the config file deployed on all hosts with a new password, they can't really fix this (except to change the password early so that no new connections to the backend can be made, which would be pretty disruptive). The statement in the description says "This embargo shall not extend past 2020-05-27 and will be made
public by or on that date if no fix is identified" -- since a fix has been identified, can we extend the embargo a bit?

[Jeremy Stanley (fungi)]

It was meant to read "will be made public by or on that date EVEN if no fix is identified" since the policy change linked in my accompanying comment says:

    Embargoes for privately-reported vulnerabilities shall not last
    more than 90 days, except under unusual circumstances. Following
    the embargo expiration, reports will be publicly visible
    regardless of whether an advisory has been issued.

(now published at https://governance.openstack.org/tc/reference/tags/vulnerability_managed.html#requirements )

We also don't have any process identified for notifying consumers privately to make configuration changes; our OSSA process is focused on applying patches since the primary audience is downstream package maintainers for distributions (though we do have some large public cloud operators subscribed to receive advance copies of those patches for evaluation as well). In the past our policy for fixes requiring configuration changes has been to make them public as quickly as possible so that all operators/users can apply mitigations sooner.

That said, the embargo expiration policy does carve out an exception for "unusual circumstances" and we could attempt to reuse our pre-OSSA notification channel for OSSN guidance (I think it's been done once, several years ago). I still wonder why we need an exception here, since the report sat ignored for 10 months and saw no activity until it was nearing the new embargo expiration. If it wasn't critical enough for anyone to even comment on for nearly a year, is it really so critical that we need to continue to keep it a secret now? And if so, how long past May 27 do you propose extending it the embargo? Three days? A week?

[Brian Rosmaita (brian-rosmaita)]

os-brick patch:
Passes py36,37,38 tests
pep8 problem:
./os_brick/tests/initiator/connectors/test_scaleio.py:49:28: E241 multiple spaces after ':'
            'config_group': 'test',
(please fix or we'll have trouble when we try to merge this)

cinder patch:
passes pep8, py36,37,38, docs, functional-py36,38, pylint
Doc changes look OK, though someone who knows more triple-O should verify the text in the "Using VxFlex OS Storage with a containerized overcloud" section.

I think the code looks OK, but definitely need more eyes on this.

[Jeremy Stanley (fungi)]

Other things to consider: Is this patch backportable (at least as far as stable/stein)? Can someone draft the intended configuration guidance to operators which would be necessary for an OSSN? I assume there's not much point to a typical DevStack+Tempest run with it, but does it still work on the VXFlexOS third-party CI environment with no test regressions?

[Brian Rosmaita (brian-rosmaita)]

Thanks, Jeremy, those are all good points. Here are my thoughts, though I encourage other members of cinder coresec to add comments:

> Is this patch backportable (at least as far as stable/stein)?

It should be a clean backport, plus the change is isolated to vxflexos/scaleio components in both cinder and os-brick, so it would be a very low risk of destabilizing deployments not using that backend.

> Can someone draft the intended configuration guidance to operators which would be necessary for an OSSN?

I can do this as soon as we've reached agreement that this is the correct fix, unless someone at Dell/EMC would prefer to do it. (I think the configuration guidance in the docs change in the cinder patch is very clear.)

> I assume there's not much point to a typical DevStack+Tempest run with it, but does it still work on the VXFlexOS third-party CI environment with no test regressions?

Ivan or someone from the Dell/EMC team: please leave some details about your testing on this patch so we have a record that it doesn't cause a regression. Bonus points if you can provide some indication that it works, e.g., nova bdm record without the password showing, calls to the Block Storage Attachments API showing no password, etc.

[Walt Boring (walter-boring)]

If I read the os-brick patch correctly, it is going to have brick read the username password from a config file. just a reminder that this wont work in the case of a user attaching a volume to a bare metal host unless the user has that password config file available as well. cinder client uses os-brick locally during a user attach from a BM host via the python brick cinder client extension. https://github.com/openstack/python-brick-cinderclient-ext

[Ivan Pchelintsev (pcheli)]

Hi, thanks all for comments/reviews.
> ./os_brick/tests/initiator/connectors/test_scaleio.py:49:28: E241 multiple spaces
Fixed, new patch is attached with the same name.

> just a reminder that this wont work in the case of a user attaching a volume to a bare metal host unless the user has that password config file available as well.
VxFlex OS volumes attach/detach operations do not work out of the box. To attach volume to bare metal host we need to install VxFlexx OS SDC component and create password file.

> Ivan or someone from the Dell/EMC team: please leave some details about your testing on this patch so we have a record that it doesn't cause a regression. Bonus points if you can provide some indication that it works, e.g., nova bdm record without the password showing, calls to the Block Storage Attachments API showing no password, etc.
Attached archive with tempest logs and screenshot from mysql block_device_mapping table.

[Walt Boring (walter-boring)]

So the security issue will still continue when users need to attach volumes to the BM host.

[Brian Rosmaita (brian-rosmaita)]

Good point, Walt. It sounds like we will have to recommend that vxflexos/scaleio not be used with bare metal, because all an attacker would have to do is boot a BM host, and they can obtain the credentials for the entire backend.

Ivan, do you see any way around that?

[Ivan Pchelintsev (pcheli)]

Sorry, can not understand the problem.
Can you give me more details?

[Vladislav Belogrudov (vlad-belogrudov)]

Do you mean Ironic-like cases with normal user BMs? It should not be allowed with the storage. Any host that needs connection to the VxFlex OS storage must have client side software installed and configured as superuser. And for OpenStack there is now additional requirement of the config file with admin credentials. Client side VxFlex software creates virtual block devices on hosts that are passed to VMs.

So I would say no user BMs are supported / recommended. VMs are welcome.

[Brian Rosmaita (brian-rosmaita)]

Vladislav: thanks for confirming.

Ivan: I looked at the tarfile, but the tempest.log is really hard to read about what tests were run and the results. Could you send the console log, or whatever would show that? What I mean is something like:

https://ftp.nimblestorage.com/openstack_ci/iscsi/83/730183/1/check/nimble-iscsi-driver-dsvm-volume/32f9c4e/console.out

About halfway through that file you can see the tempest regex filter being used and then the results of the individual tests. Something like that would be really helpful.

[Jeremy Stanley (fungi)]

As it stands now, this report is still scheduled to switch from Private Security to Public Security tomorrow with the expiration of its embargo. Do we have a reason to extend the embargo and keep it private for longer? This seems to be the current status:

1. There is a solution proposed where, once two patches are applied (comment #6 and comment #15), the operator can configure a safe operating mode for use with virtual server instances.

2. There is no identified solution for using this driver with bare metal instances, nor does one seem to be possible due to the design of the storage platform itself, so the guidance to operators is to not combine them (comment #20).

3. Tests of the proposed patch have been performed in a suitable lab environment, but sufficient evidence has not been provided for reviewers to confirm it's working to their satisfaction (comment #21).

4. Backports of the proposed patch to stable branches have not been provided yet, but it's expected to be trivial to cherry-pick and the implementation is presumed to meet OpenStack's stable branch policy (comment #13).

5. There was a suggestion that users of the driver should be provided an advance copy of the patches and configuration guidance so they will have time to secure their environments before the bug becomes public (comment #9).

If we were to extend the embargo on this report by a week, say to June 3, would that be enough time to address these points? Or will switching to public tomorrow (so patches can be pushed to Gerrit for broader review, more readable third-party test results can be generated and linked, et cetera) be preferable?

[Brian Rosmaita (brian-rosmaita)]

Jeremy, thanks for the update. If we can extend the embargo to June 3, that will be extremely helpful. While drafting the OSSN, I realized that the backports will not be as easy as I thought. To be specific, this problem applies to all stable branches, and we have a rename and some refactoring to deal with. So instead of having a single patch that could be applied to multiple releases, we're going to need multiple patches. The extra time would enable us to have these all ready for operators. I'll provide the details in another comment.

[Brian Rosmaita (brian-rosmaita)]

Draft of the OSSN for this issue attached.

[Brian Rosmaita (brian-rosmaita)]

Here's why the details in the OSSN patch are a TODO:

Cinder changes
--------------
"vxflexos" driver
master, ussuri: current cinder.patch attached to this bug
change to cinder/volume/drivers/dell_emc/vxflexos/rest_client.py
change to cinder/volume/drivers/dell_emc/vxflexos/driver.py

train: no rest client, all changes to be made to cinder/volume/drivers/dell_emc/vxflexos/driver.py

"scaleio" driver
pike, queens, rocky, stein: changes to cinder/volume/drivers/dell_emc/scaleio/driver.py
ocata: will need a different patch for cinder/volume/drivers/dell_emc/scaleio/driver.py due to refactoring in rocky

os-brick changes
----------------
Component was not renamed, so all scaleio.

os_brick/initiator/connectors/scaleio.py
looks like same change in
pike, queens, rocky, stein, train, ussuri, master

os_brick/privileged/scaleio.py
introduced in stein
ocata, pike, queens, rocky: doesn't exist

[Brian Rosmaita (brian-rosmaita)]

Vladislav and Ivan: Please get the info requested in comment #21 so we can verify that the patch isn't causing regressions. Once we're sure that patch will work, you'll need to put up some more branch-specific patches as noted in comment #25.

[Jeremy Stanley (fungi)]

After discussing with the other members of the OpenStack VMT, we've agreed bumping the embargo expiration out an additional week should be fine, so I've updated it to June 3.

[Summer Long (slong-g)]

After reading through, it seems this flaw is CVE-worthy (vs an OSSN / hardening). The main security point is that a user can retrieve their own volume's connection information and then use it to access another user's volume/info.

Per Brian's draft OSSN notes: "When using Cinder with the Dell EMC ScaleIO or VxFlex OS backend storage driver, credentials for the entire backend are exposed in the ``connection_info`` element in all Block Storage v3 Attachments API calls containing that element. This enables an end user to create a volume, make an API call to show the attachment detail information, and retrieve a username and password that may be used to connect to another user's volume. Additionally, these credentials are valid for the ScaleIO or VxFlex OS Management API, should an attacker discover the Management API endpoint."

Will the OpenStack VMT move this to a CVE (OSSA)? Or is there a reason to see/keep this flaw at the hardening level?

[Ivan Pchelintsev (pcheli)]

Sorry for delayed response, had some issues with the environment.
Tempest logs from stable/ussuri with security patch attached.
Working now on backports.

[Jeremy Stanley (fungi)]

Summer: To be handled as an advisory, the patches provided would need to completely fix the flaw, not merely allow an operator to configure the environment so that the flaw was no longer present. Basically we'd need agreement of the driver and Cinder/Brick maintainers as well as the OpenStack stable branch reviewers that such a default behavior change is appropriate to backport to all supported stable branches. In this case, I think it basically means a violation of OpenStack's typical stable branch policies because there would be new configuration settings required for the deployment, effectively breaking the storage backend until they were set. If the vulnerability is so severe that stable deployments are better off taken down until they can be reconfigured, then we should discuss this option further (I just wish people were interested enough in securing this nearly a year ago when the vulnerability was reported rather than waiting until the VMT was ready to make it public so that concerned users could at least stop using this long-vulnerable driver).

[Ivan Pchelintsev (pcheli)]

Train patches for os-brick/cinder with tempest logs attached.

[Brian Rosmaita (brian-rosmaita)]

Just noticed a typo in comment #25:

when I said:
"ocata: will need a different patch for cinder/volume/drivers/dell_emc/scaleio/driver.py due to refactoring in rocky"

I meant "refactoring in pike".

[Summer Long (slong-g)]

Hi Jeremy, absolutely agree that this could have been dealt with sooner. That said, my comment was that the flaw is serious enough to be categorized as a CVE, regardless of how it's dealt with. The patch will at least offer a mitigation. Will you be asking mitre for a CVE? RH can do that once it's public if you want.

[Ivan Pchelintsev (pcheli)]

Stein patch is ready.

[Jeremy Stanley (fungi)]

Summer: Unless the plan changes to provide patches which fix this vulnerability for existing configurations or by breaking deployments until updated configuration is applied, which is likely even impossible for anyone who's been using the driver with bare metal instances unless the vendor alters the design of their product, the OpenStack VMT won't be issuing an advisory; and we only request CVEs from MITRE if an advisory is being drafted.

Publication of guidance to deployers and distributors for securing the software is still encouraged--we have a less formal "security note" process we recommend for this purpose--and if you wish to assign a CVE along with it then feel free to do so. All we ask is that you update this bug report with the CVE identifier so that multiple organizations don't do the same and wind up with duplicate CVEs.

[Brian Rosmaita (brian-rosmaita)]

According to the stable branch policy [0], "OpenStack Vulnerability Management will be reasonable efforts only" for branches in Extended Maintenance mode.

The following cinder and os-brick branches are in EM mode:
- ocata
- pike
- queens
- rocky

I think it is reasonable to push this fix as far back as Queens.

Ocata will take unreasonable effort. The requirements team was able to make some changes to get their Ocata gate moving again so they could push some requirements changes to hopefully unblock everyone else, but it hasn't worked for cinder [1] and os-brick [2], at least, and there seems to be a consensus building on the mailing list that Ocata needs to go EOL [3]. So I will declare right now we do not need to prepare an Ocata patch for this bug.

I'm willing to hear opinions about Pike (my personal opinion is that it is *not* reasonable effort). Right now, the cinder [4] and os-brick [5] gates are broken. The word on the street is that it can be fixed by modifying some of the test jobs to native zuul v3 jobs, but I haven't looked into it because the last change to cinder:stable/pike was 6 months ago, and the last change to os-brick:stable/pike was over 1 year ago. I am dubious that we will get the cinder/os-brick Pike gates working before 3 June (the PTG is next week, which basically leaves us tomorrow to get the gates fixed, and I would rather spend the time reviewing and getting the patches in place for Queens through master (Victoria). Further, there has not been a lot of interest in Pike (as evidenced by lack of backport activity), so I think it makes sense not to put effort into making a patch available for Pike.

So, to be completely clear:
* Ocata - no patch
* Pike - I'm inclined to say no patch, but am interested in other opinions.
* Queens through Victoria - we should have patches available

[0] https://docs.openstack.org/project-team-guide/stable-branches.html#extended-maintenance
[1] https://review.opendev.org/730938
[2] https://review.opendev.org/#/c/731193/
[3] http://lists.openstack.org/pipermail/openstack-discuss/2020-May/015112.html
[4] https://review.opendev.org/730959
[5] https://review.opendev.org/731196

[Jeremy Stanley (fungi)]

There's also the option of just getting the backports through stable/stein ready to push at the same time an OSSN is published, and let any interested parties backport to older branches afterward. If you do mention EM branch commits, the way the VMT usually phrases it is along the lines of, "the stable/rocky branch is under extended maintenance and will receive no new point releases, but a patch for it is provided as a courtesy."

[Ivan Pchelintsev (pcheli)]

Rocky patches.

[Sean McGinnis (sean-mcginnis)]

I agree with Brian's assessment.

We just need Queens yet, and we should be ready to move ahead.

If we can get a patch for Pike, great. If that becomes too difficult due to code changes or other factors, I think we're fine.

Ocata is definitely too old and is likely to go EOL soon. Again, if a patch is available, that's a bonus. But I don't see that as a requirement.

[Brian Rosmaita (brian-rosmaita)]

I'll update the OSSN draft later today and get these patches reviewed (though of course, everyone should feel free to review/test them!).

With the PTG next week, we should figure out the timing of this. Jeremy, can you suggest how this should go?

rosmaita
- git review of OSSN-0086 <-- what time should I submit this?
- after it is approved, will paste the text into the wiki and send a message to the ML

pcheli
- have patched local branches set up so that you can do 'git review' from each one
- should there be a patch directly to ussuri, or will we backport the patch from master?
- rosmaita and smcginnis - be available to approve patches as they are proposed

Ivan, what time zone are you in?

[Corey Bryant (corey.bryant)]

The verification of the Stable Release Update for python-os-brick has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Corey Bryant (corey.bryant)]

This bug was fixed in the package python-os-brick - 3.0.1-0ubuntu1.2~cloud0
---------------

 python-os-brick (3.0.1-0ubuntu1.2~cloud0) bionic-ussuri; urgency=medium
 .
   * New update for the Ubuntu Cloud Archive.
 .
 python-os-brick (3.0.1-0ubuntu1.2) focal-security; urgency=medium
 .
   * d/gbp.conf: Create stable/ussuri branch.
   * SECURITY UPDATE: Dell EMC ScaleIO/VxFlex OS Backend Credentials Exposure
     (LP: #1823200)
     - debian/patches/CVE-2020-10755.patch: Remove VxFlex OS credentials
       from connection_properties. Passwords are now stored in separate file
       and are retrieved during each attach/detach operation.
     - CVE-2020-10755

[Corey Bryant (corey.bryant)]

The verification of the Stable Release Update for cinder has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Corey Bryant (corey.bryant)]

This bug was fixed in the package cinder - 2:15.2.0-0ubuntu1~cloud0
---------------

 cinder (2:15.2.0-0ubuntu1~cloud0) bionic-train; urgency=medium
 .
   [ Chris MacNaughton ]
   * New stable point release for OpenStack Train (LP: #1883892)
   * d/control: Align (Build-)Depends with upstream.
 .
   [ Corey Bryant ]
   * SECURITY UPDATE: Dell EMC ScaleIO/VxFlex OS Backend Credentials Exposure
     (LP: #1823200)
     - Remove VxFlex OS credentials from connection_properties. Passwords are
       now stored in separate file and are retrieved during each attach/detach
       operation. Cinder is patched in 15.2.0 stable point release.
     - d/control: Align (Build-)Depends with min version of python3-os-brick
       required to fix credential exposure.
     - CVE-2020-10755

[Corey Bryant (corey.bryant)]

The verification of the Stable Release Update for python-os-brick has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Corey Bryant (corey.bryant)]

This bug was fixed in the package python-os-brick - 2.10.0-0ubuntu1~cloud0.1
---------------

 python-os-brick (2.10.0-0ubuntu1~cloud0.1) bionic-train; urgency=medium
 .
   * d/gbp.conf: Create stable/train branch.
   * SECURITY UPDATE: Dell EMC ScaleIO/VxFlex OS Backend Credentials Exposure
     (LP: #1823200)
     - debian/patches/CVE-2020-10755*.patch: Remove VxFlex OS credentials
       from connection_properties. Passwords are now stored in separate file
       and are retrieved during each attach/detach operation.
     - CVE-2020-10755

[Corey Bryant (corey.bryant)]

Eoan is EOL

[Corey Bryant (corey.bryant)]

The verification of the Stable Release Update for cinder has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Corey Bryant (corey.bryant)]

This bug was fixed in the package cinder - 2:14.1.0-0ubuntu1~cloud0
---------------

 cinder (2:14.1.0-0ubuntu1~cloud0) bionic-stein; urgency=medium
 .
   [ Chris MacNaughton ]
   * New stable point release for OpenStack Stein (LP: #1884028).
 .
   [ Corey Bryant ]
   * SECURITY UPDATE: Dell EMC ScaleIO/VxFlex OS Backend Credentials Exposure
     (LP: #1823200)
     - Remove VxFlex OS credentials from connection_properties. Passwords are
       now stored in separate file and are retrieved during each attach/detach
       operation. Cinder is patched in 14.1.0 stable point release.
     - d/control: Align (Build-)Depends with min version of python3-os-brick
       required to fix credential exposure.
     - CVE-2020-10755

[Corey Bryant (corey.bryant)]

The verification of the Stable Release Update for cinder has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Corey Bryant (corey.bryant)]

This bug was fixed in the package cinder - 2:13.0.9-0ubuntu1~cloud1.1
---------------

 cinder (2:13.0.9-0ubuntu1~cloud1.1) bionic-rocky; urgency=medium
 .
   * SECURITY UPDATE: Dell EMC ScaleIO/VxFlex OS Backend Credentials Exposure
     (LP: #1823200)
     - debian/patches/CVE-2020-10755.patch: Remove VxFlex OS credentials
       from connection_properties. Passwords are now stored in separate file
       and are retrieved during each attach/detach operation.
     - d/control: Align (Build-)Depends with min version of python3-os-brick
       required to fix credential exposure.
     - CVE-2020-10755
   * d/control: Add python3-sqlalchemy-utils Build-Depends to enable successful
     test execution.

[Corey Bryant (corey.bryant)]

The verification of the Stable Release Update for python-os-brick has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Corey Bryant (corey.bryant)]

This bug was fixed in the package python-os-brick - 2.5.3-0ubuntu1~cloud0.1
---------------

 python-os-brick (2.5.3-0ubuntu1~cloud0.1) bionic-rocky; urgency=medium
 .
   * d/gbp.conf: Create stable/rocky branch.
   * SECURITY UPDATE: Dell EMC ScaleIO/VxFlex OS Backend Credentials Exposure
     (LP: #1823200)
     - debian/patches/CVE-2020-10755*.patch: Remove VxFlex OS credentials
       from connection_properties. Passwords are now stored in separate file
       and are retrieved during each attach/detach operation.
     - CVE-2020-10755

[Corey Bryant (corey.bryant)]

This bug was fixed in the package python-os-brick - 2.8.1-0ubuntu1~cloud0.1
---------------

 python-os-brick (2.8.1-0ubuntu1~cloud0.1) bionic-stein; urgency=medium
 .
   * d/gbp.conf: Create stable/stein branch.
   * SECURITY UPDATE: Dell EMC ScaleIO/VxFlex OS Backend Credentials Exposure
     (LP: #1823200)
     - debian/patches/CVE-2020-10755*.patch: Remove VxFlex OS credentials
       from connection_properties. Passwords are now stored in separate file
       and are retrieved during each attach/detach operation.
     - CVE-2020-10755

[Corey Bryant (corey.bryant)]

Hello Eric, or anyone else affected,

Accepted cinder into queens-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:queens-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-queens-needed to verification-queens-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-queens-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Corey Bryant (corey.bryant)]

The verification of the Stable Release Update for cinder has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Corey Bryant (corey.bryant)]

This bug was fixed in the package cinder - 2:12.0.9-0ubuntu1.2~cloud0
---------------

 cinder (2:12.0.9-0ubuntu1.2~cloud0) xenial-queens; urgency=medium
 .
   * New update for the Ubuntu Cloud Archive.
 .
 cinder (2:12.0.9-0ubuntu1.2) bionic-security; urgency=medium
 .
   * SECURITY UPDATE: Dell EMC ScaleIO/VxFlex OS Backend Credentials Exposure
     (LP: #1823200)
     - debian/patches/CVE-2020-10755.patch: Remove VxFlex OS credentials
       from connection_properties. Passwords are now stored in separate file
       and are retrieved during each attach/detach operation.
     - d/control: Align (Build-)Depends with min version of python3-os-brick
       required to fix credential exposure.
     - CVE-2020-10755

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/733615
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=4047948f1ac8055a025972ad73ec3ec421450775
Submitter: Zuul
Branch: stable/pike

commit 4047948f1ac8055a025972ad73ec3ec421450775
Author: Ivan Pchelintsev <email address hidden>
Date: Tue Jun 2 16:23:04 2020 +0300

    Remove VxFlex OS credentials from connection_properties

    VxFlex OS password is not stored in block_device_mapping table. Instead of this
    passwords are stored in separate file and are retrieved during each attach/detach
    operation.

    Closes-Bug: #1823200
    Change-Id: Ib7778ba9d38a68d8b56ca632c5f1c353d55830b0
    (cherry picked from commit 72c63681178286ed9cd1e1ab48969a64b9004d7c)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/733662
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=ba785eef5f515b869c0d68016e84bb74f76ab45e
Submitter: Zuul
Branch: stable/pike

commit ba785eef5f515b869c0d68016e84bb74f76ab45e
Author: Ivan Pchelintsev <email address hidden>
Date: Thu Jun 4 19:57:56 2020 +0300

    Remove VxFlex OS credentials from connection_properties

    VxFlex OS password is not stored in block_device_mapping table.
    Instead of this passwords are stored in separate file and are
    retrieved during each attach/detach operation.

    The stable/pike branch is in Extended Maintenance mode and is no
    longer released from. If you decide to apply this patch to your
    deployment, you must make a corresponding change to the os-brick
    library. You can find the os-brick patch for stable/pike here:
        https://review.opendev.org/#/c/733615

    Additionally, you must deploy a new configuration file on compute
    nodes, cinder nodes, and anywhere you would perform a volume
    attachment in your deployment. See the documentation change on
    this patch for details about the new config file.

    See OSSN-0086 for more information about this change:
        https://wiki.openstack.org/wiki/OSSN/OSSN-0086

    Change-Id: I975922b2ac951988e51743cd9f2f6f76be520840
    Closes-Bug: #1823200

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/746109

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/746109
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=54504830828757e9d72e9440dde9cff33684a74d
Submitter: Zuul
Branch: master

commit 54504830828757e9d72e9440dde9cff33684a74d
Author: Gorka Eguileor <email address hidden>
Date: Thu Aug 13 13:13:02 2020 +0200

    ScaleIO: Connection info backward compatibility

    When we fixed bug 1823200 in Change-ID
    Iab54c515fe7be252df52b1a0503a251779805759 we made the ScaleIO connector
    incompatible with the old connection properties dictionary as it only
    supported the new 'config_group' and 'failed_over' parameters to get the
    password.

    This is a problem in any system that is upgraded and has attachments to
    the array, because the connection properties of those volumes will not
    contain the new fields and detaching them will result in error
    "KeyError: 'config_group'".

    This patch adds compatibility code to support the old connection
    properties format so we can detach those volumes.

    Related-Bug: #1823200
    Change-Id: I6f01a178616b74ed9a86876ca46e7e46eb360518

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/746572

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/746572
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=31589a624fe8d2ebb56ccbd9c94a8dd559a7da89
Submitter: Zuul
Branch: stable/ussuri

commit 31589a624fe8d2ebb56ccbd9c94a8dd559a7da89
Author: Gorka Eguileor <email address hidden>
Date: Thu Aug 13 13:13:02 2020 +0200

    ScaleIO: Connection info backward compatibility

    When we fixed bug 1823200 in Change-ID
    Iab54c515fe7be252df52b1a0503a251779805759 we made the ScaleIO connector
    incompatible with the old connection properties dictionary as it only
    supported the new 'config_group' and 'failed_over' parameters to get the
    password.

    This is a problem in any system that is upgraded and has attachments to
    the array, because the connection properties of those volumes will not
    contain the new fields and detaching them will result in error
    "KeyError: 'config_group'".

    This patch adds compatibility code to support the old connection
    properties format so we can detach those volumes.

    Related-Bug: #1823200
    Change-Id: I6f01a178616b74ed9a86876ca46e7e46eb360518
    (cherry picked from commit 54504830828757e9d72e9440dde9cff33684a74d)

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/746621

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/746621
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=db95b001e2fe53a71ec0b881407ecdf7c3db32fc
Submitter: Zuul
Branch: stable/train

commit db95b001e2fe53a71ec0b881407ecdf7c3db32fc
Author: Gorka Eguileor <email address hidden>
Date: Thu Aug 13 13:13:02 2020 +0200

    ScaleIO: Connection info backward compatibility

    When we fixed bug 1823200 in Change-ID
    Iab54c515fe7be252df52b1a0503a251779805759 we made the ScaleIO connector
    incompatible with the old connection properties dictionary as it only
    supported the new 'config_group' and 'failed_over' parameters to get the
    password.

    This is a problem in any system that is upgraded and has attachments to
    the array, because the connection properties of those volumes will not
    contain the new fields and detaching them will result in error
    "KeyError: 'config_group'".

    This patch adds compatibility code to support the old connection
    properties format so we can detach those volumes.

    Related-Bug: #1823200
    Change-Id: I6f01a178616b74ed9a86876ca46e7e46eb360518
    (cherry picked from commit 54504830828757e9d72e9440dde9cff33684a74d)
    (cherry picked from commit 31589a624fe8d2ebb56ccbd9c94a8dd559a7da89)
    Conflicts:
            os_brick/initiator/connectors/scaleio.py

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/stein
Review: https://review.opendev.org/749833

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/749833
Committed: https://git.openstack.org/cgit/openstack/os-brick/commit/?id=173601116eb5e00274b10898b56b37dc42d685ac
Submitter: Zuul
Branch: stable/stein

commit 173601116eb5e00274b10898b56b37dc42d685ac
Author: Gorka Eguileor <email address hidden>
Date: Thu Aug 13 13:13:02 2020 +0200

    ScaleIO: Connection info backward compatibility

    When we fixed bug 1823200 in Change-ID
    Iab54c515fe7be252df52b1a0503a251779805759 we made the ScaleIO connector
    incompatible with the old connection properties dictionary as it only
    supported the new 'config_group' and 'failed_over' parameters to get the
    password.

    This is a problem in any system that is upgraded and has attachments to
    the array, because the connection properties of those volumes will not
    contain the new fields and detaching them will result in error
    "KeyError: 'config_group'".

    This patch adds compatibility code to support the old connection
    properties format so we can detach those volumes.

    This patch includes the release note from Change
    Ib98043358d51426ca650104ad59a7e09911ee8e9

    Related-Bug: #1823200
    Change-Id: I6f01a178616b74ed9a86876ca46e7e46eb360518
    (cherry picked from commit 54504830828757e9d72e9440dde9cff33684a74d)
    (cherry picked from commit 31589a624fe8d2ebb56ccbd9c94a8dd559a7da89)
    Conflicts:
            os_brick/initiator/connectors/scaleio.py
    (cherry picked from commit db95b001e2fe53a71ec0b881407ecdf7c3db32fc)

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/750050

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/750051

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/stein
Review: https://review.opendev.org/750052

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/750050
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=7a20d32e85f0c1614a6687eb5142ef2e020f2fd4
Submitter: Zuul
Branch: stable/ussuri

commit 7a20d32e85f0c1614a6687eb5142ef2e020f2fd4
Author: Brian Rosmaita <email address hidden>
Date: Fri Sep 4 17:28:01 2020 -0400

    Require os-brick >= 3.0.3

    This version of brick contains a fix related to OSSN-0086, among
    other things.

    Change-Id: If0216831d616fc9d6c1907fc5ea6e796ab85ac39
    Related-bug: #1823200

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/750051
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=9e3e2ab25dac1e38d2344b0dd115204aa12db971
Submitter: Zuul
Branch: stable/train

commit 9e3e2ab25dac1e38d2344b0dd115204aa12db971
Author: Brian Rosmaita <email address hidden>
Date: Fri Sep 4 17:35:20 2020 -0400

    Require os-brick >= 2.10.5

    This version of brick contains a fix related to OSSN-0086, among
    other things.

    Depends-on: https://review.opendev.org/749905
    Change-Id: I2fb85babb34add6f705adc0e6a2b5b69fef3fbe7
    Related-bug: #1823200

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/750052
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=4cc9e7b5f101fb099d4840e91ee068880e5d29bd
Submitter: Zuul
Branch: stable/stein

commit 4cc9e7b5f101fb099d4840e91ee068880e5d29bd
Author: Brian Rosmaita <email address hidden>
Date: Fri Sep 4 17:41:54 2020 -0400

    Require os-brick >= 2.8.7

    This version of brick contains a fix related to OSSN-0086, among
    other things.

    Depends-on: https://review.opendev.org/750045
    Change-Id: I68130667e9c454b7f4e6d023401796226175c4bd
    Related-bug: #1823200

[Brian Rosmaita (brian-rosmaita)]

I want to add an addendum to comment #55. That roll-out plan worked fine, except that we should have used the same Change-Id on all the cinder patches, and same Change-Id on all the os-brick patches. This would have made it easier for people looking to see which branches contained the fix, because they would have been connected in the way backports usually are.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder queens-eol release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/os-brick queens-eol release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder rocky-eol release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/os-brick rocky-eol release.