mask_passwords doesn't mask characters following a quote or double quote

Bug #1949623 reported by Herve Beraud
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
oslo.utils
Fix Released
Undecided
Herve Beraud

Bug Description

Description of problem:
mask_passwords doesn't mask characters following a " so if a user has a password containing a " in the middle such as pass"word , we would see ***"word in the debug output.

It looks like something's wrong with this:
_FORMAT_PATTERNS_1 = [r'(%(key)s[0-9]*\s*[=]\s*)[^\s^\'^\"]+']
_FORMAT_PATTERNS_2 = [r'(%(key)s[0-9]*\s*[=]\s*[\"\'])[^\"\']*([\"\'])',
                      r'(%(key)s[0-9]*\s+[\"\'])[^\"\']*([\"\'])',
                      r'([-]{2}%(key)s[0-9]*\s+)[^\'^\"^=^\s]+([\s]*)',
                      r'(<%(key)s[0-9]*>)[^<]*(</%(key)s[0-9]*>)',
                      r'([\"\']%(key)s[0-9]*[\"\']\s*:\s*[\"\'])[^\"\']*'
                      r'([\"\'])',
                      r'([\'"][^"\']*%(key)s[0-9]*[\'"]\s*:\s*u?[\'"])[^\"\']*'
                      r'([\'"])',
                      r'([\'"][^\'"]*%(key)s[0-9]*[\'"]\s*,\s*\'--?[A-z]+'
                      r'\'\s*,\s*u?[\'"])[^\"\']*([\'"])',
                      r'(%(key)s[0-9]*\s*--?[A-z]+\s*)\S+(\s*)']

Version-Release number of selected component (if applicable):
All

How reproducible:
Always

Steps to Reproduce:
1. Set a password containing a " somewhere in it
2. Use openstack --debug CLI
3.

Actual results:
Password are not fully hidden

Expected results:
No password should leak in debug output

Additional informations:

We can also reproduce it by adding a dedicated unit tests:

```
        payload = 'test = "original_password" : "aaaaa"aaaa"'
        expected = 'test = "original_password" : "***"'
        self.assertEqual(expected, strutils.mask_password(payload))
```

It will provide the following outcome:

```oslo_utils.tests.test_strutils.MaskPasswordTestCase.test_mask_password
----------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):

      File "/home/hberaud/dev/oslo/oslo.utils/oslo_utils/tests/test_strutils.py", line 612, in test_mask_password
    self.assertEqual(expected, strutils.mask_password(payload))

      File "/home/hberaud/dev/oslo.utils/.tox/py38/lib/python3.8/site-packages/testtools/testcase.py", line 393, in assertEqual
    self.assertThat(observed, matcher, message)

      File "/home/hberaud/dev/oslo.utils/.tox/py38/lib/python3.8/site-packages/testtools/testcase.py", line 480, in assertThat
    raise mismatch_error

    testtools.matchers._impl.MismatchError: !=:
reference = 'test = "original_password" : "***"'
actual = 'test = "original_password" : "***"aaaa"'
```

I'll debug the regex to see how we can fix that.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.utils (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/oslo.utils/+/816544

Changed in oslo.utils:
status: New → In Progress
Changed in oslo.utils:
assignee: nobody → Herve Beraud (herveberaud)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.utils (master)

Reviewed: https://review.opendev.org/c/openstack/oslo.utils/+/816544
Committed: https://opendev.org/openstack/oslo.utils/commit/6e17ae1f7959c64dfd20a5f67edf422e702426aa
Submitter: "Zuul (22348)"
Branch: master

commit 6e17ae1f7959c64dfd20a5f67edf422e702426aa
Author: Hervé Beraud <email address hidden>
Date: Wed Nov 3 17:16:50 2021 +0100

    Fix regex used to mask password

    Some use cases are poorly handled by the regex used
    to mask password. Indeed when the password contains
    quotes or double quotes in the middle such as `pass"word`,
    the mask_password method will return `***"word`.

    For more details please see
    https://bugs.launchpad.net/oslo.utils/+bug/1949623

    Closes-Bug: #1949623
    Change-Id: I941750b4d49d2d75f0831b24d6dd17f4040f70a2

Changed in oslo.utils:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.utils (stable/xena)

Fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/oslo.utils/+/824510

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.utils (stable/xena)

Reviewed: https://review.opendev.org/c/openstack/oslo.utils/+/824510
Committed: https://opendev.org/openstack/oslo.utils/commit/5ce8a7f0f8ecec7a85a23ec3d7a7fb1cad14ceba
Submitter: "Zuul (22348)"
Branch: stable/xena

commit 5ce8a7f0f8ecec7a85a23ec3d7a7fb1cad14ceba
Author: Hervé Beraud <email address hidden>
Date: Wed Nov 3 17:16:50 2021 +0100

    Fix regex used to mask password

    Some use cases are poorly handled by the regex used
    to mask password. Indeed when the password contains
    quotes or double quotes in the middle such as `pass"word`,
    the mask_password method will return `***"word`.

    For more details please see
    https://bugs.launchpad.net/oslo.utils/+bug/1949623

    Closes-Bug: #1949623
    Change-Id: I941750b4d49d2d75f0831b24d6dd17f4040f70a2
    (cherry picked from commit 6e17ae1f7959c64dfd20a5f67edf422e702426aa)

tags: added: in-stable-xena
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.utils (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/oslo.utils/+/824703

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.utils (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/oslo.utils/+/824703
Committed: https://opendev.org/openstack/oslo.utils/commit/143d3fbfa1e04778884de5acc08fa6f7fdabb265
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 143d3fbfa1e04778884de5acc08fa6f7fdabb265
Author: Hervé Beraud <email address hidden>
Date: Wed Nov 3 17:16:50 2021 +0100

    Fix regex used to mask password

    Some use cases are poorly handled by the regex used
    to mask password. Indeed when the password contains
    quotes or double quotes in the middle such as `pass"word`,
    the mask_password method will return `***"word`.

    For more details please see
    https://bugs.launchpad.net/oslo.utils/+bug/1949623

    Closes-Bug: #1949623
    Change-Id: I941750b4d49d2d75f0831b24d6dd17f4040f70a2
    (cherry picked from commit 6e17ae1f7959c64dfd20a5f67edf422e702426aa)
    (cherry picked from commit 5ce8a7f0f8ecec7a85a23ec3d7a7fb1cad14ceba)

tags: added: in-stable-wallaby
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.utils (stable/victoria)

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/oslo.utils/+/824912

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/oslo.utils 4.10.1

This issue was fixed in the openstack/oslo.utils 4.10.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/oslo.utils 4.12.1

This issue was fixed in the openstack/oslo.utils 4.12.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.utils (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/oslo.utils/+/824912
Committed: https://opendev.org/openstack/oslo.utils/commit/65aa4226d4bf286468da6df6759e364c1eb3e049
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 65aa4226d4bf286468da6df6759e364c1eb3e049
Author: Hervé Beraud <email address hidden>
Date: Wed Nov 3 17:16:50 2021 +0100

    Fix regex used to mask password

    Some use cases are poorly handled by the regex used
    to mask password. Indeed when the password contains
    quotes or double quotes in the middle such as `pass"word`,
    the mask_password method will return `***"word`.

    For more details please see
    https://bugs.launchpad.net/oslo.utils/+bug/1949623

    Closes-Bug: #1949623
    Change-Id: I941750b4d49d2d75f0831b24d6dd17f4040f70a2
    (cherry picked from commit 6e17ae1f7959c64dfd20a5f67edf422e702426aa)
    (cherry picked from commit 5ce8a7f0f8ecec7a85a23ec3d7a7fb1cad14ceba)
    (cherry picked from commit 143d3fbfa1e04778884de5acc08fa6f7fdabb265)

tags: added: in-stable-victoria
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.utils (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/oslo.utils/+/827041

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.utils (stable/ussuri)

Reviewed: https://review.opendev.org/c/openstack/oslo.utils/+/827041
Committed: https://opendev.org/openstack/oslo.utils/commit/5e9dfa3408a6fac7e870a9b86ee774b5fd71a595
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit 5e9dfa3408a6fac7e870a9b86ee774b5fd71a595
Author: Hervé Beraud <email address hidden>
Date: Wed Nov 3 17:16:50 2021 +0100

    Fix regex used to mask password

    Some use cases are poorly handled by the regex used
    to mask password. Indeed when the password contains
    quotes or double quotes in the middle such as `pass"word`,
    the mask_password method will return `***"word`.

    For more details please see
    https://bugs.launchpad.net/oslo.utils/+bug/1949623

    Closes-Bug: #1949623
    Change-Id: I941750b4d49d2d75f0831b24d6dd17f4040f70a2
    (cherry picked from commit 6e17ae1f7959c64dfd20a5f67edf422e702426aa)
    (cherry picked from commit 5ce8a7f0f8ecec7a85a23ec3d7a7fb1cad14ceba)
    (cherry picked from commit 143d3fbfa1e04778884de5acc08fa6f7fdabb265)
    (cherry picked from commit 65aa4226d4bf286468da6df6759e364c1eb3e049)

tags: added: in-stable-ussuri
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.utils (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/c/openstack/oslo.utils/+/827830

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/oslo.utils 4.6.1

This issue was fixed in the openstack/oslo.utils 4.6.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.utils (stable/train)

Reviewed: https://review.opendev.org/c/openstack/oslo.utils/+/827830
Committed: https://opendev.org/openstack/oslo.utils/commit/473f300b07641d79d5253208a4058e773857bec5
Submitter: "Zuul (22348)"
Branch: stable/train

commit 473f300b07641d79d5253208a4058e773857bec5
Author: Hervé Beraud <email address hidden>
Date: Wed Nov 3 17:16:50 2021 +0100

    Fix regex used to mask password

    Some use cases are poorly handled by the regex used
    to mask password. Indeed when the password contains
    quotes or double quotes in the middle such as `pass"word`,
    the mask_password method will return `***"word`.

    For more details please see
    https://bugs.launchpad.net/oslo.utils/+bug/1949623

    Closes-Bug: #1949623
    Change-Id: I941750b4d49d2d75f0831b24d6dd17f4040f70a2
    (cherry picked from commit 6e17ae1f7959c64dfd20a5f67edf422e702426aa)
    (cherry picked from commit 5ce8a7f0f8ecec7a85a23ec3d7a7fb1cad14ceba)
    (cherry picked from commit 143d3fbfa1e04778884de5acc08fa6f7fdabb265)
    (cherry picked from commit 65aa4226d4bf286468da6df6759e364c1eb3e049)
    (cherry picked from commit 5e9dfa3408a6fac7e870a9b86ee774b5fd71a595)

tags: added: in-stable-train
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/oslo.utils 4.8.1

This issue was fixed in the openstack/oslo.utils 4.8.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.utils (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/c/openstack/oslo.utils/+/837676

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on oslo.utils (stable/queens)

Change abandoned by "Roman Lubianyi <email address hidden>" on branch: stable/queens
Review: https://review.opendev.org/c/openstack/oslo.utils/+/837676

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.utils (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/c/openstack/oslo.utils/+/841408

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on oslo.utils (stable/stein)

Change abandoned by "Stephen Finucane <email address hidden>" on branch: stable/stein
Review: https://review.opendev.org/c/openstack/oslo.utils/+/841408
Reason: The CI is badly broken at this point and tbh this branch is likely to be moved to EOL soon enough. Time to close this, I think

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/oslo.utils train-eol

This issue was fixed in the openstack/oslo.utils train-eol release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.