CephFS authorize fails with unknown cap type

Bug #1847822 reported by Billy Olsen on 2019-10-11
14
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu Cloud Archive
Medium
Unassigned
Queens
Medium
Unassigned
ceph (Ubuntu)
Medium
Billy Olsen
Bionic
Medium
Unassigned

Bug Description

[Impact]

Attempting to provide access to a user within Ceph to a specific mount path fails with unknown cap type. This appears to be due to the monitor not knowing how to validate the caps that are provided with the mount path per upstream bug https://tracker.ceph.com/issues/39395 and subsequent pull requests.

This is fixed in Mimic (13.1.0+) and included in the current Luminous devel release (upcoming 12.2.13).

[Test Case]

Steps to recreate:

1. Install ceph w/ ceph-fs.

2. Mount ceph filesystem and create subdirectory for restricting access
$ ceph-fuse -k /etc/ceph/ceph.client.foo.keyring --id foo -m 10.5.0.5:6789 /mnt/ceph-fs
$ mkdir /mnt/ceph-fs/bar

3. Authorize access for ceph user to rw a directory
$ ceph fs authorize ceph-fs client.foo /bar rw

Expected Results:

The authorize command to succeed

Actual Results:

Error EINVAL: unknown cap type '/bar'

[Regression Potential]

Regression potential is low as this has already been fixed upstream and has seen additional testing without additional problem reports from the change. The change does affect the validation of capabilities, so if a problem were to arise it would likely be in the verification of capabilities when the code is parsing.

[Other Info]

Upstream pull-request: https://github.com/ceph/ceph/pull/28666

Changed in cloud-archive:
status: New → Triaged
importance: Undecided → Medium
Changed in ceph (Ubuntu):
importance: High → Medium
Billy Olsen (billy-olsen) wrote :

This appears to have been broken by commit d63fccb52241a216a08a92e615bcff008d365392 which added a validation for all of the capabilities in this code path. The fs authorize command works by adding a new capability keyed by the path. This capability with the path key was not understood by the standard capability check method being invoked so it returned False to indicate the capabilities are not valid, which results in an EINVAL value.

The patch included in the pull request fixes this by changing to only check the mds and osd capabilities instead of the full set.

Billy Olsen (billy-olsen) wrote :

I've added the regression-update tag due to this being introduced in the 12.2.12 update via the SRU process.

I've also built a package in a PPA with the latest version for testing purposes. You can find it at https://launchpad.net/~billy-olsen/+archive/ubuntu/lp1847822 and can verify that commit 9c18bd4e from the aforementioned PR fixes this.

tags: added: regression-update
Billy Olsen (billy-olsen) wrote :
description: updated

The attachment "bionic patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

tags: added: patch
tags: added: sts-sru-needed
James Page (james-page) on 2019-11-26
Changed in cloud-archive:
status: Triaged → Fix Released
Changed in ceph (Ubuntu):
status: Triaged → Fix Released
Changed in ceph (Ubuntu Bionic):
status: New → Triaged
importance: Undecided → Medium

Hello Billy, or anyone else affected,

Accepted ceph into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ceph/12.2.12-0ubuntu0.18.04.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in ceph (Ubuntu Bionic):
status: Triaged → Fix Committed
tags: added: verification-needed verification-needed-bionic
James Page (james-page) wrote :

Hello Billy, or anyone else affected,

Accepted ceph into queens-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:queens-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-queens-needed to verification-queens-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-queens-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-queens-needed
Corey Bryant (corey.bryant) wrote :

Hello Billy, or anyone else affected,

Accepted ceph into queens-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:queens-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-queens-needed to verification-queens-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-queens-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

gerald.yang (gerald-yang-tw) wrote :

The ceph packages in queens-proposed fixes this issue

Some details on different nodes

On ceph-mon node:

ubuntu@juju-590627-246395-0:~$ apt-cache policy ceph
ceph:
  Installed: 12.2.12-0ubuntu0.18.04.4
  Candidate: 12.2.12-0ubuntu0.18.04.4
  Version table:
 *** 12.2.12-0ubuntu0.18.04.4 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
     12.2.12-0ubuntu0.18.04.3 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages
     12.2.12-0ubuntu0.18.04.2 500
        500 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages
     12.2.4-0ubuntu1 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic/main amd64 Packages
ubuntu@juju-590627-246395-0:~$ ceph fs authorize ceph-fs client.foo / rw
[client.foo]
 key = AQCiAeZd+PTmHhAAh2VDxBu1JJR7qr5lfLavaw==
ubuntu@juju-590627-246395-0:~$ ceph fs authorize ceph-fs client.bar /bar rw
[client.bar]
 key = AQC2COZdxTytDxAAtY3oVcjK8XqzlFf0oRlzWQ==

On ceph-fs node:

ubuntu@juju-590627-246395-7:~$ apt-cache policy ceph
ceph:
  Installed: 12.2.12-0ubuntu0.18.04.4
  Candidate: 12.2.12-0ubuntu0.18.04.4
  Version table:
 *** 12.2.12-0ubuntu0.18.04.4 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
     12.2.12-0ubuntu0.18.04.3 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages
     12.2.12-0ubuntu0.18.04.2 500
        500 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages
     12.2.4-0ubuntu1 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic/main amd64 Packages
ubuntu@juju-590627-246395-7:~$ sudo ceph-fuse -c ceph.conf --id foo -k client.foo.keyring mnt
ceph-fuse[4198]: starting ceph client
2019-12-03 07:01:22.855304 7fb6a87f6500 -1 init, newargv = 0x559f708262e0 newargc=9
ceph-fuse[4198]: starting fuse
ubuntu@juju-590627-246395-7:~$ ls mnt/
ubuntu@juju-590627-246395-7:~$ sudo mkdir mnt/bar
ubuntu@juju-590627-246395-7:~$ ls mnt
bar

tags: added: verification-done verification-queens-done
removed: verification-needed verification-queens-needed
tags: added: verification-needed
removed: verification-done
gerald.yang (gerald-yang-tw) wrote :

The ceph packages in bionic-proposed fixes this issue

Some details on different nodes

On ceph-mon node:

ubuntu@juju-aad762-246395-1-0:~$ apt-cache policy ceph
ceph:
  Installed: 12.2.12-0ubuntu0.18.04.4
  Candidate: 12.2.12-0ubuntu0.18.04.4
  Version table:
 *** 12.2.12-0ubuntu0.18.04.4 500
        500 http://us.archive.ubuntu.com/ubuntu bionic-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
     12.2.12-0ubuntu0.18.04.3 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages
     12.2.12-0ubuntu0.18.04.2 500
        500 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages
     12.2.4-0ubuntu1 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic/main amd64 Packages
ubuntu@juju-aad762-246395-1-0:~$ ceph fs authorize ceph-fs client.foo / rw
[client.foo]
 key = AQC8EuZdrBm7ERAA7ZaAruEDz3glRMDnPvmWRQ==
ubuntu@juju-aad762-246395-1-0:~$ ceph fs authorize ceph-fs client.bar /bar rw
[client.bar]
 key = AQA+E+ZdoyyENxAAqNXma0Pdq9CaG0rlLL0otQ==

On ceph-fs node:

ubuntu@juju-aad762-246395-1-7:~$ apt-cache policy ceph
ceph:
  Installed: 12.2.12-0ubuntu0.18.04.4
  Candidate: 12.2.12-0ubuntu0.18.04.4
  Version table:
 *** 12.2.12-0ubuntu0.18.04.4 500
        500 http://us.archive.ubuntu.com/ubuntu bionic-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
     12.2.12-0ubuntu0.18.04.3 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages
     12.2.12-0ubuntu0.18.04.2 500
        500 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages
     12.2.4-0ubuntu1 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic/main amd64 Packages
ubuntu@juju-aad762-246395-1-7:~$ sudo ceph-fuse -c ceph.conf --id foo -k client.foo.keyring mnt
ceph-fuse[2553]: starting ceph client
2019-12-03 07:47:40.128237 7f423c183500 -1 init, newargv = 0x55a9692d02e0 newargc=9
ceph-fuse[2553]: starting fuse
ubuntu@juju-aad762-246395-1-7:~$ ls mnt
ubuntu@juju-aad762-246395-1-7:~$ sudo mkdir mnt/bar
ubuntu@juju-aad762-246395-1-7:~$ ls mnt/
bar

tags: added: verification-done-bionic
removed: verification-needed-bionic
tags: added: verification-done
removed: verification-needed
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers