cmsfs-fuse fails to mount file systems on Ubuntu 22.04 with error unknown option(s): '-o hard_remove'

[Frank Heimes]

SRU Justification:
==================

[Impact]

 * The cmsfs-fuse utility, that is part of the s390-tools,
   allows to mount z/VM (IBMs commercial hypervisor for IBM Z systems)
   CMS disks.

 * cmsfs-fuse (prior to the migration to fuse3) could use the
   'hard_remove' option without any problem - and it was the default.

 * While migrating to fuse3 all tools were adjusted,
   but it wasn't identified that the 'hard_remove' no longer works
   with fuse3 (only)..

 * Hence if one uses cmsfs-fuse on Ubuntu 22.04, the utility will
   fails with error "fuse: unknown option(s): '-o hard_remove'".

 * The option '-o hard_remove' is (for whatever reason) hard coded.

[Fix]

 * upstream commit 0981df62c05e71123bc14483bba347f17aafa59a "cmsfs-fuse: fix enabling of hard_remove option"

[Test Plan]

 * Setup an Ubuntu 22.04 instance on the z/VM hypervisor
   and make sure the guest aware of at least one CMS formatted z/VM disk.

 * The cmsfs-fuse utility is part of the s390-tools package,
   which is mandatory for an Ubuntu on IBM Z Installation,
   so everything is installed and available with a default installation.

 * This disk needs to be defined in the z/VM guests 'user direct',
   which is actually the guest aka virtual machine definition as minidisk,
   like:
   MDISK 1607 3390 0001 10016 LM1607 RR

 * This disk will then be visible from inside the Linux z/VM guest like:
   $ ls -l /dev/disk/by-path/ccw-0.0.1607
   lrwxrwxrwx 1 root root 11 Jun 8 03:18 /dev/disk/by-path/ccw-0.0.1607 -> ../../dasdd
   (or use 'lsdasd')

 * Now just use 'cmsfs-fuse' to mount that disk:
   $ sudo cmsfs-fuse -t /dev/disk/by-path/ccw-0.0.1607 /mnt
   $

 * If the command comes back like above (on a system with the patch included),
   the mount was successful and one may 'ls' that disk.

 * In case of a broken tool one will get the error msg:
   $ sudo cmsfs-fuse -t -o allow_other /dev/disk/by-path/ccw-0.0.1607 /mnt
   fuse: unknown option(s): `-o hard_remove'
   $

[Where problems could occur]

 * Problem could occur in case this "-o hard_remove" is really
   required, but no longer available.

 * But this is handled by forcing an immediate file removal
   in 'cmsfs_oper_init'.

 * However things could break if cmsfs_oper_init is not properly
   called, but it's done as part of 'fuse_operations cmsfs_oper'.
__________

cmsfs-fuse fails to mount any cms file system since it forces the hard_remove option and it appears as though that option is no longer recognized by fuse 3.

the recreate is very easy as I cannot yet get it to actually work. So, I would make a CMS formatted minidisk available to an Ubuntu 22.04 z/VM virtual machine. Then, I used the following syntax:

cmsfs-fuse -t -o allow_other /dev/disk/by-path/ccw-0.0.0291 /mnt. The result of this command will be
fuse: unknown option(s): `-o hard_remove'

Similarly: cmsfs-fuse -t /dev/disk/by-path/ccw-0.0.0291 /mnt will result in:
fuse: unknown option(s): `-o hard_remove'

At this point, I cannot find an example of the command that will actually allow the minidisk to be mounted.

(originally reported by 'richardflewis')

Upstream reported as: https://github.com/ibm-s390-linux/s390-tools/issues/131
Upstream fixed with: https://github.com/ibm-s390-linux/s390-tools/commit/0981df62c05e71123bc14483bba347f17aafa59a

This but is for getting this included into the Ubuntu package for jammy.

[Frank Heimes (fheimes)]

Test builds of the updated packages (that include fixes for LP#1971993, LP#1960119, LP#1959548, LP#1971959 and LP#1978323) are available here:
https://launchpad.net/~fheimes/+archive/ubuntu/lp1978323/+packages

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "debdiff_signed_kinetic_2.20.0-0ubuntu3_to_2.20.0-0ubuntu4.patch" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

[Simon Chopin (schopin)]

The fix for this bug has been uploaded to Kinetic and the SRU queue for Jammy.

[Launchpad Janitor (janitor)]

This bug was fixed in the package s390-tools - 2.20.0-0ubuntu4

---------------
s390-tools (2.20.0-0ubuntu4) kinetic; urgency=medium

  * Fix chreipl-fcp-mpath (LP: #1971993)
    - Move chreipl-fcp-mpath* from /lib/udev/rules.d to /lib/udev.
    - d/control:
      + Build-Depend on bsdextrautils for hexdump
      + Add multiple explicit Depends on udev
      + s390-tools-chreipl-fcp-mpath: drop unnecessary Depends on lvm2
    - No longer change attributes of chreipl-fcp-mpath-common.sh to 755,
      since only the input script '.in' has a she-bang, but not the '.sh'
      anymore (was done with commit c2f8988).
    - Add d/p/0d15a07-chreipl-fcp-mpath-bundle-a-pre-cooked-man-page.patch
      to bundle a pre-cooked version of the man page for chreipl-fcp-mpath
      Required minor context adjustment for CHANGELOG.md hunk
      and changes in d/rules.
    - Add missing README.md to s390-tools-chreipl-fcp-mpath.doc
      (and with that also the README.md for genprotimg to s390-tools.docs).
  * Add new CPU-MF Counters for new IBM Z hardware (LP: #1960119) by:
    - d/p/2515832-util_arch-Add-IBM-z16-as-known-machine.patch and
    - d/p/cce5f51-cpumf-lscpumf-Add-IBM-z16-extended-counter-set-def.patch
  * Add exploitation support of new IBM Z crypto hardware (LP: #1959548) with:
    - d/p/b16a6d4f-lszcrypt-add-CEX8S-support.patch
    - d/p/bcbb6fca-zcryptstats-add-CEX8-support.patch
    - d/p/4382901d-lszcrypt-show-AP-bus-msg-size-limit-capability.patch
    - d/p/27dce331-lszcrypt-add-support-for-checkstop-state.patch
    - d/p/a29b3c89-lszcrypt-new-options-to-show-only-accel-cca-or-ep11-.patch
    - d/p/a8b0d7ac-lszcrypt-new-options-to-filter-cards-queues-only.patch
    - d/p/46fd42af-lszcrypt-new-option-to-show-the-serial-numbers-of-CC.patch
  * Stabilization of data collection in dbginfo.sh script (LP: #1971959)
    by adding several upstream patches:
    - d/p/*-dbginfo.sh-*.patch
    - whereas one needed minor context adjutment for the CHANGELOG.md hunk:
      d/p/50a4740-dbginfo.sh-replace-which-by-builtin-command-type-for.patch
  * Fix cmsfs-fuse mount failure due to unknown option '-o hard_remove'
    (LP: #1978323) with:
    d/p/0981df6-cmsfs-fuse-fix-enabling-of-hard_remove-option.patch

 -- Frank Heimes <email address hidden> Fri, 20 May 2022 13:48:34 +0200

[Łukasz Zemczak (sil2100)]

This bug could also use a better test case. I think the description mentions some steps to perform, but from the SRU reviewer perspective it is not clear if that is actually the way of verifying that the fix is working.

Can someone update the description?

[Łukasz Zemczak (sil2100)]

Hello Frank, or anyone else affected,

Accepted s390-tools into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/s390-tools/2.20.0-0ubuntu3.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Frank Heimes (fheimes)]

$ lszdev dasd-eckd 1607
TYPE ID ON PERS NAMES
dasd-eckd 0.0.1607 yes yes dasdd
$ lsdasd
Bus-ID Status Name Device Type BlkSz Size Blocks
================================================================================
0.0.0200 active dasda 94:0 ECKD 4096 7042MB 1802880
0.0.0300 active dasdb 94:4 ECKD 4096 7042MB 1802880
0.0.0400 active dasdc 94:8 ECKD 4096 21128MB 5409000
0.0.1607 active(ro) dasdd 94:12 ECKD 4096 7043MB 1803060
$ ls -l /dev/disk/by-path/ccw-0.0.1607
lrwxrwxrwx 1 root root 11 Jun 23 11:45 /dev/disk/by-path/ccw-0.0.1607 -> ../../dasdd
$ dpkg -l s390-tools
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==============-=================-============-=============================>
ii s390-tools 2.20.0-0ubuntu3.1 s390x fundamental utilities for Lin>
$ sudo cmsfs-fuse -t /dev/disk/by-path/ccw-0.0.1607 /mnt
[sudo] password for ubuntu:
$ sudo ls -l /mnt
total 2021576
-r--r----- 1 root root 340 Dec 13 2020 A20041H3.EXEC
-r--r----- 1 root root 320 Dec 13 2020 A20041H3.UBUNTU
-r--r----- 1 root root 369 May 1 2019 ADD8000.EXEC
-r--r----- 1 root root 369 May 1 2019 ADD8001.EXEC
...

With that, the verification is done!

[Launchpad Janitor (janitor)]

This bug was fixed in the package s390-tools - 2.20.0-0ubuntu3.1

---------------
s390-tools (2.20.0-0ubuntu3.1) jammy; urgency=medium

  * Fix chreipl-fcp-mpath (LP: #1971993)
    - Move chreipl-fcp-mpath* from /lib/udev/rules.d to /lib/udev.
    - d/control:
      + Build-Depend on bsdextrautils for hexdump
      + Add multiple explicit Depends on udev
      + s390-tools-chreipl-fcp-mpath: drop unnecessary Depends on lvm2
    - No longer change attributes of chreipl-fcp-mpath-common.sh to 755,
      since only the input script '.in' has a she-bang, but not the '.sh'
      anymore (was done with commit c2f8988).
    - Add d/p/0d15a07-chreipl-fcp-mpath-bundle-a-pre-cooked-man-page.patch
      to bundle a pre-cooked version of the man page for chreipl-fcp-mpath
      Required minor context adjustment for CHANGELOG.md hunk
      and changes in d/rules.
    - Add missing README.md to s390-tools-chreipl-fcp-mpath.doc
      (and with that also the README.md for genprotimg to s390-tools.docs).
  * Add new CPU-MF Counters for new IBM Z hardware (LP: #1960119) by:
    - d/p/2515832-util_arch-Add-IBM-z16-as-known-machine.patch and
    - d/p/cce5f51-cpumf-lscpumf-Add-IBM-z16-extended-counter-set-def.patch
  * Add exploitation support of new IBM Z crypto hardware (LP: #1959548) with:
    - d/p/b16a6d4f-lszcrypt-add-CEX8S-support.patch
    - d/p/bcbb6fca-zcryptstats-add-CEX8-support.patch
    - d/p/4382901d-lszcrypt-show-AP-bus-msg-size-limit-capability.patch
    - d/p/27dce331-lszcrypt-add-support-for-checkstop-state.patch
    - d/p/a29b3c89-lszcrypt-new-options-to-show-only-accel-cca-or-ep11-.patch
    - d/p/a8b0d7ac-lszcrypt-new-options-to-filter-cards-queues-only.patch
    - d/p/46fd42af-lszcrypt-new-option-to-show-the-serial-numbers-of-CC.patch
  * Stabilization of data collection in dbginfo.sh script (LP: #1971959)
    by adding several upstream patches:
    - d/p/*-dbginfo.sh-*.patch
    - whereas one needed minor context adjutment for the CHANGELOG.md hunk:
      d/p/50a4740-dbginfo.sh-replace-which-by-builtin-command-type-for.patch
  * Fix cmsfs-fuse mount failure due to unknown option '-o hard_remove'
    (LP: #1978323) with:
    d/p/0981df6-cmsfs-fuse-fix-enabling-of-hard_remove-option.patch

 -- Frank Heimes <email address hidden> Fri, 20 May 2022 13:48:34 +0200

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for s390-tools has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.