[19.10 FEAT] Thin provisioning base support (kernel+390tools)

[bugproxy]

Enable use of thin provisioned ECKD volumes.
Efficient use of storage resources and reduced deployment time for ECKD volumes .

Tentative Target is kernel 5.3 + s390tools
Assumption from crystal balls
merge window closes on Sunday, 2019-07-28 and release on Sunday, 2019-09-15

[Frank Heimes (fheimes)]

The target (as of today) for 19.10 is to have:
kernel 5.2
and with that
s390-tools 2.9.0

With that it's unlikely that we include newer kernels or s390-tools versions,
hence the only options seems to be to cherry-pick selected patches.

Please share the selected commit IDs or backports once available,
so that we can verify if they can be included.

Changing to Incomplete - for now.

[Frank Heimes (fheimes)]

Please also share some more details on the DASD/ECKD thin provisioning - thx.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2019-07-15 10:37 EDT-------
Upstream commits on kernel 5.3 , to be applied to 5.2
72ddd535128e4acefaec00d52c75385fd4db40c0 s390/dasd: Remove unused structs and function prototypes
8a9f606fefadb903c323836fcd6e65a122dce66a s390/dasd: Remove old defines and function
ce6915f5343f5f2a2a937b683d8ffbf12dab3ad4 s390/dasd: Make layout analysis ESE compatible
461db0ea03755e0eeb7186f8613ba2291dff7833 s390/dasd: Put sub-order definitions in a separate section
c729696bcf8b23450043dd9c9972c15e53419ae4 s390/dasd: Recognise data for ESE volumes
5e2b17e712cf10cc3cc98fde28a88e8f1a1267e9 s390/dasd: Add dynamic formatting support for ESE volumes
2df4774cb4d1a9280c01bde0f729e56deb36141d s390/dasd: Fix whitespace
d7a4434d6013c750f6c180f9678a870abc21993b s390/dasd: Add missing intensity definition
bcf36768ea688e926b04639c88c77d8caea3b736 s390/dasd: Add dasd_sleep_on_queue_interruptible()
91dc4a197569230683ca8bad551e655a4bf14c30 s390/dasd: Add new ioctl to release space
a0610a8ad80743ff916ebef0490418a298e5bd23 s390/dasd: Make dasd_setup_queue() a discipline function
b54441534e9648f8df24b75a63bb00ed3aa123de s390/dasd: Use ALIGN_DOWN macro
7e64db1597fe114b83fe17d0ba96c6aa5fca419a s390/dasd: Add discard support for ESE volumes
9e12e54c7a8f616190beffb0f7ce778a86aec175 s390/dasd: Handle out-of-space constraint

s390-tools commits will follow, once available

[Frank Heimes (fheimes)]

New planned target kernel for eoan is 5.3.
Leaving as Incomplete for now - will be changed to Fix Committed once 5.3 arrived in eoan-proposed.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2019-08-01 05:48 EDT-------
Requested s390-tools 2.11.0 (for kernel 5.3)

[Dimitri John Ledkov (xnox)]

It's a bit late. Why is s390-tools v2.11 not released yet? 5.3-rc1 is out long time ago.

[Dimitri John Ledkov (xnox)]

If you need v5.3 kernel to test s390-tools against, you can use the kernels from the bootstrap PPA where v5.3 is being prepared. And is regularly reved against -rc kernels. It's currently at -rc4

https://launchpad.net/~canonical-kernel-team/+archive/ubuntu/bootstrap

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2019-08-14 08:43 EDT-------
We, IBM need to leave the s390-tools 2.11.0 open for bugfixes from development until mid of September. We will provide the new level for kernel 5.3 as soon as possible und I will keep you informed about the availability....
If this a Feature Freeze exception, we need to go for it....

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2019-09-09 02:48 EDT-------
Now s390-tools 2.11.0 for kernel 5.3 is available.
https://github.com/ibm-s390-tools/s390-tools/releases/tag/v2.11.0

Please integrate it into Eoan.

[Frank Heimes (fheimes)]

I quickly double checked the kernel commits, almost all got upstream accepted with 5.3-rc2, hence all are incl. in Eoan's master-next tree that contains 5.3-rc8
(that's available from proposed:
 linux-generic | 5.3.0.10.11 | eoan-proposed | s390x)
Hence changing the kernel entry here to Fix Committed.

[Launchpad Janitor (janitor)]

This bug was fixed in the package s390-tools - 2.11.0-0ubuntu1

---------------
s390-tools (2.11.0-0ubuntu1) eoan; urgency=medium

  * New upstream release LP: #1830731
    - update from master snapshot, to v2.11.0 final
    - only minor bugfixes and clarifications included

 -- Dimitri John Ledkov <email address hidden> Tue, 10 Sep 2019 16:36:31 +0100

[Frank Heimes (fheimes)]

Kernel 5.3 finally landed in eoan's release pocket:
linux-generic | 5.3.0.10.11 | eoan | s390x
hence setting kernel status entry to Fix Released - and with that also the project entry.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2019-09-13 04:48 EDT-------
IBM Bugzilla status -> closed, Fix Released with Eoan

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2019-10-01 08:40 EDT-------
Comment from Jan Hoeppner 2019-09-24 07:16:57 CDT

The thin provisioning feature introduces an IOCTL and the discard support to allow userspace tools and filesystems to release unused and previously allocated space respectively.

During some internal performance improvements and further tests, the release of allocated space revealed some issues that may lead to data corruption in some configurations when filesystems are mounted with discard support enabled.

As we're working on a fix and trying to clarify the situation, it is highly recommended at this point to _not_ enable discard for any filesystem
when working with ESE DASDs.
mkfs should also be called with discard disabled for ESE DASDs
(e.g. mkfs.ext4 -E nodiscard /dev/dasdX).

Fixes and clarifications will be provided as soon as possible.

Regards,
Jan

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2019-10-01 09:15 EDT-------
In addition to Jan's update to _not_ enable discard support I would recommend to completely revert the patch:

7e64db1597fe114b83fe17d0ba96c6aa5fca419a s390/dasd: Add discard support for ESE volumes

to prevent the user from enabling the discard support for DASD devices.
This patch will also be reverted upstream until further clarification.

Regards,
Stefan

[Dimitri John Ledkov (xnox)]

Ubuntu enables discard by default in all the installers, all filesystems, and all layers (e.g. mdadm/lvm/luks discard).

Assumption until now, has been that it is harmless, when underlying storage doesn't support discard and always non-harmful when it is supported.

I agree, we should revert discard support for ESE volumes in Ubuntu for now.

[Dimitri John Ledkov (xnox)]

Revert of discard is to be tracked in https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1846219

[Frank Heimes (fheimes)]

Stefan, do you also plan to revert it from upstream - or what's the plan there?

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2019-10-07 12:18 EDT-------
(In reply to comment #24)
> Stefan, do you also plan to revert it from upstream - or what's the plan
> there?

I saw that you guys reverted the patch with LP1846219 already. But for the sake of completeness, Stefan reverted the patch for upstream as well: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=964ce509e2ded52c1a61ad86044cc4d70abd9eb8

Regards,
Jan