DASDFMT fails with a buffer overflow error

[Andy Hartman]

[Impact]

 * In certain hardware configurations, dasdfmt fails to format a drive, thus preventing successful installation and/or initialisation of a new disk.

"there is a bug in the parsing of /proc/dasd/devices which is being triggered only if an FBA device is listed prior to the ECKD device in question. And since you've had FBA devices attached to your system during the installation, you ran into that bug."

[Test Case]

 * Activate on a system and FBA device and a ECKD device
 * Check that FBA device is listed first in /proc/dasd/devices
 * Try to dasdfmt the ECKD device with $ dasdfmt -t /dev/dasdX
 * One can change ECKD to "FBA " in /proc/dasd/devices by bind-mounting an edited file in place to reproduce the bug

[Regression Potential]

 * Minimal. Bug fix cherrypick from upstream.

[Original Description]

During installation, either through the menu or in an ssh session , dasdfmt fails with a buffer overflow error and does not format the dasd volume. The volume is a z/VM minidisk - defined as 1 to end, this is common in this environment. The DASD volume was formatted from z/VM as a minidisk volume. These are the same steps taken for other distributions running under z/VM. I've attached some screen shots.

[dann frazier (dannf)]

Marking s390-tools affected due to image #5:

/dev # dasdfmt -b 4096 /dev/dasdc
*** buffer overflow detected ***: dasdfmt terminated
Aborted
/dev #

[Frank Heimes (fheimes)]

We have similar MDISK setups in our machine, but mostly 3390 0001 10016 (so mod9s),
but did not saw such a overflow situations yet.
We cpfmtxa format our disks once and label them (but with perm 0-end) - are you using cpfmtxa too ?
So 'sudo dasdfmt -b 4096 -f /dev/dasdf' recently worked for me ...

Looks like your 9336 EDEV devices (a and b) are fine.

Please can you paste your MDISK settings of the user direct (at least for the disk 0200),
the 'lsdasd -a' output
and the output of 'dasdfmt --version'?

Thanks

Nevertheless a buffer overflow is not acceptable ...

[Andy Hartman (andy-hartman)]

Hi Frank,

I used ickdsf cpvol format under Zvm (which is what cpfmtxa calls) to format the dasd as perm space (entire volume) before I used it in my Ubuntu guest. This volume is a mod 9.

From Maint630:

Ickdsf
Console
Console
Cpvol format unit(d700) verify(ahd700) volid(ahd700)

Once Ubuntu is fully installed and you log back in, dasdfmt works fine, it's just during the installation process.

I've also bypassed the issue by doing a cms format on the minidisk before I installed Ubuntu on the disk, this causes Ubuntu to think the disk is already lowlevel formatted and proceeds with the install with no issues. (Still can't do a dasdfmt , but it uses the dasd for the install)

From Maint630:

Link ahubovpn 200 999 mw
Format 999 k (blksize 4096

The problem with this is, if I don't have z/VM in the environment and I'm using ECKD dasd then I have no way of low level formatting it before I install Ubuntu.

The minidisk statement I used for this guest is listed below:

  MDISK 0200 3390 0001 END AHD700 MR RAH WAH MAH

I can recreate this issue at any time, so if you need the output from the ickdsf run or any additional output please let me know.

I'm using the distro I downloaded from this link - http://partners.ubuntu.com/ibm , it's dated April 20 2016.

Thanks,

Andy Hartman

Senior Consultant
Mainline Information Systems
Cell/Phone: [phone]
Email: <email address hidden>

Notice: This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message.

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of Frank Heimes
Sent: Tuesday, May 17, 2016 4:20 PM
To: Andrew Hartman <email address hidden>
Subject: [Bug 1582728] Re: DASDFMT fails with a buffer overflow error

We have similar MDISK setups in our machine, but mostly 3390 0001 10016 (so mod9s), but did not saw such a overflow situations yet.
We cpfmtxa format our disks once and label them (but with perm 0-end) - are you using cpfmtxa too ?
So 'sudo dasdfmt -b 4096 -f /dev/dasdf' recently worked for me ...

Looks like your 9336 EDEV devices (a and b) are fine.

Please can you paste your MDISK settings of the user direct (at least for the disk 0200), the 'lsdasd -a' output and the output of 'dasdfmt --version'?

Thanks

Nevertheless a buffer overflow is not acceptable ...

--
You received this bug notification because you are subscribed to the bug report.
https://bugs.launchpad.net/bugs/1582728

Title:
  DASDFMT fails with a buffer overflow error

Status in Ubuntu on IBM z Systems:
  New
Status in s390-tools package in Ubuntu:
  New

Bug descri...

[Frank Heimes (fheimes)]

<adding response in behalf of Andy Hartman (andy-hartman)>

Hi Frank,

I used ickdsf cpvol format under Zvm (which is what cpfmtxa calls) to
format the dasd as perm space (entire volume) before I used it in my
Ubuntu guest. This volume is a mod 9.

>From Maint630:

Ickdsf
Console
Console
Cpvol format unit(d700) verify(ahd700) volid(ahd700)

Once Ubuntu is fully installed and you log back in, dasdfmt works fine, it's just during the installation process.

I've also bypassed the issue by doing a cms format on the minidisk before I installed Ubuntu on the disk, this causes Ubuntu to think the disk is already lowlevel formatted and proceeds with the install with no issues. (Still can't do a dasdfmt , but it uses the dasd for the install)

>From Maint630:

Link ahubovpn 200 999 mw
Format 999 k (blksize 4096

The problem with this is, if I don't have z/VM in the environment and
I'm using ECKD dasd then I have no way of low level formatting it before
I install Ubuntu.

The minidisk statement I used for this guest is listed below:

  MDISK 0200 3390 0001 END AHD700 MR RAH WAH MAH

I can recreate this issue at any time, so if you need the output from
the ickdsf run or any additional output please let me know.

I'm using the distro I downloaded from this link -
http://partners.ubuntu.com/ibm , it's dated April 20 2016.

Thanks,

Andy Hartman

Senior Consultant
Mainline Information Systems

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2016-06-01 07:28 EDT-------
Hi everyone,

unfortunately, I'm not able to reproduce the error here. However, with some additional data, we might be able to track it down.

Can you please mount the MDISK in question in raw-track-access mode and copy the first 2-3 tracks so that we can see if there is
any data in the label information on the disk that dasdfmt doesn't like?

Here is how you can do it in the restricted installer environment:

# Set device offline
$ echo 0 > /sys/bus/ccw/devices/0.0.0200/online
# Enable raw-track-access
$ echo 1 > /sys/bus/ccw/devices/0.0.0200/raw_track_access
# Set device online
$ echo 1 > /sys/bus/ccw/devices/0.0.0200/online
# Copy the first few tracks using dd (assuming it is dasda, look at /sys/bus/ccw/devices/0.0.0200/block to be certain )
$ dd if=/dev/dasda of=dasda_first_tracks bs=4096 count=30

You won't be able to scp the file to another machine (at least I wasn't) so you might want to attach another disk
where you can copy the file to.

Furthermore, it would be very helpful, if you could get 'strace' running within the installer and attach any trace here.
Strace should be able to tell us where dasdfmt is crashing.

Best regards,
Jan

[Andy Hartman (andy-hartman)]

I've attached the output that was requested , an strace of dasdfmt when it fails as well as the dd output

[bugproxy (bugproxy)]

------- Comment on attachment From <email address hidden> 2016-06-02 11:30 EDT-------

With the strace output I was finally able to reproduce the error.

The problem lies in libu2s which we use to get the busid of a block device. There is a bug
in the parsing of /proc/dasd/devices which is being triggered only if an FBA device
is listed prior to the ECKD device in question. And since you've had FBA devices
attached to your system during the installation, you ran into that bug.

A simple workaround would be to set the FBA devices offline if they're not really needed
at that point.

This bug was fixed in our development branch a while ago already but wasn't released yet.
The next s390-tools release will include the fix.

However, I also attached the patch that fixes this issue.

Best regards,
Jan

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "libu2s parsing fix" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

[Dimitri John Ledkov (xnox)]

@ Jan or Andy

Could you please attach /proc/dasd/devices which triggers this bug?

[bugproxy (bugproxy)]

------- Comment on attachment From <email address hidden> 2016-06-03 08:27 EDT-------

This is an example output of my test system. With that setup, trying to
format 'dasdd' or 'dasdh' would trigger the bug, because an FBA device
is listed prior to them. (The whitespace after FBA is the actual parsing
issue by the way)

Best regards,
Jan

[Andy Hartman (andy-hartman)]

Contents of /proc/dasd/devices:

/proc/dasd # cat devices
0.0.0100(FBA ) at ( 94: 0) is dasda : active at blocksize: 512, 774144 blocks, 378 MB
0.0.0101(FBA ) at ( 94: 4) is dasdb : active at blocksize: 512, 774144 blocks, 378 MB
0.0.0200(ECKD) at ( 94: 8) is dasdc : unformatted

[Dimitri John Ledkov (xnox)]

s390-tools (1.34.0-0ubuntu9) yakkety; urgency=medium

  * Cherrypick upstream fix for buffer overflow in dasdfmt. LP: 1582728

 -- Dimitri John Ledkov <email address hidden> Mon, 06 Jun 2016 11:51:16 +0100

[Brian Murray (brian-murray)]

Hello Andy, or anyone else affected,

Accepted s390-tools into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/s390-tools/1.34.0-0ubuntu8.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Dimitri John Ledkov (xnox)]

d-i needs a rebuild, after current d-i is released out of the sru-queue. then i can retest this bug.

[Adam Conrad (adconrad)]

Hello Andy, or anyone else affected,

Accepted debian-installer into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/debian-installer/20101020ubuntu451.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Dimitri John Ledkov (xnox)]

verifyin on hwe0006 instance.

[Launchpad Janitor (janitor)]

This bug was fixed in the package s390-tools - 1.34.0-0ubuntu8.1

---------------
s390-tools (1.34.0-0ubuntu8.1) xenial; urgency=medium

  * Fix ts-shell maintainer scripts LP: #1567473:
    - create /var/log/ts-shell directory with the right permissions
    - remove errorous directory
  * Install iuctty-login@.service systemd unit, with a correct path LP:
    #1580226
  * Cherrypick upstream fix for buffer overflow in dasdfmt. LP: #1582728

 -- Dimitri John Ledkov <email address hidden> Mon, 06 Jun 2016 11:51:16 +0100

[Martin Pitt (pitti)]

The verification of the Stable Release Update for s390-tools has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package debian-installer - 20101020ubuntu451.2

---------------
debian-installer (20101020ubuntu451.2) xenial; urgency=medium

  * Rebuild d-i with updated s390-tools package:
    - Cherrypick upstream fix for buffer overflow in dasdfmt. LP: #1582728

 -- Dimitri John Ledkov <email address hidden> Thu, 16 Jun 2016 10:41:23 +0300

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2016-08-01 06:44 EDT-------
Sucessfully verified with installer 451.4 from xenial-updates as follows:

1. Get installer 451 from http://bistro/ubuntu/xenial/current/
activate disk fba
now activate an unformatted DASD
it is unformatted and installer should format it. However, this should fail! Just to be sure to have the proper setup to trigger this bug.

2. Now get installer 451.4 from http://bistro/ubuntu/xenial-updates/current/
activate disk fba
now activate an unformatted DASD
it is unformatted and installer should format it. That worked perfectly in contradiction to step 1, the bug is fixed and successfully verified.