[FFe] Support for ISCSI block devices

Bug #1645515 reported by Blake Rouse
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
curtin
High
Nish Aravamudan
curtin (Ubuntu)
Medium
Unassigned
Xenial
Medium
Unassigned
Yakkety
Medium
Unassigned

Bug Description

==== Begin Curtin SRU Template ====
[Impact]
Curtin currently does not support specifying iSCSI targets in the storage
configuration. That means that MAAS or other users of curtin are not able
to install to use iscsi disks during installation.

The changes in this bug add the ability to specify non-root iSCSI targets
to curtin, and include integration tests (vmtest) and unit tests to excercise
the functionality.

[Test Case]
Run curtin vmtest on tests/vmtests/test_iscsi.py.
 ./tools/jenkins-runner tests/vmtests/test_iscsi.py

The named test sets up an iscsi server with tgt and does an installation
utilizing the disks provided by that iscsi server.

[Regression Potential]
Regression would most likely be a result of the new iscsi functionality
mis-identifying configuration for a local disk as a iscsi disk, and then
failing to do things as a result.

==== End Curtin SRU Template ====

[FFe Justification]

 - Curtin currently does not support specifying iSCSI targets in the storage configuration.

 - Users have iSCSI targets they would like to install to via curtin.

 - The changes in this bug add the ability to specify non-root iSCSI targets in the YAML and include vmtests and unittests for that functionality.

 - Existing users are unaffected by this change, as if they specified a iSCSI compatible value in their YAML currently it would fail to parse. The added code is only used when iSCSI is used.

---

Curtin needs to support the ability to attach an ISCSI block device during the installation and perform disk operations just like a locally attached block device. The disk also needs to be auto mounted when Ubuntu is rebooted after installation.

Possible format:

- id: sdb
  type: disk
  iscsi:
    target: iqn.2016-11.io.maas:storage.lun1
    portal: 192.168.122.2:3260

Related branches

Jon Grimm (jgrimm)
Changed in curtin:
assignee: nobody → Jon Grimm (jgrimm)
assignee: Jon Grimm (jgrimm) → nobody
Nish Aravamudan (nacc)
Changed in curtin:
assignee: nobody → Nish Aravamudan (nacc)
Revision history for this message
Scott Moser (smoser) wrote :

It might be sane or nice to support specification of the iscsi target in RFC 4173 format.
See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=804162 for some more information.

Revision history for this message
Scott Moser (smoser) wrote :

In order to test this through vmtest, we'll need some iscsi server.
I've done some work at [1] which at minimum will show how to register a new device into iscsi, and add it and then remove it within a given xkvm. the 'tgt-boot-test' there handles adding a block device to tgt and then tearing it down.

[1] https://git.launchpad.net/~smoser/ubuntu/+source/open-iscsi/tree/debian/tests/README-boot-test.md?h=diglett-master

Revision history for this message
Jon Grimm (jgrimm) wrote :

I'm assuming this will need FFe, might as well file it now.

Nish Aravamudan (nacc)
summary: - Support for ISCSI block devices
+ [FFe] Support for ISCSI block devices
Nish Aravamudan (nacc)
description: updated
Scott Moser (smoser)
Changed in curtin (Ubuntu):
status: New → Confirmed
importance: Undecided → Medium
Changed in curtin:
status: Confirmed → Fix Committed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package curtin - 0.1.0~bzr470-0ubuntu1

---------------
curtin (0.1.0~bzr470-0ubuntu1) zesty; urgency=medium

  * New upstream snapshot.
    - vmtest: allow time-based skipping of tests, apply to ZestyMdadmBcache
      (LP: #1667078)
    - Add iSCSI disk support. (LP: #1645515)
    - vmtest: handle change in btrfs-progs when finding fs UUID. (LP: #1665701)
    - pycodestyle: fix bare except in vmtests
    - unittests: add subp coverage for retries, and rcs
    - reporting: Add better descriptions and granularity of events
    - tox: do not bother with pep8 from trusty in trusty-check
    - tests: add two scripts used for running vmtest with installed package.
    - Make add-apt-repository command retry
    - Add tests for network bridging configuration
  - debian/rules: remove debug 'find'

 -- Scott Moser <email address hidden> Tue, 28 Feb 2017 10:30:43 -0500

Changed in curtin (Ubuntu):
status: Confirmed → Fix Released
Revision history for this message
Jon Grimm (jgrimm) wrote :

This will also be needed in Xenial.

Scott Moser (smoser)
Changed in curtin (Ubuntu Xenial):
status: New → Confirmed
Changed in curtin (Ubuntu Yakkety):
status: New → Confirmed
Changed in curtin (Ubuntu Xenial):
importance: Undecided → Medium
Changed in curtin (Ubuntu Yakkety):
importance: Undecided → Medium
Scott Moser (smoser)
description: updated
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Blake, or anyone else affected,

Accepted curtin into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/curtin/0.1.0~bzr470-0ubuntu1~16.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in curtin (Ubuntu Yakkety):
status: Confirmed → Fix Committed
tags: added: verification-needed
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Blake, or anyone else affected,

Accepted curtin into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/curtin/0.1.0~bzr470-0ubuntu1~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in curtin (Ubuntu Xenial):
status: Confirmed → Fix Committed
Revision history for this message
Scott Moser (smoser) wrote :

Ubuntu server jenkins now runs integration tests with for yakkety-proposed.
It's latest run at [1] passed and ran the tests listed below that verify
this bug if fixed.

  bug 1645515 tests/vmtests/test_iscsi.py
  bug 1665701 tests/vmtests/test_nvme.py tests/vmtests/test_basic.py

I've attached 2 files for posterity
  curtin-vmtest-proposed-y-23-archive.zip [2]
  curtin-vmtest-proposed-y-23-console.log [3]
artifacts of curtin vmtest-proposed-y build 23

You can verify that the installation of curtin used the version shown by:
$ grep -r "Installation started" archive/ | head -n 1
archive/output/ZestyTestNetworkIPV6Vlan/logs/install-serial.log:[ 10.529395] cloud-init[1503]: curtin: Installation started. (0.1.0~bzr470-0ubuntu1~16.10.1)

--
[1] https://jenkins.ubuntu.com/server/view/Curtin/job/curtin-vmtest-proposed-y/23/
[2] https://bugs.launchpad.net/ubuntu/+source/btrfs-progs/+bug/1665701/+attachment/4834793/+files/curtin-vmtest-proposed-y-23-archive.zip
[3] https://bugs.launchpad.net/ubuntu/+source/btrfs-progs/+bug/1665701/+attachment/4834794/+files/curtin-vmtest-proposed-y-23-console.log

Revision history for this message
Scott Moser (smoser) wrote :

Ubuntu server jenkins now runs integration tests with for xenial-proposed.
It's latest run at [1] passed and ran the tests listed below that verify
this bug if fixed.

  bug 1645515 tests/vmtests/test_iscsi.py
  bug 1665701 tests/vmtests/test_nvme.py tests/vmtests/test_basic.py

I've attached 2 files to bug 1665701 for posterity
  curtin-vmtest-proposed-x-22-archive.zip [2]
  curtin-vmtest-proposed-x-22-console.log [3]
artifacts of curtin vmtest-proposed-x build 22

You can verify that the installation of curtin used the version shown by:
$ grep -r "Installation started" archive/ | head -n 1
archive/output/ZestyTestNetworkIPV6Vlan/logs/install-serial.log:[ 9.954624] cloud-init[1518]: curtin: Installation started. (0.1.0~bzr470-0ubuntu1~16.04.1)

--
[1] https://jenkins.ubuntu.com/server/view/Curtin/job/curtin-vmtest-proposed-x/22/
[2] https://bugs.launchpad.net/ubuntu/+source/btrfs-progs/+bug/1665701/+attachment/4834791/+files/curtin-vmtest-proposed-x-22-archive.zip
[3] https://bugs.launchpad.net/ubuntu/+source/btrfs-progs/+bug/1665701/+attachment/4834792/+files/curtin-vmtest-proposed-x-22-console.log

tags: added: verification-done-xenial verification-done-yakkety
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package curtin - 0.1.0~bzr470-0ubuntu1~16.04.1

---------------
curtin (0.1.0~bzr470-0ubuntu1~16.04.1) xenial-proposed; urgency=medium

  * debian/rules: record debian package version in installed package.
    (LP: #1666986)
  * New upstream snapshot.
    - vmtest: allow time-based skipping of tests, apply to ZestyMdadmBcache
      to temporarily avoid bug 1667078.
    - Add iSCSI disk support. (LP: #1645515)
    - vmtest: handle change in btrfs-progs when finding fs UUID. (LP: #1665701)
    - pycodestyle: fix bare except in vmtests
    - unittests: add subp coverage for retries, and rcs
    - reporting: Add better descriptions and granularity of events
    - tox: do not bother with pep8 from trusty in trusty-check
    - tests: add two scripts used for running vmtest with installed package.
    - Make add-apt-repository command retry
    - Add tests for network bridging configuration

 -- Scott Moser <email address hidden> Wed, 01 Mar 2017 11:13:58 -0500

Changed in curtin (Ubuntu Xenial):
status: Fix Committed → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote : Update Released

The verification of the Stable Release Update for curtin has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package curtin - 0.1.0~bzr470-0ubuntu1~16.10.1

---------------
curtin (0.1.0~bzr470-0ubuntu1~16.10.1) yakkety-proposed; urgency=medium

  * debian/rules: record debian package version in installed package.
    (LP: #1666986)
  * New upstream snapshot.
    - vmtest: allow time-based skipping of tests, apply to ZestyMdadmBcache
      to temporarily avoid bug 1667078.
    - Add iSCSI disk support. (LP: #1645515)
    - vmtest: handle change in btrfs-progs when finding fs UUID. (LP: #1665701)
    - pycodestyle: fix bare except in vmtests
    - unittests: add subp coverage for retries, and rcs
    - reporting: Add better descriptions and granularity of events
    - tox: do not bother with pep8 from trusty in trusty-check
    - tests: add two scripts used for running vmtest with installed package.
    - Make add-apt-repository command retry
    - Add tests for network bridging configuration

 -- Scott Moser <email address hidden> Wed, 01 Mar 2017 11:17:12 -0500

Changed in curtin (Ubuntu Yakkety):
status: Fix Committed → Fix Released
Revision history for this message
Scott Moser (smoser) wrote : Fixed in Curtin 17.1

This bug is believed to be fixed in curtin in 17.1. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

Changed in curtin:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.