OpenStack Compute (nova)

qemu-img execute not mocked in unit tests

Bug #1765122 reported by Jay Pipes on 2018-04-18

This bug affects 1 person

	Status	Importance	Assigned to
OpenStack Compute (nova)	Fix Released	Low	Jay Pipes
Ubuntu Cloud Archive	Fix Released	Low	Unassigned
Ocata	Fix Released	Low	Unassigned
Pike	Fix Released	Low	Unassigned
Queens	Fix Released	Low	Unassigned
Rocky	Fix Released	Low	Unassigned
nova (Ubuntu)	Fix Released	Low	Unassigned
Bionic	Fix Released	Low	Unassigned
Cosmic	Fix Released	Low	Unassigned

Bug Description

nova.tests.unit.virt.test_images.QemuTestCase.test_qemu_info_with_errors is failing in both py27 and py36 tox environments due to a missing mock.

This system does not have qemu(-img) installed in it and running unit tests returns the following:

==============================
Failed 1 tests - output below:
==============================

nova.tests.unit.virt.test_images.QemuTestCase.test_qemu_info_with_errors
------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    b'Traceback (most recent call last):'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/nova/virt/images.py", line 73, in qemu_img_info'
    b' out, err = utils.execute(*cmd, prlimit=QEMU_IMG_LIMITS)'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/nova/utils.py", line 231, in execute'
    b' return processutils.execute(*cmd, **kwargs)'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/oslo_concurrency/processutils.py", line 424, in execute'
    b' cmd=sanitized_cmd)'
    b'oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.'
    b'Command: /home/jaypipes/src/git.openstack.org/openstack/nova/.tox/py36/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /fake/path'
    b'Exit code: 127'
    b"Stdout: ''"
    b"Stderr: '/usr/bin/env: \xe2\x80\x98qemu-img\xe2\x80\x99: No such file or directory\\n'"
    b''
    b'During handling of the above exception, another exception occurred:'
    b''
    b'Traceback (most recent call last):'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/mock/mock.py", line 1305, in patched'
    b' return func(*args, **keywargs)'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/nova/tests/unit/virt/test_images.py", line 37, in test_qemu_info_with_errors'
    b" '/fake/path')"
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/testtools/testcase.py", line 485, in assertRaises'
    b' self.assertThat(our_callable, matcher)'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/testtools/testcase.py", line 496, in assertThat'
    b' mismatch_error = self._matchHelper(matchee, matcher, message, verbose)'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/testtools/testcase.py", line 547, in _matchHelper'
    b' mismatch = matcher.match(matchee)'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/testtools/matchers/_exception.py", line 108, in match'
    b' mismatch = self.exception_matcher.match(exc_info)'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/testtools/matchers/_higherorder.py", line 62, in match'
    b' mismatch = matcher.match(matchee)'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/testtools/testcase.py", line 475, in match'
    b' reraise(*matchee)'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/testtools/_compat3x.py", line 16, in reraise'
    b' raise exc_obj.with_traceback(exc_tb)'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/testtools/matchers/_exception.py", line 101, in match'
    b' result = matchee()'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/testtools/testcase.py", line 1049, in __call__'
    b' return self._callable_object(*self._args, **self._kwargs)'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/nova/virt/images.py", line 87, in qemu_img_info'
    b' raise exception.InvalidDiskInfo(reason=msg)'
    b'nova.exception.InvalidDiskInfo: Disk info file is invalid: qemu-img failed to execute on /fake/path : Unexpected error while running command.'
    b'Command: /home/jaypipes/src/git.openstack.org/openstack/nova/.tox/py36/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /fake/path'
    b'Exit code: 127'
    b"Stdout: ''"
    b"Stderr: '/usr/bin/env: \xe2\x80\x98qemu-img\xe2\x80\x99: No such file or directory\\n'"
    b''

Adding Ubuntu SRU details:
--------------------------
[Impact]
When building the latest package version of nova, the following unit test fails:

nova.tests.unit.virt.test_images.QemuTestCase.test_qemu_info_with_errors

[Test Case]
Simply building the package will confirm whether or not the test fails.

[Regression Potential]
Extremely low. This is just a test update.

See original description

Tags:

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2018-04-18: Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/562339

Changed in nova:
status:	New → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2018-04-20: Fix merged to nova (master)

Reviewed: https://review.openstack.org/562339
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=815b11f70a3fd31888fb3e3e7b38e3a697bf6244
Submitter: Zuul
Branch: master

commit 815b11f70a3fd31888fb3e3e7b38e3a697bf6244
Author: Jay Pipes <email address hidden>
Date: Wed Apr 18 12:39:35 2018 -0400

mock utils.execute() in qemu-img unit test

    There was a unit test that was not mocking utils.execute() and relying
    on qemu-img being installed on the machine running the tests. This fixes
    the test to mock out utils.execute() and raise a ProcessExecutionError
    that simulates the expected behaviour from qemu-img.

Fixes bug: #1765122

Change-Id: Ia6fc089fce0cc0ba1fb8d4d4ffbf7f47968a0507

Changed in nova:
status:	In Progress → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2018-06-11: Fix proposed to nova (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/574255

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2018-06-15: Fix merged to nova (stable/queens)

Reviewed: https://review.openstack.org/574255
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=cf927f177be3ff0d1df069bc66f889ebcce93754
Submitter: Zuul
Branch: stable/queens

commit cf927f177be3ff0d1df069bc66f889ebcce93754
Author: Jay Pipes <email address hidden>
Date: Wed Apr 18 12:39:35 2018 -0400

mock utils.execute() in qemu-img unit test

Fixes bug: #1765122

Change-Id: Ia6fc089fce0cc0ba1fb8d4d4ffbf7f47968a0507
(cherry picked from commit 815b11f70a3fd31888fb3e3e7b38e3a697bf6244)

tags:

added: in-stable-queens

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2018-06-26: Fix proposed to nova (stable/pike)

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/578058

Revision history for this message

Dr. Jens Harbott (j-harbott) wrote on 2018-06-26:

stable/pike (16.1.4) is also affected by this.

Corey Bryant (corey.bryant) on 2018-06-26

Changed in nova (Ubuntu):
status:	New → Triaged
importance:	Undecided → Low

Corey Bryant (corey.bryant) on 2018-06-26

Changed in nova (Ubuntu Bionic):
status:	New → Triaged
importance:	Undecided → Low

Corey Bryant (corey.bryant) on 2018-06-26

Changed in nova (Ubuntu Cosmic):
status:	Triaged → Fix Released

Revision history for this message

Corey Bryant (corey.bryant) wrote on 2018-06-27: Please test proposed package

Hello Jay, or anyone else affected,

Accepted nova into pike-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

sudo add-apt-repository cloud-archive:pike-proposed
sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-pike-needed to verification-pike-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-pike-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags:

added: verification-pike-needed

Revision history for this message

Corey Bryant (corey.bryant) wrote on 2018-06-27:

I will cover testing of this for Ubuntu.

Revision history for this message

Corey Bryant (corey.bryant) wrote on 2018-06-27:

Hello Jay, or anyone else affected,

Accepted nova into ocata-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

sudo add-apt-repository cloud-archive:ocata-proposed
sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-ocata-needed to verification-ocata-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-ocata-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags:

added: verification-ocata-needed

Corey Bryant (corey.bryant) on 2018-06-27

description:

updated

Revision history for this message

Brian Murray (brian-murray) wrote on 2018-06-28:

#10

Hello Jay, or anyone else affected,

Accepted nova into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/nova/2:17.0.5-0ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in nova (Ubuntu Bionic):
status:	Triaged → Fix Committed
tags:	added: verification-needed verification-needed-bionic

Revision history for this message

Corey Bryant (corey.bryant) wrote on 2018-06-29:

#11

Hello Jay, or anyone else affected,

Accepted nova into queens-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

sudo add-apt-repository cloud-archive:queens-proposed
sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-queens-needed to verification-queens-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-queens-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags:

added: verification-queens-needed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2018-06-29: Fix merged to nova (stable/pike)

#12

Reviewed: https://review.openstack.org/578058
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=ba0eb30a75c0e20dfe0e8065d9765292fc963e95
Submitter: Zuul
Branch: stable/pike

commit ba0eb30a75c0e20dfe0e8065d9765292fc963e95
Author: Jay Pipes <email address hidden>
Date: Wed Apr 18 12:39:35 2018 -0400

mock utils.execute() in qemu-img unit test

Fixes bug: #1765122

    Change-Id: Ia6fc089fce0cc0ba1fb8d4d4ffbf7f47968a0507
    (cherry picked from commit 815b11f70a3fd31888fb3e3e7b38e3a697bf6244)
    (cherry picked from commit cf927f177be3ff0d1df069bc66f889ebcce93754)

Revision history for this message

Łukasz Zemczak (sil2100) wrote on 2018-07-16:

#13

Could we get this bug verified? We don't want to release all the other openstack SRUs before this one is done as well.

Revision history for this message

Corey Bryant (corey.bryant) wrote on 2018-07-16:

#14

Hi Łukasz!

This has been verified successfully. The fixed unit test is executed during the package build. This is from the buildlog for bionic-proposed at https://launchpad.net/ubuntu/+source/nova/2:17.0.5-0ubuntu1/+build/15065539:

nova.tests.unit.virt.test_images.QemuTestCase.test_qemu_info_with_errors [0.024548s] ... ok

Thanks,
Corey

tags:

added: verification-done-bionic
removed: verification-needed-bionic

Revision history for this message

Corey Bryant (corey.bryant) wrote on 2018-07-16:

#15

Verified successfully for queens-proposed, pike-proposed, and ocata-proposed as well:

queens-proposed: https://launchpad.net/~ubuntu-cloud-archive-private/+archive/ubuntu/queens-proposed/+build/15067653

pike-proposed: https://launchpad.net/~ubuntu-cloud-archive-private/+archive/ubuntu/pike-proposed/+build/15080565

ocata-proposed: https://launchpad.net/~ubuntu-cloud-archive-private/+archive/ubuntu/ocata-proposed/+build/15080580

tags:	added: verification-ocatdonea-needed verification-pike-done verification-queens-done removed: verification-ocata-needed verification-pike-needed verification-queens-needed
tags:	added: verification-ocata-done removed: verification-ocatdonea-needed

Revision history for this message

Łukasz Zemczak (sil2100) wrote on 2018-07-16: Update Released

#16

The verification of the Stable Release Update for nova has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message

Launchpad Janitor (janitor) wrote on 2018-07-16:

#17

This bug was fixed in the package nova - 2:17.0.5-0ubuntu1

---------------
nova (2:17.0.5-0ubuntu1) bionic; urgency=medium

  * New stable point release for OpenStack Queens (LP: #1778747).
  * d/p/mock-utils.execute-in-qemu-img-unit-test.patch: Cherry-picked from
    upstream stable/queens branch to fix failing unit test (LP: #1765122).

-- Corey Bryant <email address hidden> Tue, 26 Jun 2018 11:55:44 -0400

Changed in nova (Ubuntu Bionic):
status:	Fix Committed → Fix Released

Revision history for this message

Corey Bryant (corey.bryant) wrote on 2018-07-16:

#18

The verification of the Stable Release Update for nova has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message

Corey Bryant (corey.bryant) wrote on 2018-07-16:

#19

This bug was fixed in the package nova - 2:17.0.5-0ubuntu1~cloud0
---------------

nova (2:17.0.5-0ubuntu1~cloud0) xenial-queens; urgency=medium
.
   * New upstream release for the Ubuntu Cloud Archive.
.
nova (2:17.0.5-0ubuntu1) bionic; urgency=medium
.
   * New stable point release for OpenStack Queens (LP: #1778747).
   * d/p/mock-utils.execute-in-qemu-img-unit-test.patch: Cherry-picked from
     upstream stable/queens branch to fix failing unit test (LP: #1765122).

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.