qemu-img execute not mocked in unit tests

Bug #1765122 reported by Jay Pipes on 2018-04-18
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Low
Jay Pipes
Ubuntu Cloud Archive
Status tracked in Rocky
Ocata
Low
Unassigned
Pike
Low
Unassigned
Queens
Low
Unassigned
Rocky
Low
Unassigned
nova (Ubuntu)
Status tracked in Cosmic
Bionic
Low
Unassigned
Cosmic
Low
Unassigned

Bug Description

nova.tests.unit.virt.test_images.QemuTestCase.test_qemu_info_with_errors is failing in both py27 and py36 tox environments due to a missing mock.

This system does not have qemu(-img) installed in it and running unit tests returns the following:

==============================
Failed 1 tests - output below:
==============================

nova.tests.unit.virt.test_images.QemuTestCase.test_qemu_info_with_errors
------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    b'Traceback (most recent call last):'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/nova/virt/images.py", line 73, in qemu_img_info'
    b' out, err = utils.execute(*cmd, prlimit=QEMU_IMG_LIMITS)'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/nova/utils.py", line 231, in execute'
    b' return processutils.execute(*cmd, **kwargs)'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/oslo_concurrency/processutils.py", line 424, in execute'
    b' cmd=sanitized_cmd)'
    b'oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.'
    b'Command: /home/jaypipes/src/git.openstack.org/openstack/nova/.tox/py36/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /fake/path'
    b'Exit code: 127'
    b"Stdout: ''"
    b"Stderr: '/usr/bin/env: \xe2\x80\x98qemu-img\xe2\x80\x99: No such file or directory\\n'"
    b''
    b'During handling of the above exception, another exception occurred:'
    b''
    b'Traceback (most recent call last):'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/mock/mock.py", line 1305, in patched'
    b' return func(*args, **keywargs)'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/nova/tests/unit/virt/test_images.py", line 37, in test_qemu_info_with_errors'
    b" '/fake/path')"
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/testtools/testcase.py", line 485, in assertRaises'
    b' self.assertThat(our_callable, matcher)'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/testtools/testcase.py", line 496, in assertThat'
    b' mismatch_error = self._matchHelper(matchee, matcher, message, verbose)'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/testtools/testcase.py", line 547, in _matchHelper'
    b' mismatch = matcher.match(matchee)'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/testtools/matchers/_exception.py", line 108, in match'
    b' mismatch = self.exception_matcher.match(exc_info)'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/testtools/matchers/_higherorder.py", line 62, in match'
    b' mismatch = matcher.match(matchee)'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/testtools/testcase.py", line 475, in match'
    b' reraise(*matchee)'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/testtools/_compat3x.py", line 16, in reraise'
    b' raise exc_obj.with_traceback(exc_tb)'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/testtools/matchers/_exception.py", line 101, in match'
    b' result = matchee()'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/testtools/testcase.py", line 1049, in __call__'
    b' return self._callable_object(*self._args, **self._kwargs)'
    b' File "/home/jaypipes/src/git.openstack.org/openstack/nova/nova/virt/images.py", line 87, in qemu_img_info'
    b' raise exception.InvalidDiskInfo(reason=msg)'
    b'nova.exception.InvalidDiskInfo: Disk info file is invalid: qemu-img failed to execute on /fake/path : Unexpected error while running command.'
    b'Command: /home/jaypipes/src/git.openstack.org/openstack/nova/.tox/py36/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /fake/path'
    b'Exit code: 127'
    b"Stdout: ''"
    b"Stderr: '/usr/bin/env: \xe2\x80\x98qemu-img\xe2\x80\x99: No such file or directory\\n'"
    b''

Adding Ubuntu SRU details:
--------------------------
[Impact]
When building the latest package version of nova, the following unit test fails:

nova.tests.unit.virt.test_images.QemuTestCase.test_qemu_info_with_errors

[Test Case]
Simply building the package will confirm whether or not the test fails.

[Regression Potential]
Extremely low. This is just a test update.

Fix proposed to branch: master
Review: https://review.openstack.org/562339

Changed in nova:
status: New → In Progress

Reviewed: https://review.openstack.org/562339
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=815b11f70a3fd31888fb3e3e7b38e3a697bf6244
Submitter: Zuul
Branch: master

commit 815b11f70a3fd31888fb3e3e7b38e3a697bf6244
Author: Jay Pipes <email address hidden>
Date: Wed Apr 18 12:39:35 2018 -0400

    mock utils.execute() in qemu-img unit test

    There was a unit test that was not mocking utils.execute() and relying
    on qemu-img being installed on the machine running the tests. This fixes
    the test to mock out utils.execute() and raise a ProcessExecutionError
    that simulates the expected behaviour from qemu-img.

    Fixes bug: #1765122

    Change-Id: Ia6fc089fce0cc0ba1fb8d4d4ffbf7f47968a0507

Changed in nova:
status: In Progress → Fix Released

Reviewed: https://review.openstack.org/574255
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=cf927f177be3ff0d1df069bc66f889ebcce93754
Submitter: Zuul
Branch: stable/queens

commit cf927f177be3ff0d1df069bc66f889ebcce93754
Author: Jay Pipes <email address hidden>
Date: Wed Apr 18 12:39:35 2018 -0400

    mock utils.execute() in qemu-img unit test

    There was a unit test that was not mocking utils.execute() and relying
    on qemu-img being installed on the machine running the tests. This fixes
    the test to mock out utils.execute() and raise a ProcessExecutionError
    that simulates the expected behaviour from qemu-img.

    Fixes bug: #1765122

    Change-Id: Ia6fc089fce0cc0ba1fb8d4d4ffbf7f47968a0507
    (cherry picked from commit 815b11f70a3fd31888fb3e3e7b38e3a697bf6244)

tags: added: in-stable-queens
Dr. Jens Harbott (j-harbott) wrote :

stable/pike (16.1.4) is also affected by this.

Changed in nova (Ubuntu):
status: New → Triaged
importance: Undecided → Low
Changed in nova (Ubuntu Bionic):
status: New → Triaged
importance: Undecided → Low
Changed in nova (Ubuntu Cosmic):
status: Triaged → Fix Released

Hello Jay, or anyone else affected,

Accepted nova into pike-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:pike-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-pike-needed to verification-pike-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-pike-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-pike-needed
Corey Bryant (corey.bryant) wrote :

I will cover testing of this for Ubuntu.

Corey Bryant (corey.bryant) wrote :

Hello Jay, or anyone else affected,

Accepted nova into ocata-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:ocata-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-ocata-needed to verification-ocata-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-ocata-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-ocata-needed
description: updated
Brian Murray (brian-murray) wrote :

Hello Jay, or anyone else affected,

Accepted nova into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/nova/2:17.0.5-0ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in nova (Ubuntu Bionic):
status: Triaged → Fix Committed
tags: added: verification-needed verification-needed-bionic
Corey Bryant (corey.bryant) wrote :

Hello Jay, or anyone else affected,

Accepted nova into queens-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:queens-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-queens-needed to verification-queens-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-queens-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-queens-needed

Reviewed: https://review.openstack.org/578058
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=ba0eb30a75c0e20dfe0e8065d9765292fc963e95
Submitter: Zuul
Branch: stable/pike

commit ba0eb30a75c0e20dfe0e8065d9765292fc963e95
Author: Jay Pipes <email address hidden>
Date: Wed Apr 18 12:39:35 2018 -0400

    mock utils.execute() in qemu-img unit test

    There was a unit test that was not mocking utils.execute() and relying
    on qemu-img being installed on the machine running the tests. This fixes
    the test to mock out utils.execute() and raise a ProcessExecutionError
    that simulates the expected behaviour from qemu-img.

    Fixes bug: #1765122

    Change-Id: Ia6fc089fce0cc0ba1fb8d4d4ffbf7f47968a0507
    (cherry picked from commit 815b11f70a3fd31888fb3e3e7b38e3a697bf6244)
    (cherry picked from commit cf927f177be3ff0d1df069bc66f889ebcce93754)

Łukasz Zemczak (sil2100) wrote :

Could we get this bug verified? We don't want to release all the other openstack SRUs before this one is done as well.

Corey Bryant (corey.bryant) wrote :

Hi Łukasz!

This has been verified successfully. The fixed unit test is executed during the package build. This is from the buildlog for bionic-proposed at https://launchpad.net/ubuntu/+source/nova/2:17.0.5-0ubuntu1/+build/15065539:

nova.tests.unit.virt.test_images.QemuTestCase.test_qemu_info_with_errors [0.024548s] ... ok

Thanks,
Corey

tags: added: verification-done-bionic
removed: verification-needed-bionic
Corey Bryant (corey.bryant) wrote :
tags: added: verification-ocatdonea-needed verification-pike-done verification-queens-done
removed: verification-ocata-needed verification-pike-needed verification-queens-needed
tags: added: verification-ocata-done
removed: verification-ocatdonea-needed

The verification of the Stable Release Update for nova has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package nova - 2:17.0.5-0ubuntu1

---------------
nova (2:17.0.5-0ubuntu1) bionic; urgency=medium

  * New stable point release for OpenStack Queens (LP: #1778747).
  * d/p/mock-utils.execute-in-qemu-img-unit-test.patch: Cherry-picked from
    upstream stable/queens branch to fix failing unit test (LP: #1765122).

 -- Corey Bryant <email address hidden> Tue, 26 Jun 2018 11:55:44 -0400

Changed in nova (Ubuntu Bionic):
status: Fix Committed → Fix Released
Corey Bryant (corey.bryant) wrote :

The verification of the Stable Release Update for nova has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Corey Bryant (corey.bryant) wrote :

This bug was fixed in the package nova - 2:17.0.5-0ubuntu1~cloud0
---------------

 nova (2:17.0.5-0ubuntu1~cloud0) xenial-queens; urgency=medium
 .
   * New upstream release for the Ubuntu Cloud Archive.
 .
 nova (2:17.0.5-0ubuntu1) bionic; urgency=medium
 .
   * New stable point release for OpenStack Queens (LP: #1778747).
   * d/p/mock-utils.execute-in-qemu-img-unit-test.patch: Cherry-picked from
     upstream stable/queens branch to fix failing unit test (LP: #1765122).

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers