python3 test failures running tox

Bug #1813980 reported by Corey Bryant on 2019-01-30
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
zVM Cloud Connector
High
Ya Lian Pan

Bug Description

I'm hitting a lot of python 3 test failures running tox on Ubuntu Bionic:
https://paste.ubuntu.com/p/FmPtvghHGH/

This is with the master branch as of Jan 30, 2019.

description: updated
jichenjc (jichenjc) wrote :

https://review.gerrithub.io/c/mfcloud/python-zvm-sdk/+/442815

just merged and we passed local test ,please help to check again, thanks

Corey Bryant (corey.bryant) wrote :

That fixed all but one test for me when I cherry-picked it to 1.4.0. It's possible there are other fixes in master that haven't been released yet. Here's the remaining failure:

======================================================================
FAIL: test_guest_deploy_smut_request_failed (test_smutclient.SDKSMUTClientTestCases)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/mock/mock.py", line 1305, in patched
    return func(*args, **keywargs)
  File "/<<PKGBUILDDIR>>/zvmsdk/tests/unit/test_smutclient.py", line 329, in test_guest_deploy_smut_request_failed
    execute.assert_has_calls([mock.call(unpack_cmd), mock.call(scp_cmd)])
  File "/usr/lib/python3/dist-packages/mock/mock.py", line 969, in assert_has_calls
    ), cause)
  File "<string>", line 3, in raise_from
AssertionError: Calls not found.
Expected: [call(['sudo', '/opt/zthin/bin/unpackdiskimage', 'fakeuser', '0100', '/var/lib/zvmsdk/images/netboot/rhel7/fakeimg/0100']),
 call(['/usr/bin/scp', '-B', '-P', '22', '-o StrictHostKeyChecking=no', 'user@1.1.1.1:/faketran', '/tmp/tmpdir/faketran'])]
Actual: [call(['sudo', '/opt/zthin/bin/unpackdiskimage', 'fakeuser', '0100', '/var/lib/zvmsdk/images/netboot/rhel7/fakeimg/0100']),
 call(['/usr/bin/scp', '-B', 'user@1.1.1.1:/faketran', '/tmp/tmpdir/faketran'])]

jichenjc (jichenjc) wrote :

Ella, can you help to take a look at this? Thanks

Changed in python-zvm-sdk:
assignee: nobody → Ya Lian Pan (yalian)
importance: Undecided → High
status: New → Confirmed
jichenjc (jichenjc) wrote :

per Ella mentioned, the patch already fixed and cherry picked to
https://review.gerrithub.io/c/mfcloud/python-zvm-sdk/+/442817

so we need someone in the team to have new version released then you can consume it

Corey Bryant (corey.bryant) wrote :

I do have that patch applied but the expected scp output isn't the same:

Expected: [call(['sudo', '/opt/zthin/bin/unpackdiskimage', 'fakeuser', '0100', '/var/lib/zvmsdk/images/netboot/rhel7/fakeimg/0100']),
 call(['/usr/bin/scp', '-B', '-P', '22', '-o StrictHostKeyChecking=no', 'user@1.1.1.1:/faketran', '/tmp/tmpdir/faketran'])]
Actual: [call(['sudo', '/opt/zthin/bin/unpackdiskimage', 'fakeuser', '0100', '/var/lib/zvmsdk/images/netboot/rhel7/fakeimg/0100']),
 call(['/usr/bin/scp', '-B', 'user@1.1.1.1:/faketran', '/tmp/tmpdir/faketran'])]

Huang Rui (bjhuangr) wrote :

release 1.4.1 available for both github and pypi. Hopefully this can fix the UT failure.

jichenjc (jichenjc) wrote :

ella, let's test it on 1.4.1 first and see whether we need 1.4.2 ..

Huang Rui (bjhuangr) wrote :

I believe 1.4.1 includes the fix to the failure in commment #6.

@Corey, could you try 1.4.1?

Corey Bryant (corey.bryant) wrote :

Looking at the code it doesn't appear to be fixed. It seems the scp calls can have different arguments, perhaps that varies depending on what release of a distro you run on.

    def test_guest_deploy_smt_request_failed(self, get_image_path, request,
                                              execute, mkdtemp, cleantemp):
        ...
        scp_cmd = ["/usr/bin/scp", "-B", '-P', '22',
                   '-o StrictHostKeyChecking=no', 'user@1.1.1.1:/faketran',
                  '/tmp/tmpdir/faketran']
        execute.assert_has_calls([mock.call(unpack_cmd), mock.call(scp_cmd)])
        ...

If you compare that vs the failure that I hit you can see that the scp call in my case didn't have '-P', '22', '-o StrictHostKeyChecking=no':

Expected: [call(['sudo', '/opt/zthin/bin/unpackdiskimage', 'fakeuser', '0100', '/var/lib/zvmsdk/images/netboot/rhel7/fakeimg/0100']),
 call(['/usr/bin/scp', '-B', '-P', '22', '-o StrictHostKeyChecking=no', 'user@1.1.1.1:/faketran', '/tmp/tmpdir/faketran'])]
Actual: [call(['sudo', '/opt/zthin/bin/unpackdiskimage', 'fakeuser', '0100', '/var/lib/zvmsdk/images/netboot/rhel7/fakeimg/0100']),
 call(['/usr/bin/scp', '-B', 'user@1.1.1.1:/faketran', '/tmp/tmpdir/faketran'])]

Huang Rui (bjhuangr) wrote :

The fix is not distro related. I also downloaded the 1.4.1, run both py27 and py36 unittests, all passed in my local env.

Corey Bryant (corey.bryant) wrote :

Is /usr/bin/scp installed by the distro?

Corey Bryant (corey.bryant) wrote :

I'm on Ubuntu disco fwiw.

jichenjc (jichenjc) wrote :

it's not related to scp ,it's

'-P', '22', '-o StrictHostKeyChecking=no',

was not added, so need double check why this happens, @Rui, can you help to double check this? Thanks

Ya Lian Pan (yalian) wrote :

The "'-P', '22', '-o StrictHostKeyChecking=no'" is added by the following path

https://review.gerrithub.io/c/mfcloud/python-zvm-sdk/+/437583

I checked code for release 1.4.1, it has already include the fix

Huang Rui (bjhuangr) wrote :

@jichenjc, I've done the check before my last reply.

@Corey, /usr/bin/scp does installed by distro, but the UT testcase mock the real call, just verify the params to the call, that's why I said the fix is not related to distro.

jichenjc (jichenjc) wrote :

@Rui, thanks for confirm, let's wait for reply from Corey

Corey Bryant (corey.bryant) wrote :

1.4.1 is working, thanks very much!

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers