tox gives invalid uuid warnings in unit test

[Srushti Gadadare]

Currently 'tox -epy27' emits 318 FutureWarnings from oslo.versionedobjects about invalid uuids.
When invalid uuids are used in the test cases, tox gives FutureWarnings.

Steps to reproduce:

Run tox -epy27 command.

Sample Output:

{3}nova.tests.unit.api.openstack.compute.legacy_v2.test_servers.ServersControllerCreateTest.test_create_instance_with_disk_config_disabled [0.266687s] ... ok
{1}nova.tests.unit.api.openstack.compute.test_server_groups.ServerGroupTestV2.test_create_server_group_with_duplicate_policies [0.059925s] ... ok
{0} nova.tests.unit.api.openstack.compute.test_server_group_quotas.ServerGroupQuotasTestV2.test_delete_server_group_by_id [0.128866s] ... ok

Captured stderr:
~~~~~~~~~~~~~~~~
    /opt/stack/nova/.tox/py27/local/lib/python2.7/site-packages/oslo_versionedobjects/fields.py:325: FutureWarning: fake-uuid is an invalid UUID. Using UUIDFields with invalid UUIDs is no longer supported, and will be removed in a future release. Please update your code to input valid UUIDs or accept ValueErrors for invalid UUIDs. See http://docs.openstack.org/developer/oslo.versionedobjects/api/fields.html#oslo_versionedobjects.fields.UUIDField for further details
      "for further details" % value, FutureWarning)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/293840

[Markus Zoeller (markus_z) (mzoeller)]

That's the ML discussion about the emit of warnings: http://lists.openstack.org/pipermail/openstack-dev/2016-January/085252.html

[Markus Zoeller (markus_z) (mzoeller)]

IMO it's ok to use this bug as a tracker item as it probably involves the effort of multiple people.

[Markus Zoeller (markus_z) (mzoeller)]

Also good to know:
* "Add uuidsentinel test module " [1]
* "Add UUID Validation" [2]

Possible contact: migi (https://launchpad.net/~michal-pryc/)

References:
[1] https://github.com/openstack/nova/commit/efe7df92393a6d9f2587ce9e13d9f90e5c4f3ee9
[2] https://review.openstack.org/#/c/250493/11

[Markus Zoeller (markus_z) (mzoeller)]

Changes needed as pre-req: https://review.openstack.org/#/q/topic:%255EUUID-VALIDATION-PREP.%252B

[Michal Pryc (mpryc)]

To implement uuid properly across nova tests the easiest is to apply patch from 4th patch set to the oslo.versionedobjects (one that throws error, even if it's not yet merged/approved) and then run tests against specific test cases in nova:

https://review.openstack.org/#/c/250493/4

this will highlight the number of current places to be changed.

[Michal Pryc (mpryc)]

The attached file shows py27 run to validate uuids, currently there are 827 tests failing (source based on the review https://review.openstack.org/#/c/293840/)

[Michal Pryc (mpryc)]

The attached file shows functional run to validate uuids, currently there are 18 tests failing (source based on the review https://review.openstack.org/#/c/293840/)

[Michal Pryc (mpryc)]

Note that comments #7 and #8 are valid only if oslo_versionedobjects/fields.py is set to be strict enough to raise error on non valid UUID rather then just show warning.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/300453

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/300458

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/300477

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/300482

[Matt Riedemann (mriedem)]

This is a duplicate of bug 1554230.

[Matt Riedemann (mriedem)]

Ignore comment 14, I've duplicated that bug against this one since this one has changes up for review and that one doesn't.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/301650

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/302655

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/302658

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/300477
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=3e527265b6da73f73a669c88ae2239ed6bcf69b1
Submitter: Jenkins
Branch: master

commit 3e527265b6da73f73a669c88ae2239ed6bcf69b1
Author: srushti <email address hidden>
Date: Thu Mar 24 06:38:06 2016 +0000

    Fix invalid uuid warnings in api testcases

    Currently, unit testcases of api emit FutureWarnings
    from oslo.versionedobjects about invalid uuids as documented here[1].

    This commit changes these tests to use valid uuids so
    that when they are run these warnings are no longer emitted.

    Made changes to project_id in fakes.py to use valid project_id
    which removes warnings related to invalid uuids in testcases.

    Also made changes to project_id in JSON headers to use valid
    uuids in functional testcases same as that in unit testcases.

    [1] http://docs.openstack.org/developer/oslo.versionedobjects/api/fields.html#oslo_versionedobjects.fields.UUIDField

    Partial-Bug: #1557378
    Change-Id: I5f69a2c3dc4d0279aed11750e6758b624debb64f

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/300482
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=54aaa829d6aad50528541e1a79b07f7a76bdb8bc
Submitter: Jenkins
Branch: master

commit 54aaa829d6aad50528541e1a79b07f7a76bdb8bc
Author: srushti <email address hidden>
Date: Mon Mar 28 10:21:15 2016 +0000

    Fix invalid uuid warnings in cell api testcases

    Currently, unit testcases of cell api emit FutureWarnings
    from oslo.versionedobjects about invalid uuids as documented here[1].

    This commit changes these tests to use valid uuids so
    that when they are run these warnings are no longer emitted.

    [1] http://docs.openstack.org/developer/oslo.versionedobjects/api/fields.html#oslo_versionedobjects.fields.UUIDField

    Partial-Bug: #1557378
    Change-Id: If4176e59780e9d43bf7e0e48ee8230ce006e8d8b

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/293840
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=50354963cecbfe5a6231afc7fc09fe236c64d4d8
Submitter: Jenkins
Branch: master

commit 50354963cecbfe5a6231afc7fc09fe236c64d4d8
Author: srushti <email address hidden>
Date: Tue Apr 12 10:50:16 2016 +0000

    Fix invalid uuid warnings in server-group unit tests

    Currently, testcases of server-group api emit FutureWarnings
    from oslo.versionedobjects about invalid uuids as documented here[1].

    This commit changes these tests to use valid uuids so
    that when they are run these warnings are no longer emitted.

    Fake constants are defined for project_id and user_id as they
    are used without '-' at keystone side.

    [1] http://docs.openstack.org/developer/oslo.versionedobjects/api/fields.html#oslo_versionedobjects.fields.UUIDField

    Partial-Bug: #1557378
    Change-Id: If2b2976ee339431cf5c32df88ad713ab2b6cd880

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/301650
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=69700590db831dd1718ed3ed57ca6ab9bd445728
Submitter: Jenkins
Branch: master

commit 69700590db831dd1718ed3ed57ca6ab9bd445728
Author: srushti <email address hidden>
Date: Wed Apr 13 09:42:12 2016 +0000

    Fix invalid uuid warnings in objects testcases

    Currently, unit testcases of objects emit FutureWarnings
    from oslo.versionedobjects about invalid uuids as documented here[1].

    This commit changes these tests to use valid uuids so
    that when they are run these warnings are no longer emitted.

    Fake constants are defined for project_id and user_id as they
    are used without '-' at keystone side

    [1] http://docs.openstack.org/developer/oslo.versionedobjects/api/fields.html#oslo_versionedobjects.fields.UUIDField

    Partial-Bug: #1557378
    Change-Id: Ia28952ecfeaa5090fa65638af488f88400dc02d2

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/300458
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=60aa0a18c69a2b004477689b519b780bdb81f2fb
Submitter: Jenkins
Branch: master

commit 60aa0a18c69a2b004477689b519b780bdb81f2fb
Author: srushti <email address hidden>
Date: Wed Mar 30 05:40:26 2016 +0000

    Fix invalid uuid warnings

    Currently, unit testcases of conductor, image and network
    emit FutureWarnings from oslo.versionedobjects about invalid
    uuids as documented here[1].

    This commit changes these tests to use valid uuids so
    that when they are run these warnings are no longer emitted.

    Fake constants are defined for project_id and user_id as they
    are used without '-' at keystone side

    [1] http://docs.openstack.org/developer/oslo.versionedobjects/api/fields.html#oslo_versionedobjects.fields.UUIDField

    Partial-Bug: #1557378
    Change-Id: I93cd3782e5b40e381d15bacbe8720a8c4a3bef45

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/300453
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=09b7c7b9336732a34d3273a917bcdda288645220
Submitter: Jenkins
Branch: master

commit 09b7c7b9336732a34d3273a917bcdda288645220
Author: srushti <email address hidden>
Date: Tue Mar 29 12:48:19 2016 +0000

    Fix invalid uuid warnings in compute api testcases

    Currently, unit testcases of compute api emit FutureWarnings
    from oslo.versionedobjects about invalid uuids as documented here[1].

    This commit changes these tests to use valid uuids so
    that when they are run these warnings are no longer emitted.

    Fake constants are defined for project_id and user_id as they
    are used without '-' at keystone side.

    [1] http://docs.openstack.org/developer/oslo.versionedobjects/api/fields.html#oslo_versionedobjects.fields.UUIDField

    Partial-Bug: #1557378
    Change-Id: I71ae53edd183a9d7d80f6cf1b6bb23902f5c6f35

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/311294

[Srushti Gadadare (srushti-gadadare)]

Hi Michal Stolarczyk,

I have already submitted 8 patches to fix this uuid warnings out of which 6 are merged and two are remaining.
There are no more warnings after applying these 8 patches on the master code.

I have covered the changes which you have submitted in your patch. The patch which you have submitted does not make any sense to me.
Please Refer: https://review.openstack.org/#/c/302658/

Not sure, why you submitted this patch?

[Michal Stolarczyk (zerodivisible)]

Hi Srushti,

Sorry for submitting this patch - I thought I had picked a part of UUID warnings which wasn't worked on as I had been looking for something simple to do, to start digging in the Nova source code. I somehow missed the fact that change which I had worked on is covered by one of your patches. I'll abandon my review.

Thanks for reaching out and pointing me to your review.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Michal Stolarczyk (<email address hidden>) on branch: master
Review: https://review.openstack.org/311294
Reason: Abandoning as this change is already covered by https://review.openstack.org/#/c/302658/

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/317400

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/302658
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=b1d9c3edd87fb9e547c5e374d836b673989747a7
Submitter: Jenkins
Branch: master

commit b1d9c3edd87fb9e547c5e374d836b673989747a7
Author: srushti <email address hidden>
Date: Tue Apr 5 13:27:43 2016 +0000

    Fix invalid uuid warnings

    Currently, unit testcases emit FutureWarnings from oslo.versionedobjects
    about invalid uuids as documented here[1].

    This commit changes these tests to use valid uuids so
    that when they are run these warnings are no longer emitted.

    [1] http://docs.openstack.org/developer/oslo.versionedobjects/api/fields.html#oslo_versionedobjects.fields.UUIDField

    Closes-Bug: #1557378
    Change-Id: I21a60e916e183f7c0fc6193d4b7caa88ae3b6d56

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/316723
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=e0f61dbbbc0d64a4c4fbbeeb5a3221d1947447f4
Submitter: Jenkins
Branch: master

commit e0f61dbbbc0d64a4c4fbbeeb5a3221d1947447f4
Author: ChangBo Guo(gcb) <email address hidden>
Date: Mon May 16 18:38:45 2016 +0800

    Fixes invalid uuid usages in functional tests

    oslo_versionedobjects.fields.UUIDField will validate UUID in next major
    release. So need fix invalid uuid usage before moving to new major
    release of oslo.versionedobjects.

    Releted-Bug: #1557378
    Change-Id: I1dc56cbec42cc385a87886c63f4cf33b68397a29

[Michal Pryc (mpryc)]

Hi,

If I may ask to not work/propose UUID change for the following file - I will propose it Today as part of the 1st Wrocław OpenStack Meetup life session:

nova/tests/unit/network/test_neutronv2.py

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/324757

[Davanum Srinivas (DIMS) (dims-v)]

This issue was fixed in the openstack/nova 14.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/324757
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=ca8f4d9ea98e2a51cdc104daf6229b1dfee405c0
Submitter: Jenkins
Branch: master

commit ca8f4d9ea98e2a51cdc104daf6229b1dfee405c0
Author: Michal Pryc <email address hidden>
Date: Thu Jun 2 20:36:24 2016 +0200

    Fixes invalid uuid usages in test_neutronv2

    Change-Id: If60e73c85bf6f860dec59c5bc73c6b13e8449f9f
    Related-Bug: #1557378

[Yatin Kumbhare (yatinkumbhare-c)]

@DIMS

https://github.com/openstack/nova/commits/14.0.0.0b1

Looked over it, sorry, couldn't find the commit you are refering.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/302655
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=7c10322b1738da90ecd3687dd49277f9e18a18e5
Submitter: Jenkins
Branch: master

commit 7c10322b1738da90ecd3687dd49277f9e18a18e5
Author: srushti <email address hidden>
Date: Wed Apr 6 05:46:56 2016 +0000

    Fix invalid uuid warnings in virt testcases

    Currently, unit testcases of virt emit FutureWarnings
    from oslo.versionedobjects about invalid uuids as documented here[1].

    This commit changes these tests to use valid uuids so
    that when they are run these warnings are no longer emitted.

    Fake constants are defined for project_id and user_id as they
    are used without '-' at keystone side.

    [1] http://docs.openstack.org/developer/oslo.versionedobjects/api/fields.html#oslo_versionedobjects.fields.UUIDField

    Partial-Bug: #1557378
    Change-Id: If69412eddb5e64cb8ccc5dce9176d19382c0b3f1

[OpenStack Infra (hudson-openstack)]

Change abandoned by Michael Still (<email address hidden>) on branch: master
Review: https://review.openstack.org/317400
Reason: This patch has been sitting unchanged for more than 12 weeks. I am therefore going to abandon it to keep the nova review queue sane. Please feel free to restore the change if you're still working on it.