Murano API tests may fail if running in parallel

[Victor Ryzhenkin]

During parallel test ran the following error may occur.

ft12.2: murano_tempest_tests.tests.api.application_catalog.test_repository.TestRepositorySanity.test_upload_and_delete_package[smoke]_StringException: Empty attachments:
  stderr
  stdout

pythonlogging:'': {{{
2016-02-12 08:38:40,697 12586 INFO [tempest_lib.common.rest_client] Request (TestRepositorySanity:test_upload_and_delete_package): 200 GET http://127.0.0.1:8082/v1/catalog/packages 0.064s
2016-02-12 08:38:40,698 12586 DEBUG [tempest_lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
        Body: None
    Response - Headers: {'content-type': 'application/json', 'connection': 'close', 'status': '200', 'content-location': 'http://127.0.0.1:8082/v1/catalog/packages', 'content-length': '2246', 'x-openstack-request-id': 'req-e5799aad-df0d-4650-892f-f0b8e85fee30', 'date': 'Fri, 12 Feb 2016 08:38:40 GMT'}
        Body: {"packages": [{"updated": "2016-02-12T08:36:41", "description": "Core MuranoPL library\n", "tags": ["MuranoPL"], "class_definitions": ["io.murano.resources.ExistingCinderVolume", "io.murano.system.SecurityGroupManager", "io.murano.resources.CinderVolume", "io.murano.resources.CinderVolumeSnapshot", "io.murano.resources.ExistingNeutronNetwork", "io.murano.resources.NovaNetwork", "io.murano.resources.Network", "io.murano.resources.LinuxMuranoInstance", "io.murano.resources.NeutronNetworkBase", "io.murano.SharedIp", "io.murano.test.TestFixture", "io.murano.resources.Instance", "io.murano.system.Logger", "io.murano.Environment", "io.murano.File", "io.murano.resources.LinuxInstance", "io.murano.system.StatusReporter", "io.murano.Exception", "io.murano.StackTrace", "io.murano.resources.NeutronNetwork", "io.murano.configuration.Linux", "io.murano.resources.ConfLangInstance", "io.murano.resources.Volume", "io.murano.system.AwsSecurityGroupManager", "io.murano.system.Resources", "io.murano.system.NetworkExplorer", "io.murano.resources.HeatSWConfigLinuxInstance", "io.murano.resources.WindowsInstance", "io.murano.system.NeutronSecurityGroupManager", "io.murano.Object", "io.murano.system.HeatStack", "io.murano.resources.CinderVolumeBackup", "io.murano.system.InstanceNotifier", "io.murano.system.AgentListener", "io.murano.resources.LinuxUDInstance", "io.murano.system.Agent", "io.murano.system.MistralClient", "io.murano.Application", "io.murano.resources.HeatSWConfigInstance"], "is_public": true, "id": "e48b3682b6944173b17ece651266e561", "categories": [], "name": "Core library", "created": "2016-02-12T08:36:41", "author": "murano.io", "enabled": true, "supplier": {}, "fully_qualified_name": "io.murano", "type": "Library", "owner_id": ""}, {"updated": "2016-02-12T08:38:37", "description": "New description", "tags": [], "class_definitions": ["io.murano.apps.test_repository_class_bb265997"], "is_public": true, "id": "21a3c9ef77ab44f4a779aec9106941b8", "categories": [], "name": "New name", "created": "2016-02-12T08:38:36", "author": "Mirantis, Inc", "enabled": true, "supplier": {}, "fully_qualified_name": "io.murano.apps.test_repository_class_bb265997", "type": "Application", "owner_id": "b9a2d6f441e149229d3089cfd0411d81"}]}
2016-02-12 08:38:40,816 12586 INFO [tempest_lib.common.rest_client] Request (TestRepositorySanity:test_upload_and_delete_package): 200 GET http://127.0.0.1:8082/v1/catalog/packages 0.046s
2016-02-12 08:38:40,817 12586 DEBUG [tempest_lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
        Body: None
    Response - Headers: {'content-type': 'application/json', 'connection': 'close', 'status': '200', 'content-location': 'http://127.0.0.1:8082/v1/catalog/packages', 'content-length': '2269', 'x-openstack-request-id': 'req-cc8fe400-75f2-4015-baa6-550034af5a47', 'date': 'Fri, 12 Feb 2016 08:38:40 GMT'}
        Body: {"packages": [{"updated": "2016-02-12T08:36:41", "description": "Core MuranoPL library\n", "tags": ["MuranoPL"], "class_definitions": ["io.murano.resources.ExistingCinderVolume", "io.murano.system.SecurityGroupManager", "io.murano.resources.CinderVolume", "io.murano.resources.CinderVolumeSnapshot", "io.murano.resources.ExistingNeutronNetwork", "io.murano.resources.NovaNetwork", "io.murano.resources.Network", "io.murano.resources.LinuxMuranoInstance", "io.murano.resources.NeutronNetworkBase", "io.murano.SharedIp", "io.murano.test.TestFixture", "io.murano.resources.Instance", "io.murano.system.Logger", "io.murano.Environment", "io.murano.File", "io.murano.resources.LinuxInstance", "io.murano.system.StatusReporter", "io.murano.Exception", "io.murano.StackTrace", "io.murano.resources.NeutronNetwork", "io.murano.configuration.Linux", "io.murano.resources.ConfLangInstance", "io.murano.resources.Volume", "io.murano.system.AwsSecurityGroupManager", "io.murano.system.Resources", "io.murano.system.NetworkExplorer", "io.murano.resources.HeatSWConfigLinuxInstance", "io.murano.resources.WindowsInstance", "io.murano.system.NeutronSecurityGroupManager", "io.murano.Object", "io.murano.system.HeatStack", "io.murano.resources.CinderVolumeBackup", "io.murano.system.InstanceNotifier", "io.murano.system.AgentListener", "io.murano.resources.LinuxUDInstance", "io.murano.system.Agent", "io.murano.system.MistralClient", "io.murano.Application", "io.murano.resources.HeatSWConfigInstance"], "is_public": true, "id": "e48b3682b6944173b17ece651266e561", "categories": [], "name": "Core library", "created": "2016-02-12T08:36:41", "author": "murano.io", "enabled": true, "supplier": {}, "fully_qualified_name": "io.murano", "type": "Library", "owner_id": ""}, {"updated": "2016-02-12T08:38:40", "description": "MockApp for API tests", "tags": [], "class_definitions": ["io.murano.apps.package_test_upload_70f90241"], "is_public": false, "id": "b30fcec7206142f49e57fb7ccc6398a5", "categories": [], "name": "package_test_upload_70f90241", "created": "2016-02-12T08:38:40", "author": "Mirantis, Inc", "enabled": true, "supplier": {}, "fully_qualified_name": "io.murano.apps.package_test_upload_70f90241", "type": "Application", "owner_id": "3c1666ffc0d244a8bb874faad1f34731"}]}
}}}

Traceback (most recent call last):
  File "/opt/stack/new/murano/murano_tempest_tests/tests/api/application_catalog/test_repository.py", line 42, in test_upload_and_delete_package
    self.assertEqual(len(packages_list) + 1, len(updated_packages_list))
  File "/usr/local/lib/python2.7/dist-packages/testtools/testcase.py", line 362, in assertEqual
    self.assertThat(observed, matcher, message)
  File "/usr/local/lib/python2.7/dist-packages/testtools/testcase.py", line 447, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: 3 != 2

This happened due to this test using package list for verification.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/279957

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/279957
Committed: https://git.openstack.org/cgit/openstack/murano/commit/?id=3658e2e20a74cd8a335405f027d31424e8c303e3
Submitter: Jenkins
Branch: master

commit 3658e2e20a74cd8a335405f027d31424e8c303e3
Author: Victor Ryzhenkin <email address hidden>
Date: Sat Feb 13 17:08:17 2016 +0300

    Don't use list lenght check in Repository test suite

    - Replace assertEqual for package list checks to assertIn and assertNotIn
    for object in TestRepositorySanity class.

    Change-Id: Iec3c1edbb98de3332888dd01b2b428c8496c3fa1
    Closes-Bug: #1545090

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/287268

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/287268
Committed: https://git.openstack.org/cgit/openstack/murano/commit/?id=039114c36fa30681dfaa50ca4b6d2a5731fe2adb
Submitter: Jenkins
Branch: master

commit 039114c36fa30681dfaa50ca4b6d2a5731fe2adb
Author: Victor Ryzhenkin <email address hidden>
Date: Wed Mar 2 18:37:26 2016 +0300

    Do not use list lenght checks in Env-Templates test suite

    - Replace assertEqual for package list checks to assertIn and assertNotIn
    for object in TestEnvironmentTemplatesSanity class.

    Change-Id: I0ef348438bd8ec3cc4ef6027fdc95165495390d5
    Related-Bug: #1545090

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/murano 2.0.0.0b3 development milestone.