Compare of volume in tempest.scenario.test_minimum_basic.TestMinimumBasicScenario.test_minimum_basic_scenario shouldn't check "updated_at" attribute

[Slawek Kaplonski]

I noticed failure of one of the scenario jobs with error in tempest.scenario.test_minimum_basic.TestMinimumBasicScenario.test_minimum_basic_scenario. Error was like:

Traceback (most recent call last):
  File "/opt/stack/tempest/tempest/common/utils/__init__.py", line 89, in wrapper
    return f(*func_args, **func_kwargs)
  File "/opt/stack/tempest/tempest/scenario/test_minimum_basic.py", line 118, in test_minimum_basic_scenario
    self.cinder_show(volume)
  File "/opt/stack/tempest/tempest/scenario/test_minimum_basic.py", line 64, in cinder_show
    self.assertEqual(volume, got_volume)
  File "/opt/stack/tempest/.tox/tempest/lib/python3.6/site-packages/testtools/testcase.py", line 411, in assertEqual
    self.assertThat(observed, matcher, message)
  File "/opt/stack/tempest/.tox/tempest/lib/python3.6/site-packages/testtools/testcase.py", line 498, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: !=:
reference = {'attachments': [],
 'availability_zone': 'nova',
 'bootable': 'false',
 'consistencygroup_id': None,
 'created_at': '2020-02-07T19:55:11.000000',
 'description': None,
 'encrypted': False,
 'id': '4daa2ca2-cc4f-4ed0-9d1c-4e49aa8127a7',
 'links': [{'href': 'https://149.202.185.27/volume/v3/e5c8a87889934809908d20c1b780d5a0/volumes/4daa2ca2-cc4f-4ed0-9d1c-4e49aa8127a7',
            'rel': 'self'},
           {'href': 'https://149.202.185.27/volume/e5c8a87889934809908d20c1b780d5a0/volumes/4daa2ca2-cc4f-4ed0-9d1c-4e49aa8127a7',
            'rel': 'bookmark'}],
 'metadata': {},
 'multiattach': False,
 'name': 'tempest-TestMinimumBasicScenario-volume-686560435',
 'os-vol-tenant-attr:tenant_id': 'e5c8a87889934809908d20c1b780d5a0',
 'replication_status': None,
 'size': 1,
 'snapshot_id': None,
 'source_volid': None,
 'status': 'available',
 'updated_at': '2020-02-07T19:55:20.000000',
 'user_id': '7ef370587325403aac96ab3e328149dd',
 'volume_type': 'lvmdriver-1'}
actual = {'attachments': [],
 'availability_zone': 'nova',
 'bootable': 'false',
 'consistencygroup_id': None,
 'created_at': '2020-02-07T19:55:11.000000',
 'description': None,
 'encrypted': False,
 'id': '4daa2ca2-cc4f-4ed0-9d1c-4e49aa8127a7',
 'links': [{'href': 'https://149.202.185.27/volume/v3/e5c8a87889934809908d20c1b780d5a0/volumes/4daa2ca2-cc4f-4ed0-9d1c-4e49aa8127a7',
            'rel': 'self'},
           {'href': 'https://149.202.185.27/volume/e5c8a87889934809908d20c1b780d5a0/volumes/4daa2ca2-cc4f-4ed0-9d1c-4e49aa8127a7',
            'rel': 'bookmark'}],
 'metadata': {},
 'multiattach': False,
 'name': 'tempest-TestMinimumBasicScenario-volume-686560435',
 'os-vol-tenant-attr:tenant_id': 'e5c8a87889934809908d20c1b780d5a0',
 'replication_status': None,
 'size': 1,
 'snapshot_id': None,
 'source_volid': None,
 'status': 'available',
 'updated_at': '2020-02-07T19:55:25.000000',
 'user_id': '7ef370587325403aac96ab3e328149dd',
 'volume_type': 'lvmdriver-1'}

Failure logs are at https://fbc85e1f3d41b6a16109-d3b52ba1934e6e5e97c6b41e7118c3af.ssl.cf1.rackcdn.com/681400/9/check/neutron-tempest-linuxbridge/d5890cc/testr_results.html
The only difference between actual and expected volume is "updated_at" key.
I don't think that "updated_at" key is critical in this test and should be really checked. Maybe tempest should filter this key from comparison?

[Martin Kopec (mkopec)]

Confirming, I've seen the issue in the gates as well, I agree updated_at shouldn't be compared although I'm curious why it is flaky, I'll look into it.