[FT] Test "test_agent_updated_at_use_nb_cfg_timestamp" failing

[Rodolfo Alonso]

Log: https://210aac0b09f7752823da-532d928235cdca0f1f7d2e9417dacd2f.ssl.cf2.rackcdn.com/842133/1/gate/neutron-functional-with-uwsgi/8001997/testr_results.html

Snippet: https://paste.opendev.org/show/bIdm72ReAc9QVURZFrG9/

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/842471

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/842471
Committed: https://opendev.org/openstack/neutron/commit/1b51ad28f3c8de78a8af71dab76f868cad5f6561
Submitter: "Zuul (22348)"
Branch: master

commit 1b51ad28f3c8de78a8af71dab76f868cad5f6561
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Fri May 13 14:58:34 2022 +0000

    Check the Chassis_Private nb_cfg_timestamp with current value

    In ``test_agent_updated_at_use_nb_cfg_timestamp`` now the test
    checks the local cache agent "updated_at" value with the current
    SB "Chassis_Private.nb_cfg_timestamp" value, read from the DB each
    time the value is tested.

    Closes-Bug: #1974149
    Change-Id: Ibd9ebfbd628d98a6a5de2b4fe47a9cbc65500c84

[yatin (yatinkarel)]

Seen again https://a725fc0d7f8b52d360c4-66ce5f117c645ca152390f12473225b2.ssl.cf5.rackcdn.com/797120/19/gate/neutron-functional-with-uwsgi/4b9fd54/testr_results.html

reopening bug

[yatin (yatinkarel)]

It failed as below:-
Traceback (most recent call last):
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/base.py", line 182, in func
    return f(self, *args, **kwargs)
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/plugins/ml2/drivers/ovn/mech_driver/ovsdb/test_ovsdb_monitor.py", line 483, in test_agent_updated_at_use_nb_cfg_timestamp
    self.fail('Chassis timestamp: %s, agent updated_at: %s' %
  File "/usr/lib/python3.8/unittest/case.py", line 753, in fail
    raise self.failureException(msg)
AssertionError: Chassis timestamp: 1655824139000, agent updated_at: 2022-06-21 15:08:58+00:00

So this happened as 1655824139000 equals to 2022-06-21 15:08:59+00:00, 1 second difference with agent time. agent time never updates when nb_cfg_timestamp changes as the event only triggers either on ROW_CREATE or if nb_cfg updates with chassis being set in chassis_private table.[1]
The test passes if nb_cfg_timestamp being updated(it happens withing a second of original db write) is same as what in already in Chassis_Private table.
I see two options to fix it:-
1) generate event also on nb_cfg_timestamp changes
2) Also update nb_cfg, which will trigger the event which will update the AgentCache and with this we can check agent.updated_at matches with nb_cfg_timestamp in chassis_private table

1 doesn't make sense to do just for the test case and i think nb_cfg_timestamp would not update alone, it get's updated with nb_cfg. Will push a patch as per 2.

[1] https://github.com/openstack/neutron/blob/dda8c70fbe3948de45bf72c5f9be7c657ebe42a9/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovsdb_monitor.py#L328-L330

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/847349

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/neutron/+/854541

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/854541
Committed: https://opendev.org/openstack/neutron/commit/1fda7ab887861250bc4d1ac87620d72b412eec5d
Submitter: "Zuul (22348)"
Branch: stable/yoga

commit 1fda7ab887861250bc4d1ac87620d72b412eec5d
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Fri May 13 14:58:34 2022 +0000

    Check the Chassis_Private nb_cfg_timestamp with current value

    In ``test_agent_updated_at_use_nb_cfg_timestamp`` now the test
    checks the local cache agent "updated_at" value with the current
    SB "Chassis_Private.nb_cfg_timestamp" value, read from the DB each
    time the value is tested.

    Closes-Bug: #1974149
    Change-Id: Ibd9ebfbd628d98a6a5de2b4fe47a9cbc65500c84
    (cherry picked from commit 1b51ad28f3c8de78a8af71dab76f868cad5f6561)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 21.0.0.0rc1 release candidate.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/847349
Committed: https://opendev.org/openstack/neutron/commit/1c89db495c884b0b562f64b263c3cd9c1f101699
Submitter: "Zuul (22348)"
Branch: master

commit 1c89db495c884b0b562f64b263c3cd9c1f101699
Author: yatinkarel <email address hidden>
Date: Thu Jun 23 18:49:51 2022 +0530

    [OVN][FT] Also update nb_cfg along with nb_cfg_timestamp

    FT ``test_agent_updated_at_use_nb_cfg_timestamp`` updates
    "nb_cfg_timestamp" in Chassis_Private table and then
    checks if agent cache get's updated to the same value.

    But "ChassisAgentWriteEvent" which is responsible for
    updating AgentCache doesn't get triggered with updates on
    "nb_cfg_timestamp", so this patch proposes also to increment
    "nb_cfg" by 1 to trigger the Event as in general scenario
    timestamep get's updated along with nb_cfg.

    Closes-Bug: #1974149
    Change-Id: I8889746ff2ee2629ee7c38ac162ea7c4d9ee0734

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/zed
Review: https://review.opendev.org/c/openstack/neutron/+/870507

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/neutron/+/870508

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/870507
Committed: https://opendev.org/openstack/neutron/commit/f88b61c8326f165226ff4af1b1fa0c89c96fa706
Submitter: "Zuul (22348)"
Branch: stable/zed

commit f88b61c8326f165226ff4af1b1fa0c89c96fa706
Author: yatinkarel <email address hidden>
Date: Thu Jun 23 18:49:51 2022 +0530

    [OVN][FT] Also update nb_cfg along with nb_cfg_timestamp

    FT ``test_agent_updated_at_use_nb_cfg_timestamp`` updates
    "nb_cfg_timestamp" in Chassis_Private table and then
    checks if agent cache get's updated to the same value.

    But "ChassisAgentWriteEvent" which is responsible for
    updating AgentCache doesn't get triggered with updates on
    "nb_cfg_timestamp", so this patch proposes also to increment
    "nb_cfg" by 1 to trigger the Event as in general scenario
    timestamep get's updated along with nb_cfg.

    Closes-Bug: #1974149
    Change-Id: I8889746ff2ee2629ee7c38ac162ea7c4d9ee0734
    (cherry picked from commit 1c89db495c884b0b562f64b263c3cd9c1f101699)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/870508
Committed: https://opendev.org/openstack/neutron/commit/2a2fda29c9a934174310627f10ade7ffe90879c1
Submitter: "Zuul (22348)"
Branch: stable/yoga

commit 2a2fda29c9a934174310627f10ade7ffe90879c1
Author: yatinkarel <email address hidden>
Date: Thu Jun 23 18:49:51 2022 +0530

    [OVN][FT] Also update nb_cfg along with nb_cfg_timestamp

    FT ``test_agent_updated_at_use_nb_cfg_timestamp`` updates
    "nb_cfg_timestamp" in Chassis_Private table and then
    checks if agent cache get's updated to the same value.

    But "ChassisAgentWriteEvent" which is responsible for
    updating AgentCache doesn't get triggered with updates on
    "nb_cfg_timestamp", so this patch proposes also to increment
    "nb_cfg" by 1 to trigger the Event as in general scenario
    timestamep get's updated along with nb_cfg.

    Closes-Bug: #1974149
    Change-Id: I8889746ff2ee2629ee7c38ac162ea7c4d9ee0734
    (cherry picked from commit 1c89db495c884b0b562f64b263c3cd9c1f101699)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 22.0.0.0rc1 release candidate.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 20.3.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 21.1.0 release.