Comment 6 for bug 1831224

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to networking-ovn (stable/stein)

Reviewed: https://review.opendev.org/663314
Committed: https://git.openstack.org/cgit/openstack/networking-ovn/commit/?id=5fe83cbcb5d2acc8af5e05d65345b114f859a359
Submitter: Zuul
Branch: stable/stein

commit 5fe83cbcb5d2acc8af5e05d65345b114f859a359
Author: Lucas Alvares Gomes <email address hidden>
Date: Fri May 31 11:39:39 2019 +0100

    Fix metadata agent proxy list updates race condition

    When multiple instances are being spawned and/or teared down at the
    same time, multiple updates to the "neutron-metadata-proxy-networks"
    list may race each other resulting in some networks UUIDs to go missing
    in that process.

    One of the problems with these updates is that the external_ids column
    in the OVSDBs only supports string type values so, the "list" is an
    actual comma-separated string. In order to add or remove an item from
    from it the code in networking-ovn needs to transform it from and back
    to string and commit the whole thing to OVN database when saving it.

    When it races, commits may overwrite each other.

    In order to fix this problem this patch is adding a file lock on the
    method that manipulates this list and commit it to the database. It's
    important to note that the metadata agent only updates the Chassis entry
    which it's currently running on, therefore we do not need a distributed
    lock (the file lock, being local works).

    I was able to reproduce this problem fairly consistently by concurrently
    spawning 50 instances each time with BrowBeat (see bug ticket for more
    information). After this fix was applied I could attest that the problem
    was resolved.

    Change-Id: Idc7bae88ed6f3ec44541b38f559a0296d2159449
    Closes-Bug: #1831224
    Signed-off-by: Lucas Alvares Gomes <email address hidden>
    (cherry picked from commit d84cb7938a153bd22a6fae5adbfbf8ec3b819bda)