Comment 7 for bug 1979089

Revision history for this message
Edward Hope-Morley (hopem) wrote :

This is proving quite a hard issue to reproduce in the lab but further analysis of the logs seems to indicate this might be related to the router being deleted immediately after creation. Here is the sequence of updates and transitions that occur for the mentioned router. Immediately after the delete update finishes, the neutron.agent.linux.external_process.ProcessMonitor wakes up, notices haproxy is no longer running, attempts to spawn a new one which then fails because the namespace no longer exists. I still believe this is related to the agent putting the transition thread to sleep just before the delete is processes and it re-awakens after assuming the router is still there.

router update dfe9df14-d58f-4114-a909-dddb0679736b
    type: ADD_UPDATE_ROUTER
    start: 2022-05-10 23:33:09.473
    finish: 2022-05-10 23:33:40.168
router transition:
    state: backup
    enqueue: 2022-05-10 23:33:23.272
    transition: 2022-05-10 23:33:23.272
router update 285660c8-58df-4fb0-9401-13b079eadb
    type: ADD_UPDATE_ROUTER
    start: 2022-05-10 23:33:40.168
    finish: 2022-05-10 23:33:47.745
router update 4c9f263a-51d5-440d-bc44-7a10a2beb36e
    type: ADD_UPDATE_ROUTER
    start: 2022-05-10 23:34:10.909
    finish: 2022-05-10 23:34:33.659
router update 9478e3bb-b68d-41b6-8088-0888d6f9a681
    type: ADD_UPDATE_ROUTER
    start: 2022-05-10 23:35:29.244
    finish: 2022-05-10 23:35:37.686
router update 6137df6c-237c-4740-9c91-aa97f1777a7e
    type: ADD_UPDATE_ROUTER
    start: 2022-05-10 23:35:37.687
    finish: 2022-05-10 23:35:45.696
router update 49c31d3c-44ba-4fe7-9e3c-6af6b5679875
    type: ADD_UPDATE_ROUTER
    start: 2022-05-10 23:35:56.177
    finish: 022-05-10 23:36:10.645
router transition:
    state: master
    enqueue: 2022-05-10 23:36:08.479
    transition: 2022-05-10 23:36:10.480
router update f3825e32-e03e-486d-8891-5fd38a8389a7
    type: DELETE_ROUTER
    start: 2022-05-10 23:36:10.646
    finish: 2022-05-10 23:36:17.032