Undercloud upgrade seems not idempotent if repeated after a failed attempt

[Bogdan Dobrelya]

While I was testing https://review.opendev.org/695736 fix for https://launchpad.net/bugs/1853183,

I discovered the following errors in the UC upgrade logs:

        "Error: error creating container storage: the container name \"rabbitmq_init_logs\" is already in use by \"b9c77154d6df062e8eca442fdb48c20e5a108b5ef407c26e28e478c33e393fd3\". You have to remove that container to be able to reuse that name.: that name is already in use",
        "Error: error creating container storage: the container name \"mysql_init_logs\" is already in use by \"4e94d119836bce6fc512a4bfdc87302d2a10c62b4fbb97712648fad935de6265\". You have to remove that container to be able to reuse that name.: that name is already in use",
        "Error: error creating container storage: the container name \"memcached\" is already in use by \"d856c43f807bf26d8d2fe947f71324b31541d80c2ef43a2e8b7d4580bb2fad36\". You have to remove that container to be able to reuse that name.: that name is already in use",
        "Error: error creating container storage: the container name \"keepalived\" is already in use by \"2f29f8b577a3c160592c1b6a856b82677f10a1ba2aed0be0f9774202f397b2f0\". You have to remove that container to be able to reuse that name.: that name is already in use",
        "Error: error creating container storage: the container name \"haproxy\" is already in use by \"8173648393cb0ec50e3dbfaa1f467fcc66fadc6712277965821e49396cddd420\". You have to remove that container to be able to reuse that name.: that name is already in use",
        "Error: error creating container storage: the container name \"rabbitmq_bootstrap\" is already in use by \"e6918662b4a470141377d2746ce9c9545a4f82e9869cbd01847b3ecdef59b027\". You have to remove that container to be able to reuse that name.: that name is already in use",
        "Error: error creating container storage: the container name \"rabbitmq\" is already in use by \"08a3bb98733360236d01fb3db380bda677b6f17b1f731d506894bb67d786c38a\". You have to remove that container to be able to reuse that name.: that name is already in use"

Those containers are real (so that's not a podman issue) and can be inspected.
That seems like an idempotency issue

[Cédric Jeanneret (cjeanner)]

hmmm... might be linked to paunch. Would be interesting to get some more logs.

[Bogdan Dobrelya (bogdando)]

@Cedric, here it is for an example container:

[zuul@subnode-1 ~]$ grep 08a3bb98733360236d01fb3db380bda677b6f17b1f731d506894bb67d786c38a /var/log/paunch.log
2019-11-21 15:02:18.762 28492 INFO paunch [ ] stdout: 08a3bb98733360236d01fb3db380bda677b6f17b1f731d506894bb67d786c38a
2019-11-21 16:50:18.484 246462 ERROR paunch [ ] stderr: Error: error creating container storage: the container name "rabbitmq" is already in use by "08a3bb98733360236d01fb3db380bda677b6f17b1f731d506894bb67d786c38a". You have to remove that container to be able to reuse that name.: that name is already in use