Magnum pod-delete didn't remove record in DB

[hongbin]

If a pod is in Magnum DB but not exist remotely. The pod delete command succeeded but it didn't delete the record at DB. Below is the details:

# this success
$ magnum pod-delete --id 5b694989-31f5-43d7-a2fc-9be8bad223a2

# but the pod still there
$ magnum pod-list
+--------------------------------------+---------+
| uuid | name |
+--------------------------------------+---------+
| 5b694989-31f5-43d7-a2fc-9be8bad223a2 | mariadb |
+--------------------------------------+---------+

# this is the log of conductor
DEBUG magnum.conductor.handlers.common.kube_utils [-] pod_delete mariadb pod_delete /home/ubuntu/magnum/magnum/conductor/handlers/common/kube_utils.py:190
DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): kubectl delete pod mariadb -s http://192.168.1.228:8080 execute /usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:191
DEBUG oslo_concurrency.processutils [-] CMD "kubectl delete pod mariadb -s http://192.168.1.228:8080" returned: 255 in 0.038s execute /usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:216
DEBUG oslo_concurrency.processutils [-] u'kubectl delete pod mariadb -s http://192.168.1.228:8080' failed. Not Retrying. execute /usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:249

# I manually run the command
$ kubectl delete pod mariadb -s http://192.168.1.228:8080
F0119 22:25:18.947963 23864 delete.go:60] pod "mariadb" not found

[Andrew Melton (andrew-melton)]

This is happening because when the delete call to the CLI fails, the delete is short circuited and the database delete never happens.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/157078

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/157078
Committed: https://git.openstack.org/cgit/stackforge/magnum/commit/?id=a5598a52a9b992740a6ae164545279b4d2ee42e9
Submitter: Jenkins
Branch: master

commit a5598a52a9b992740a6ae164545279b4d2ee42e9
Author: Andrew Melton <email address hidden>
Date: Wed Feb 18 09:31:45 2015 -0800

    Allow pod delete to succeed when not found on bay

    If a pod is deleted from kubernetes outside of magnum, and then
    a pod-delete is issued through magnum, the delete will succeed
    but will leave the record in the database. This change raises
    an exception when the pod is not found, but catches it in the
    conductor and proceeds to delete the record from the database.

    Change-Id: I83a6b3e68eef434fb7d1b4cba92acb17abc6bc4d
    Closes-bug: #1412587