Comment 19 for bug 1826886

Latest case reported by Mitchell is not the same as Peng reported I think.

For the latest report, I think this error is caused by resource limited.
Let's take req-aaffb3cb-f95d-45a3-8716-8e237ee86d39 as an example. As posted in #comment-15, cinder-api received HTTP-500. What I have found in the log of controller-1 (/var/log/containers/cinder-api-8548c4c585-s9lxl_openstack_cinder-api-5c984b95c1058fb9900da2125445bcc4d41dfac2353d7bb0700796d1d611c981.log) at line-7461 is
{"log":"2019-11-29 22:31:43.910 19 ERROR oslo_db.sqlalchemy.exc_filters [req-aaffb3cb-f95d-45a3-8716-8e237ee86d39 f9f2c5d355544777808fc2ea96ae5c9a 109492cf8aed4438b7ecb126063c8025 - default default] DBAPIError exception wrapped from (pymysql.err.InternalError) (1021, u'Disk full (/tmp/#sql_136_1.MAI); waiting for someone to free some space... (errno: 28 \"No space left on device\")')\n","stream":"stdout","time":"2019-11-29T22:31:43.911447596Z"}

DB returned "disk full", which I think caused this error. Plus I found a lot of 'Evicted' status of cinder-volume pods in #comment-16, maybe the "not-enough-space-error" happened on the storage node caused the eviction of cinder-volume pods.