Stale data causes keystone to raise an HTTP 500 during user password updates
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Identity (keystone) |
Fix Released
|
Undecided
|
Lance Bragstad |
Bug Description
If two clients attempt to update the same user, they are susceptible to a race condition in the update_user method of the sql driver:
I ran into this by using multiple (16) clients to run a script that updates the password for a user:
#!/bin/bash
set -xo
function cleanup {
for key in $( set | awk '{FS="="} /^OS_/ {print $1}' ); do unset $key ; done
exit
}
trap cleanup SIGINT
USERNAME=test-user
while true
do
PASSWORD=
openstack --os-cloud openstack user set --password $PASSWORD $USERNAME
if [ $? -ne 0 ]; then
echo -e "\e[31mFAIL"
cleanup
fi
done
Depending on the number of processes and threads, you will see an HTTP 500 with the following error either in keystone's log or http error logs:
[Tue Jun 30 16:29:45.241150 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241152 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241157 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241159 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241163 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241166 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241170 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241172 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241176 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241179 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241183 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241185 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241189 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241192 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241195 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241198 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241202 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241204 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241208 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241212 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241216 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241219 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241223 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241225 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241229 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241232 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241236 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241238 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241242 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241245 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241248 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241251 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241255 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241257 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241261 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241264 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241267 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241270 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241274 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241276 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241280 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241283 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241286 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241289 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241293 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241295 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241299 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241303 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241307 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241310 2020] [wsgi:error] [pid 33] [remote 192.168.
[Tue Jun 30 16:29:45.241325 2020] [wsgi:error] [pid 33] [remote 192.168.
Changed in keystone: | |
assignee: | Lance Bragstad (lbragstad) → Harry Rybacki (hrybacki-h) |
Changed in keystone: | |
assignee: | Harry Rybacki (hrybacki-h) → Lance Bragstad (lbragstad) |
summary: |
- Stale data causes keystone to raise an HTTP 500 during update updates + Stale data causes keystone to raise an HTTP 500 during user password + updates |
Changed in keystone: | |
status: | In Progress → Fix Released |
Fix proposed to branch: master /review. opendev. org/738677
Review: https:/