Empty Fernet Key Files causing problems with token issue

Bug #1728907 reported by Divya K Konoor on 2017-10-31
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Identity (keystone)

Bug Description

The problem being reported is very similar to the one reported at https://bugs.launchpad.net/keystone/+bug/1642457 but not the same.

Step to reproduce (Not sure of the chances of being able to reproduce this again but these were the steps that happened when the problem was found):

1. Fernet token rotation is configured in this environment to run via a cron job every 3 hours. The primary key when things were working was 58. The system (where OpenStack was installed) went out of memory and in an attempt to recover, a reboot was initiated. As fate would have it, the reboot was initiated at 14.58 and the key rotate was to happen at 15:02. Keystone logs don't have any logging between 14.58 and 15:15. When the system was up, token issue was failing with

File "/usr/lib64/python2.7/site-packages/cryptography/fernet.py", line 37, in __init__
2017-10-26 15:46:30.613 4767 ERROR keystone.common.wsgi "Fernet key must be 32 url-safe base64-encoded bytes."
2017-10-26 15:46:30.613 4767 ERROR keystone.common.wsgi ValueError: Fernet key must be 32 url-safe base64-encoded bytes.

2. Soon after the above was noticed , a key rotation was attempted to see if that fixes anything (/usr/bin/keystone-manage fernet_rotate --keystone-user keystone --keystone-group keystone). And it did not.

3. When the fernet-keys directory was checked after step 3, an empty primary key file was found(60) . No other files were empty. This file was manually deleted after which the primary key became 59 and token issue continued to work.

System has no problem with disk space.

tags: added: fernet
Changed in keystone:
importance: Undecided → Low
status: New → Triaged
Lance Bragstad (lbragstad) wrote :

This seems somewhat familiar to bug 1642457.

tags: added: office-hours
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers