Barbican

MultipleResultsFound error in _find_or_create_kek_objects()

Bug #1726378 reported by Dave McCowan
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Barbican
Fix Released
High
Unassigned

Bug Description

Found during Castellan functional tests gate jobs.

Trackback and Exception.
Seems to be a race condition due to not critical section between find and create calls for _find_or_create_kek_objects().

Oct 23 11:40:22.073734 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.069 3590 ERROR barbican.tasks.resources [req-34b61d48-0a29-407a-b690-a8edd91e91b4 7cbcaa3318c342999fec01b5c671d87d - - default default] Could not perform processing for task 'Process TypeOrder'.: MultipleResultsFound: Multiple rows were found for one()
Oct 23 11:40:22.074080 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.069 3590 ERROR barbican.tasks.resources Traceback (most recent call last):
Oct 23 11:40:22.074391 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.069 3590 ERROR barbican.tasks.resources File "/opt/stack/new/barbican/barbican/tasks/resources.py", line 100, in process
Oct 23 11:40:22.074693 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.069 3590 ERROR barbican.tasks.resources result = self.handle_processing(entity, *args, **kwargs)
Oct 23 11:40:22.075518 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.069 3590 ERROR barbican.tasks.resources File "/opt/stack/new/barbican/barbican/tasks/resources.py", line 247, in handle_processing
Oct 23 11:40:22.075815 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.069 3590 ERROR barbican.tasks.resources return self.handle_order(order)
Oct 23 11:40:22.076121 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.069 3590 ERROR barbican.tasks.resources File "/opt/stack/new/barbican/barbican/tasks/resources.py", line 291, in handle_order
Oct 23 11:40:22.076399 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.069 3590 ERROR barbican.tasks.resources project)
Oct 23 11:40:22.076675 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.069 3590 ERROR barbican.tasks.resources File "/opt/stack/new/barbican/barbican/plugin/resources.py", line 220, in generate_asymmetric_secret
Oct 23 11:40:22.077133 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.069 3590 ERROR barbican.tasks.resources content_type
Oct 23 11:40:22.077637 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.069 3590 ERROR barbican.tasks.resources File "/opt/stack/new/barbican/barbican/plugin/resources.py", line 310, in _generate_asymmetric_key
Oct 23 11:40:22.077932 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.069 3590 ERROR barbican.tasks.resources key_spec, context)
Oct 23 11:40:22.078214 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.069 3590 ERROR barbican.tasks.resources File "/opt/stack/new/barbican/barbican/plugin/store_crypto.py", line 207, in generate_asymmetric_key
Oct 23 11:40:22.078665 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.069 3590 ERROR barbican.tasks.resources generating_plugin, context.project_model)
Oct 23 11:40:22.078903 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.069 3590 ERROR barbican.tasks.resources File "/opt/stack/new/barbican/barbican/plugin/store_crypto.py", line 283, in _find_or_create_kek_objects
Oct 23 11:40:22.079275 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.069 3590 ERROR barbican.tasks.resources full_plugin_name)
Oct 23 11:40:22.079978 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.069 3590 ERROR barbican.tasks.resources File "/opt/stack/new/barbican/barbican/model/repositories.py", line 975, in find_or_create_kek_datum
Oct 23 11:40:22.080174 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.069 3590 ERROR barbican.tasks.resources kek_datum = query.one()
Oct 23 11:40:22.080363 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.069 3590 ERROR barbican.tasks.resources File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2817, in one
Oct 23 11:40:22.080549 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.069 3590 ERROR barbican.tasks.resources "Multiple rows were found for one()")
Oct 23 11:40:22.080732 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.069 3590 ERROR barbican.tasks.resources MultipleResultsFound: Multiple rows were found for one()
Oct 23 11:40:22.080917 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.069 3590 ERROR barbican.tasks.resources
Oct 23 11:40:22.081110 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.075 3589 DEBUG barbican.api.controllers.secrets [req-2bdb74a6-0c72-498c-a23f-56f7b64f3cc6 7cbcaa3318c342999fec01b5c671d87d - - default default] Start on_post for project-ID 7c66f71af58644bf96b8777e2c56a151:... on_post /opt/stack/new/barbican/barbican/api/controllers/secrets.py:418
Oct 23 11:40:22.081372 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.075 3590 ERROR barbican.tasks.resources [req-34b61d48-0a29-407a-b690-a8edd91e91b4 7cbcaa3318c342999fec01b5c671d87d - - default default] Suppressing exception while trying to process task 'Process TypeOrder'.: MultipleResultsFound: Multiple rows were found for one()
Oct 23 11:40:22.081585 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.075 3590 ERROR barbican.tasks.resources Traceback (most recent call last):
Oct 23 11:40:22.081900 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.075 3590 ERROR barbican.tasks.resources File "/opt/stack/new/barbican/barbican/tasks/resources.py", line 68, in process_and_suppress_exceptions
Oct 23 11:40:22.082262 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.075 3590 ERROR barbican.tasks.resources return self.process(*args, **kwargs)
Oct 23 11:40:22.082643 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.075 3590 ERROR barbican.tasks.resources File "/opt/stack/new/barbican/barbican/tasks/resources.py", line 113, in process
Oct 23 11:40:22.082907 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.075 3590 ERROR barbican.tasks.resources raise e_orig
Oct 23 11:40:22.083168 ubuntu-xenial-rax-ord-0000383099 <email address hidden>[3587]: 2017-10-23 11:40:22.075 3590 ERROR barbican.tasks.resources MultipleResultsFound: Multiple rows were found for one()

Dave McCowan (dave-mccowan)
Changed in barbican:
status: New → Triaged
importance: Undecided → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to barbican (master)

Reviewed: https://review.openstack.org/632931
Committed: https://git.openstack.org/cgit/openstack/barbican/commit/?id=6061e88fc195456e03eb46c964ca28862247a38f
Submitter: Zuul
Branch: master

commit 6061e88fc195456e03eb46c964ca28862247a38f
Author: Ade Lee <email address hidden>
Date: Thu Jan 24 00:31:14 2019 -0500

    Address race condition in KEKDatum

    Change-Id: I7b2f088c8cb9f74507313072d901bcedea18fc34
    Fixes-Bug: 1726378

Changed in barbican:
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.