Designate

404 NotFound returned from DELETE record, but record was exist.

Bug #1507539 reported by Keisuke Yamamoto on 2015-10-19

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	Designate	Won't Fix	Undecided	Keisuke Yamamoto

Bug Description

Repeated create and delete a record in multi-thread,
there are case of 404 "Could not find RecordSet" is returned from the delete record.
Recordset was deleted in same domain by "poolmanager - update_status()" of other requests.

* Steps to reproduce:
  Repeat the following APIs.
    1. Create Record
      - POST /v1/domains/(uuid: domain_id)/records
        - Use a different "name" each time.
    2. List Records in a Domain
      - GET /v1/domains/(uuid: domain_id)/records
    3. Delete a record
      - DELETE /v1/domains/(uuid: domain_id)/records/(uuid: id)

    - concurrency: 4
    - total delete requests: 40
    - 404 occurred: 1~2

* Actual results:
There is a case where 404 NotFound returned in the "3. Delete a record".

* Expected results:
Return 200 in all DELETE APIs.

* Additional note:
  Log has been output in the following order.
   1. designate-api output "delete_recordset: Calling central's delete_recordset.".
   2. designate-central output "Expected exception during message handling (Could not find RecordSet)"
   3. designate-api output "DELETE /v1/domains/9aad7ffd-eec1-498b-84e9-a97b0301dc99/records/6bf55bca-a788-4ddd-ad5a-a602272875d2 HTTP/1.1" 404"

+ designate-api.log
...
2015-10-05 12:06:58.548 7836 INFO designate.central.rpcapi [req-2baa78a3-15b8-4030-b507-01ce63878c29 211109a4f2284e1e920b1cc83db55c1a 7d41a1c6e70344b6bbd4fd6e9e055204 - - -] find_domain: Calling central's find_domain.
2015-10-05 12:06:58.874 7836 INFO designate.central.rpcapi [req-2baa78a3-15b8-4030-b507-01ce63878c29 211109a4f2284e1e920b1cc83db55c1a 7d41a1c6e70344b6bbd4fd6e9e055204 - - -] find_record: Calling central's find_record.
2015-10-05 12:06:59.310 7836 INFO designate.central.rpcapi [req-2baa78a3-15b8-4030-b507-01ce63878c29 211109a4f2284e1e920b1cc83db55c1a 7d41a1c6e70344b6bbd4fd6e9e055204 - - -] delete_record: Calling central's delete_record.
2015-10-05 12:07:00.069 7836 INFO designate.central.rpcapi [req-2baa78a3-15b8-4030-b507-01ce63878c29 211109a4f2284e1e920b1cc83db55c1a 7d41a1c6e70344b6bbd4fd6e9e055204 - - -] find_recordset: Calling central's find_recordset.
2015-10-05 12:07:00.703 7836 INFO designate.central.rpcapi [req-2baa78a3-15b8-4030-b507-01ce63878c29 211109a4f2284e1e920b1cc83db55c1a 7d41a1c6e70344b6bbd4fd6e9e055204 - - -] delete_recordset: Calling central's delete_recordset.
2015-10-05 12:07:00.911 7836 INFO eventlet.wsgi [req-2baa78a3-15b8-4030-b507-01ce63878c29 211109a4f2284e1e920b1cc83db55c1a 7d41a1c6e70344b6bbd4fd6e9e055204 - - -] 100.99.0.200 - - [05/Oct/2015 12:07:00] "DELETE /v1/domains/9aad7ffd-eec1-498b-84e9-a97b0301dc99/records/6bf55bca-a788-4ddd-ad5a-a602272875d2 HTTP/1.1" 404 347 2.370351

+ designate-central.log
...
2015-10-05 12:07:00.781 24935 DEBUG oslo_concurrency.lockutils [req-2baa78a3-15b8-4030-b507-01ce63878c29 211109a4f2284e1e920b1cc83db55c1a 7d41a1c6e70344b6bbd4fd6e9e055204 - - -] Acquired semaphore "domain-9aad7ffd-eec1-498b-84e9-a97b0301dc99" lock /opt/venv/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:391
2015-10-05 12:07:00.883 24935 DEBUG oslo_concurrency.lockutils [req-2baa78a3-15b8-4030-b507-01ce63878c29 211109a4f2284e1e920b1cc83db55c1a 7d41a1c6e70344b6bbd4fd6e9e055204 - - -] Releasing semaphore "domain-9aad7ffd-eec1-498b-84e9-a97b0301dc99" lock /opt/venv/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:404
2015-10-05 12:07:00.887 24935 DEBUG designate.central.service [req-2baa78a3-15b8-4030-b507-01ce63878c29 211109a4f2284e1e920b1cc83db55c1a 7d41a1c6e70344b6bbd4fd6e9e055204 - - -] Emitting 0 notifications wrapper /opt/venv/lib/python2.7/site-packages/designate/central/service.py:235
2015-10-05 12:07:00.891 24935 DEBUG oslo_messaging.rpc.dispatcher [req-2baa78a3-15b8-4030-b507-01ce63878c29 211109a4f2284e1e920b1cc83db55c1a 7d41a1c6e70344b6bbd4fd6e9e055204 - - -] Expected exception during message handling (Could not find RecordSet) _dispatch_and_reply /opt/venv/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py:145
...
2015-10-05 12:07:01.530 24935 DEBUG designate.central.service [req-2baa78a3-15b8-4030-b507-01ce63878c29 211109a4f2284e1e920b1cc83db55c1a 7d41a1c6e70344b6bbd4fd6e9e055204 - - -] Setting record bfee2b03-cc56-427e-af65-9eac150b7598, serial 1444046820: action NONE, status DELETED _update_record_status /opt/venv/lib/python2.7/site-packages/designate/central/service.py:2221
2015-10-05 12:07:01.544 24935 DEBUG designate.central.service [req-2baa78a3-15b8-4030-b507-01ce63878c29 211109a4f2284e1e920b1cc83db55c1a 7d41a1c6e70344b6bbd4fd6e9e055204 - - -] Deleting record bfee2b03-cc56-427e-af65-9eac150b7598, serial 1444046820: action NONE, status DELETED _update_record_status /opt/venv/lib/python2.7/site-packages/designate/central/service.py:2230
2015-10-05 12:07:01.603 24935 DEBUG designate.central.service [req-2baa78a3-15b8-4030-b507-01ce63878c29 211109a4f2284e1e920b1cc83db55c1a 7d41a1c6e70344b6bbd4fd6e9e055204 - - -] Setting domain 9aad7ffd-eec1-498b-84e9-a97b0301dc99, serial 1444046821: action UPDATE, status PENDING _update_domain_status /opt/venv/lib/python2.7/site-packages/designate/central/service.py:2178

Keisuke Yamamoto (keisuk-yamamoto) on 2015-10-19

Changed in designate:
assignee:	nobody → Keisuke Yamamoto (keisuk-yamamoto)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-10-19: Fix proposed to designate (master)

Fix proposed to branch: master
Review: https://review.openstack.org/236934

Changed in designate:
status:	New → In Progress

Graham Hayes (grahamhayes) on 2016-03-30

Changed in designate:
status:	In Progress → Won't Fix

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-05-10: Change abandoned on designate (master)

Change abandoned by Graham Hayes (<email address hidden>) on branch: master
Review: https://review.openstack.org/236934
Reason: We cannot make such a change to our v1 stable API

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.