"Duplicate RecordSet" error while trying to update Nameservers Pool

[Vladimir Grevtsev]

Summary: Unable to update a Nameservers in Nameservers Pool

Version: stable/kilo

Steps to reproduce:

1) Install a stable/kilo Designate from scratch
2) Insert a new nameserver via api call
3) Try to update a Nameserver Pool - at this moment API will raise an "Duplicate RecordSet" error.

3.1) Make sure that ns1.xxx is not busy
http://designate-dev.zzz..xxx.ru:9001/v2/zones/cf5c1b26-a945-449d-890e-0a1db5a2f339/recordsets?name=ns1*

{
recordsets: [ ],
links: {
self: "http://127.0.0.1:9001/v2/zones/cf5c1b26-a945-449d-890e-0a1db5a2f339/recordsets?name=ns1%2A"
},
metadata: {
total_count: 0
}
}

3.2) Trying to update NS Pool

http -v --json PATCH http://designate-dev.zzz..xxx.ru:9001/v2/pools/794ccc2c-d751-44fe-b57f-8894c9f5c842 ns_records:='[{"hostname": "ns1.yyy.xxx.ru.", "priority": 10}]'
PATCH /v2/pools/794ccc2c-d751-44fe-b57f-8894c9f5c842 HTTP/1.1
Accept: application/json
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 112
Content-Type: application/json
Host: designate-dev.zzz..xxx.ru:9001
User-Agent: HTTPie/0.9.2

{
    "ns_records": [
        {
            "hostname": "ns1.yyy.xxx.ru.",
            "priority": 10
        }
    ]
}

HTTP/1.1 409 CONFLICT
Connection: keep-alive
Content-Length: 136
Content-Type: application/json
Date: Tue, 29 Sep 2015 08:46:38 GMT
X-Openstack-Request-Id: req-3b62845e-42a7-4edc-a258-2fcc00d94fab

{
    "code": 409,
    "message": "Duplicate RecordSet",
    "request_id": "req-3b62845e-42a7-4edc-a258-2fcc00d94fab",
    "type": "duplicate_recordset"
}

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/228832

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/kilo
Review: https://review.openstack.org/228833

[Vladimir Grevtsev (vlgrevtsev)]

After short investigation we've clarified a following: bug is reproducible when another NS recordset is exists in the zone (if no any NSs - no problems with NS Pool update)

and, as seen from IRC:
<Kiall> ekarlso has tracked it down to a bug in Designate caused by the second NS RRSet in the zone
<Kiall> That's clearly not a case we've tested before

DB dump (where bug is reproducible): http://paste.openstack.org/show/jv40EYNiwblMY6W1tkrd/
Query with error: http://paste.openstack.org/show/urwbpEDZg3ETvDT4GnFe/

[Endre Karlson (endre-karlson)]

    def _add_ns(self, context, zone, ns_record):
        # Get NS recordset
        # If the zone doesn't have an NS recordset yet, create one
        try:
            ns_recordset = self.find_recordset(
                context, criterion={'domain_id': zone['id'], 'type': "NS"})

        except exceptions.RecordSetNotFound:
            self._create_ns(context, zone, [ns_record])
            return

        # Add new record to recordset based on the new nameserver
        ns_recordset.records.append(
            objects.Record(data=ns_record, managed=True))

        self._update_recordset_in_storage(context, zone, ns_recordset)

So what happens is that we're looking up if a NS record exists ^ will blow up

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/229096

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/229096
Committed: https://git.openstack.org/cgit/openstack/designate/commit/?id=c59a2a012e61a80ea4d985db04ac29a582c745d1
Submitter: Jenkins
Branch: master

commit c59a2a012e61a80ea4d985db04ac29a582c745d1
Author: Endre Karlson <email address hidden>
Date: Tue Sep 29 19:50:32 2015 +0200

    Fix handling of Pool NS changes

    Closes-Bug: #1500798

    Change-Id: Ie697f35a523a63617ba386f0dcde63600ce5dac6

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/231008

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/231008
Committed: https://git.openstack.org/cgit/openstack/designate/commit/?id=bfe79196c9557559f351ae9a854c73ac757051ee
Submitter: Jenkins
Branch: stable/liberty

commit bfe79196c9557559f351ae9a854c73ac757051ee
Author: Endre Karlson <email address hidden>
Date: Tue Sep 29 19:50:32 2015 +0200

    Fix handling of Pool NS changes

    Closes-Bug: #1500798

    Change-Id: Ie697f35a523a63617ba386f0dcde63600ce5dac6
    (cherry picked from commit c59a2a012e61a80ea4d985db04ac29a582c745d1)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/kilo
Review: https://review.openstack.org/231370

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/235162

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/235162
Committed: https://git.openstack.org/cgit/openstack/designate/commit/?id=b5b54334208e38954829ea7618de3697e73dab15
Submitter: Jenkins
Branch: master

commit bfe79196c9557559f351ae9a854c73ac757051ee
Author: Endre Karlson <email address hidden>
Date: Tue Sep 29 19:50:32 2015 +0200

    Fix handling of Pool NS changes

    Closes-Bug: #1500798

    Change-Id: Ie697f35a523a63617ba386f0dcde63600ce5dac6
    (cherry picked from commit c59a2a012e61a80ea4d985db04ac29a582c745d1)

commit 63a45b6a9010de9dacb9656a4fe20c1bdef1768d
Author: Andreas Jaeger <email address hidden>
Date: Thu Sep 24 21:09:49 2015 +0200

    Cleanup of Translations

    In preparation for the release, do some cleanups for translations.

    Removes all po files that are partially translated. The translation
    team has decided to exclude files with less than 66 % of translated
    content. There is no content lost, all data is in the translation
    server, we just remove it from the repository.

    This updates also recreates pot (translation source files) to
    reflect the state of the repository in case there was no recent
    import for them.

    This change needs to be done manually since the automatic import does
    not handle some of these cases.

    Change-Id: I9616eb5e756f694b55cadb57b7b2aacf46833bb9

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/228833
Committed: https://git.openstack.org/cgit/openstack/designate/commit/?id=4de68f4608c4ebf723e75bc175ce86193c955649
Submitter: Jenkins
Branch: stable/kilo

commit 4de68f4608c4ebf723e75bc175ce86193c955649
Author: Kiall Mac Innes <email address hidden>
Date: Tue Sep 29 11:29:54 2015 +0100

    Pool NS update test should validate zones are updated

    Change-Id: Ie2f5ebebc8cf01ba08c222a9fc2d9d1608bef239
    Related-Bug: 1500798

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/231370
Committed: https://git.openstack.org/cgit/openstack/designate/commit/?id=c034c930863dcb62923709af3f21ba9fd8ca755e
Submitter: Jenkins
Branch: stable/kilo

commit c034c930863dcb62923709af3f21ba9fd8ca755e
Author: Endre Karlson <email address hidden>
Date: Tue Sep 29 19:50:32 2015 +0200

    Fix handling of Pool NS changes

    Closes-Bug: #1500798

    Change-Id: Ie697f35a523a63617ba386f0dcde63600ce5dac6

[Thierry Carrez (ttx)]

This issue was fixed in the openstack/designate 2.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Graham Hayes (<email address hidden>) on branch: master
Review: https://review.openstack.org/228832
Reason: As