Cannot delete a service or endpoint

[John Hardy]

I created an endpoint with an error by mistake as follows;

keystone endpoint-create --service_id 410de5fed7544c90a36ebdb8b38f0cea --publicurl "http://127.0.0.1:8774 /v1.1/\$(tenant_i d)s" --adminurl " http://127.0.0.1:8774 /v1.1/\$(tenant_i d)s" --internalurl "http://127.0.0.1:8774 /v1.1/\$(tenant_i d)s"

notice the space in each tenant_i d should be tenant_id.

Should easily be able to delete the endpoint using

keystone service-delete 410de5fed7544c90a36ebdb8b38f0cea

but I get

No handlers could be found for logger "keystoneclient.v2_0.client"
Authorization Failed: An unexpected error prevented the server from fulfilling your request. u'tenant_i d' (HTTP 500)

This is as ~(keystone_admin)

thanks

[Dolph Mathews (dolph)]

Interesting issue; let try to explain what's going on and how to work around it.

First, to delete and endpoint created with endpoint-create you should be using endpoint-delete, not service-delete:

  keystone endpoint-delete 410de5fed7544c90a36ebdb8b38f0cea

However, that's still going to fail in your case because you have an invalid endpoint in the catalog, so the client is failing to authenticate itself -- it's never getting as far as attempting to delete anything.

As a workaround, bypass authentication (the credentials specified by your environment variables) using token/endpoint values:

  keystone --os-token=<keystone.conf's admin_token> and --os-endpoint=http://localhost:35357/v2.0/ endpoint-delete 410de5fed7544c90a36ebdb8b38f0cea

Although this technically counts as user-error, I'd vastly prefer to see invalid endpoints suppressed from the service catalog output instead of raising an error back to the API and killing the authentication process. The current error message raised to the API should instead be logged to ERROR.

[Dolph Mathews (dolph)]

John: Can you also confirm that this issue was experienced with folsom? The error feedback in this scenario should be a bit different in grizzly, although the same workaround would be required to fix it.

[Theodore Ilie (theoilie-ti)]

Unless I did something wrong, this seems to work. It looks to me like the problem is that you copied the service id. Here is my successful log:

vagrant@Ubuntu64:/opt/stack/keystone$ keystone endpoint-create --service_id ba35dad1f0434ad79465082f4875ef47 --publicurl "http://127.0.0.1:8774 /v1.1/\$(tenant_i d)s" --adminurl " http://127.0.0.1:8774 /v1.1/\$(tenant_i d)s" --internalurl "http://127.0.0.1:8774 /v1.1/\$(tenant_i d)s"
+-------------+---------------------------------------------+
| Property | Value |
+-------------+---------------------------------------------+
| adminurl | http://127.0.0.1:8774 /v1.1/$(tenant_i d)s |
| id | 7dfe7eaa8ccf4cbba2227152bc7e3dbc |
| internalurl | http://127.0.0.1:8774 /v1.1/$(tenant_i d)s |
| publicurl | http://127.0.0.1:8774 /v1.1/$(tenant_i d)s |
| region | regionOne |
| service_id | ba35dad1f0434ad79465082f4875ef47 |
+-------------+---------------------------------------------+
vagrant@Ubuntu64:/opt/stack/keystone$ keystone endpoint-delete 7dfe7eaa8ccf4cbba2227152bc7e3dbc
Endpoint has been deleted.

Now I will put in placeholders to hopefully illustrate how you put in the wrong value:

vagrant@Ubuntu64:/opt/stack/keystone$ keystone endpoint-create --service_id <SERVICE ID> --publicurl "http://127.0.0.1:8774 /v1.1/\$(tenant_i d)s" --adminurl " http://127.0.0.1:8774 /v1.1/\$(tenant_i d)s" --internalurl "http://127.0.0.1:8774 /v1.1/\$(tenant_i d)s"
+-------------+---------------------------------------------+
| Property | Value |
+-------------+---------------------------------------------+
| adminurl | http://127.0.0.1:8774 /v1.1/$(tenant_i d)s |
| id | <ENDPOINT ID> |
| internalurl | http://127.0.0.1:8774 /v1.1/$(tenant_i d)s |
| publicurl | http://127.0.0.1:8774 /v1.1/$(tenant_i d)s |
| region | regionOne |
| service_id | <SERVICE_ID> |
+-------------+---------------------------------------------+
vagrant@Ubuntu64:/opt/stack/keystone$ keystone endpoint-delete <ENDPOINT ID>
Endpoint has been deleted.

So you put <SERVICE ID> in the final command (keystone endpoint-delete) rather than <ENDPOINT ID>. I took <SERVICE ID> from one of the ids listed in keystone service-list. When deleting the endpoint, you need to copy the <ENDPOINT ID> that is given to you, not the old <SERVICE_ID>. Let me know if I'm wrong about any of this. It looks like this isn't actually a bug to me.

[Dolph Mathews (dolph)]

Awesome to hear! I dug around in Keystone's code and found this, which I'm hoping is what's suppressing invalid URLs:

  https://github.com/openstack/keystone/blob/a06d5b/keystone/catalog/backends/sql.py#L314-L315

If the comment is correct, you should see logs about malformed endpoints while you're performing the above tests? If so, I think we can mark this as Invalid since it's apparently been fixed without updating this bug.

[Theodore Ilie (theoilie-ti)]

I tried this bug again with keystone endpoint-list after each command and got this output: https://paste.ee/p/pdyrS. It does not appear to be an actual bug, as the endpoint was deleted when using both tenant_id and tenant_i d.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/kilo
Review: https://review.openstack.org/196880

[OpenStack Infra (hudson-openstack)]

Change abandoned by Theodore Ilie (<email address hidden>) on branch: stable/kilo
Review: https://review.openstack.org/196880
Reason: Wrong branch

[David Stanek (dstanek)]

Also interesting thought...why do we allow invalid URLs at all?

[Theodore Ilie (theoilie-ti)]

I made a separate report about suppressing invalid URLs here: https://bugs.launchpad.net/keystone/+bug/1471034