session endpoint lookup doesn't handle endpoints with paths

[Matt Riedemann]

We have a service catalog that looks like this:

+-----------------------+-----------+--------------------------------------------------------+
| Name | Type | Endpoints |
+-----------------------+-----------+--------------------------------------------------------+
| Compute Service | compute | RegionOne |
| | | publicURL: http://localhost:5000/compute/v2/720429 |
| | | internalURL: http://localhost:5000/compute/v2/720429 |
| | | adminURL: http://localhost:5000/compute/v2/720429 |
| | | |
| Network Service | network | RegionOne |
| | | publicURL: http://localhost:5000/network |
| | | internalURL: http://localhost:5000/network |
| | | adminURL: http://localhost:5000/network |
| | | |
| Image Service | image | RegionOne |
| | | publicURL: http://localhost:5000/image |
| | | internalURL: http://localhost:5000/image |
| | | adminURL: http://localhost:5000/image |
| | | |
| Block Storage Service | volume | RegionOne |
| | | publicURL: http://localhost:5000/volume/v1/720429 |
| | | internalURL: http://localhost:5000/volume/v1/720429 |
| | | adminURL: http://localhost:5000/volume/v1/720429 |
| | | |
| Identity Service | identity | RegionOne |
| | | publicURL: http://localhost:5000/v2.0 |
| | | internalURL: http://localhost:5000/v2.0 |
| | | adminURL: http://localhost:5000/v2.0 |
| | | |
+-----------------------+-----------+--------------------------------------------------------+

When using openstackclient + openstacksdk>9.1.0, openstack network list fails because the SDK is not properly parsing the network endpoint.

That's because of this change:

https://review.openstack.org/#/c/351398/3/openstack/session.py@118

The code assumes you have an endpoint of the form: scheme://<host>:<port>

But we have: scheme://<host>:<port>/<type>

So when doing a network list operation from openstackclient, the sdk isn't able to figure out the version of the network API to use (v2.0), and makes the request against http://localhost:5000/ which is actually the identity service root.

[Matt Riedemann (mriedem)]

Note that before https://review.openstack.org/#/c/351398/ things work because the SDK is using the service catalog. I can see why it was changed, and getting the versions from the actual services is better, but the SDK is making assumptions on how the endpoints are formed in the service catalog which is the bug.

[Matt Riedemann (mriedem)]

To reiterate, this is our network endpoint public URL: http://localhost:5000/network

And when I make a request to that I get this:

(jumpgate)user@xubuntu:~/git/python-openstacksdk$ curl -g -H "X-Auth-Token: $OS_TOKEN" http://localhost:5000/network | json_pp
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
100 125 100 125 0 0 15636 0 --:--:-- --:--:-- --:--:-- 17857
{
   "versions" : [
      {
         "links" : [
            {
               "href" : "http://localhost:5000/network/v2.0",
               "rel" : "self"
            }
         ],
         "id" : "v2.0",
         "status" : "CURRENT"
      }
   ]
}

And when I make a request to http://localhost:5000/network/v2.0 I get this:

(jumpgate)user@xubuntu:~/git/python-openstacksdk$ curl -g -H "X-Auth-Token: $OS_TOKEN" http://localhost:5000/network/v2.0 | json_pp
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
100 272 100 272 0 0 32893 0 --:--:-- --:--:-- --:--:-- 38857
{
   "resources" : [
      {
         "links" : [
            {
               "rel" : "self",
               "href" : "http://localhost:5000/network/v2.0/subnets"
            }
         ],
         "name" : "subnet",
         "collection" : "subnets"
      },
      {
         "collection" : "networks",
         "name" : "network",
         "links" : [
            {
               "rel" : "self",
               "href" : "http://localhost:5000/network/v2.0/networks"
            }
         ]
      }
   ]
}

[Matt Riedemann (mriedem)]

I've patched in https://review.openstack.org/#/c/369499/ but things still fail. This is the debug output:

http://paste.openstack.org/show/584903/

[Brian Curtin (brian.curtin)]

I guess we have to take the reverse of the path accumulator inside Session._get_endpoint_versions. We currently try paths emanating outward from the root, but since it's possible to create a root that is also another service that reports version info, I guess we need to go the other way. That'll cause more roundtrips on startup to determine versions, but it's perhaps more likely to work.

[Brian Curtin (brian.curtin)]

Ugh, that won't really work either. In your case, no matter what direction we go we'll never end up with the right info. In some cases it'll actually work out alright, but it's not all correct.

Given something like http://cloud.com:1234/service/vX

In the proposed change, we start at the root and work outward. That's how we end up getting your Keystone versions.

If we try the reverse and go outward in, we'd end up just getting the version info for what's configured in the service catalog, not for all available versions, which is what we need this for.

[Brian Curtin (brian.curtin)]

The more I look at your service catalog, the less I feel like we can provide a solution that you can use. However, I'm looking at an actual real solution that turns out to be a joke from a Silicon Valley episode.

http://localhost:5000/v2/baremetal - Is this the root of the baremetal service or is that v2? Meaning, if I go to that URL, will it give me JSON showing all available baremetal versions (which is what I need)? What's going to happen if I hit http://localhost:5000/v2/?

http://localhost:5000/compute/v2/720429 -- This we could only figure out by going from the middle outward. From the left we'd get a Keystone version, from the right we'd get v2-only.

http://localhost:5000/network and http://localhost:5000/image -- These we can only figure out by going from the right, assuming that those /network and /image urls are returning version info for all available versions. At this point we probably can't assume anything reasonable ever again.

http://localhost:5000/volume/v1/720429 -- Same as compute.

Where does this cloud exist? Are there any rules or documentation for how these services can actually be installed and structured? Like everything else, this has mostly been a guess and trying it in real life, which only accounts for like 25% of what people can think up.

[Brant Knudson (blk-u)]

Don't be thrown off by the baremetal entry in the service catalog. It was a typo on our part.

[Brant Knudson (blk-u)]

I edited the bug description to remove the baremetal entry.