OpenStack nova service responds with an erroneous httpd redirect to a "GET,version_controller,show" request.

[Colin Leavett-Brown]

Description:

When a client, eg. the OpenStack dashboard, makes a nova service request of "http://controller:8774/v2.1" (show controller versions), it receives a redirect to "http://controller:8774/v2.1/". This is erroneous for at least the following two reasons:

1. If, for security reasons, you place the nova service behind an SSL termination proxy, the redirect generated as follows:

      from : https://controller:proxy_port/v2.1

      to : http://controller:proxy_port/v2.1/

is invalid because the proxy_port requires encrypted traffic and the replacement URL is using the wrong protocol (http). The request fails on the client side with "Unable to establish connection to http://controller:proxy_port/v2.1/: ('Connection aborted.', BadStatusLine("''",))".

2. Even if we are not using a proxy server, the nova service is effectively complaining about a missing trailing forward slash ("/"), telling the client to reissue the same request but with the missing character. This creates unnecessary network traffic (the redirect plus a second request) and additional server load (two requests instead of one). It should be noted that "http://controller:8774/v2.1" is the endpoint specification recommended in the OpenStack nova installation guides for the ocata, and pike releases. This will result in unnecessary traffic and load on many installations, which will go unnoticed because the request eventually works.
Solution:

Replace the first ROUTE_LIST entry (and associated comments) in nova.api.openstack.compute.routes, changing it from:

    # NOTE: This is a redirection from '' to '/'. The request to the '/v2.1'
    # or '/2.0' without the ending '/' will get a response with status code
    # '302' returned.
    ('', '/'),

to:

    # The following 3 lines replaces a redirect specification that caused additional network traffic and load. See bug #xxxxx.
    ('', {
        'GET': [version_controller, 'show']

I've applied/tested a fix/workaround here: https://github.com/hep-gc/nova/commit/b9c27bf29f7042cf637b58c87d6a9b2f3a9b78b6

To recreate:
1. Install Openstack (ocata/pike) as per https://docs.openstack.org/pike/install/
2. Monitor network traffic (tcpdump) on client.
3. Login to the dashboard, and view compute->project->overview

To see "Unable to establish connection to http://controller:proxy_port/v2.1/: ('Connection aborted.', BadStatusLine("''",))" error:
4. Install HAProxy.
5. Serve the nova public endpoint via the SSL termination proxy server. Our HAProxy configuration for this is as follows:
  frontend nova_public
    bind controller_fqdn:18774 ssl crt /etc/letsencrypt/live/controller_fqdn/web_crt_key.pem
    reqadd X-Forwarded-Proto:\ https
    default_backend nova_internal

  backend nova_internal
    redirect scheme https code 301 if !{ ssl_fc }
    server controller controller:8774 check
6. Redefine the nova public endpoint in the sql database:
  mysql -ukeystone -p
  connect keystone;
  update endpoint set url="https://otter.heprc.uvic.ca:18774/v2.1" where id="xxxxxxx"
7. Dashboard will display "Unable to retrieve usage data" red flag each time the project overview page is displayed, and the http error log will report the connection failure.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/516796

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/518119

[Eric Fried (efried)]

This bug is also fixed by https://review.opendev.org/#/c/685181/

[OpenStack Infra (hudson-openstack)]

Change abandoned by Stephen Finucane (<email address hidden>) on branch: master
Review: https://review.opendev.org/516796
Reason: Looks to be a dupe of https://review.opendev.org/#/c/518119

[OpenStack Infra (hudson-openstack)]

Change abandoned by Eric Fried (<email address hidden>) on branch: master
Review: https://review.opendev.org/518119
Reason: I believe this is covered by [1]. If not, please feel free to restore and update to pass tests.

[1] https://review.opendev.org/#/c/685181/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/685181
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=1e907602e37fb55bbe5a20164db6d074f87369af
Submitter: Zuul
Branch: master

commit 1e907602e37fb55bbe5a20164db6d074f87369af
Author: Eric Fried <email address hidden>
Date: Thu Sep 26 16:52:12 2019 -0500

    Allow versioned discovery unauthenticated

    Make routes to the versioned discovery documents (/v2, /v2.1) go through
    paste pipelines that don't require authentication, while leaving their
    sub-URLs (/v2.1/servers etc) requiring authentication.

    To make this work, our URLMap matcher gets support for a very
    rudimentary wildcard syntax, whereby api-paste.ini can differentiate
    between {/v2.1, /v2.1/} and /v2.1/$anything_else. The former points to
    the unauthenticated discovery app pipeline; the latter points to the
    existing "real API" pipeline. Similar for legacy v2.

    This entails a slight behavior change: requests to /v2 and /v2.1 used to
    302 redirect to /v2/ and /v2.1/, respectively. Now they just work.

    Change-Id: Id47515017982850b167d5c637d93b96ae00ba793
    Closes-Bug: #1845530
    Closes-Bug: #1728732