OpenStack nova service responds with an erroneous httpd redirect to a "GET,version_controller,show" request.

Bug #1728732 reported by Colin Leavett-Brown on 2017-10-30
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Undecided
Eric Fried

Bug Description

Description:

When a client, eg. the OpenStack dashboard, makes a nova service request of "http://controller:8774/v2.1" (show controller versions), it receives a redirect to "http://controller:8774/v2.1/". This is erroneous for at least the following two reasons:

1. If, for security reasons, you place the nova service behind an SSL termination proxy, the redirect generated as follows:

      from : https://controller:proxy_port/v2.1

      to : http://controller:proxy_port/v2.1/

is invalid because the proxy_port requires encrypted traffic and the replacement URL is using the wrong protocol (http). The request fails on the client side with "Unable to establish connection to http://controller:proxy_port/v2.1/: ('Connection aborted.', BadStatusLine("''",))".

2. Even if we are not using a proxy server, the nova service is effectively complaining about a missing trailing forward slash ("/"), telling the client to reissue the same request but with the missing character. This creates unnecessary network traffic (the redirect plus a second request) and additional server load (two requests instead of one). It should be noted that "http://controller:8774/v2.1" is the endpoint specification recommended in the OpenStack nova installation guides for the ocata, and pike releases. This will result in unnecessary traffic and load on many installations, which will go unnoticed because the request eventually works.
Solution:

Replace the first ROUTE_LIST entry (and associated comments) in nova.api.openstack.compute.routes, changing it from:

    # NOTE: This is a redirection from '' to '/'. The request to the '/v2.1'
    # or '/2.0' without the ending '/' will get a response with status code
    # '302' returned.
    ('', '/'),

to:

    # The following 3 lines replaces a redirect specification that caused additional network traffic and load. See bug #xxxxx.
    ('', {
        'GET': [version_controller, 'show']

I've applied/tested a fix/workaround here: https://github.com/hep-gc/nova/commit/b9c27bf29f7042cf637b58c87d6a9b2f3a9b78b6

To recreate:
1. Install Openstack (ocata/pike) as per https://docs.openstack.org/pike/install/
2. Monitor network traffic (tcpdump) on client.
3. Login to the dashboard, and view compute->project->overview

To see "Unable to establish connection to http://controller:proxy_port/v2.1/: ('Connection aborted.', BadStatusLine("''",))" error:
4. Install HAProxy.
5. Serve the nova public endpoint via the SSL termination proxy server. Our HAProxy configuration for this is as follows:
  frontend nova_public
    bind controller_fqdn:18774 ssl crt /etc/letsencrypt/live/controller_fqdn/web_crt_key.pem
    reqadd X-Forwarded-Proto:\ https
    default_backend nova_internal

  backend nova_internal
    redirect scheme https code 301 if !{ ssl_fc }
    server controller controller:8774 check
6. Redefine the nova public endpoint in the sql database:
  mysql -ukeystone -p
  connect keystone;
  update endpoint set url="https://otter.heprc.uvic.ca:18774/v2.1" where id="xxxxxxx"
7. Dashboard will display "Unable to retrieve usage data" red flag each time the project overview page is displayed, and the http error log will report the connection failure.

Tags: api Edit Tag help
Changed in nova:
assignee: nobody → Colin Leavett-Brown (crlb-f)
status: New → In Progress
melanie witt (melwitt) on 2017-10-31
tags: added: api

Fix proposed to branch: master
Review: https://review.openstack.org/518119

Changed in nova:
assignee: Colin Leavett-Brown (crlb-f) → Eric Fried (efried)
Eric Fried (efried) wrote :

This bug is also fixed by https://review.opendev.org/#/c/685181/

Change abandoned by Stephen Finucane (<email address hidden>) on branch: master
Review: https://review.opendev.org/516796
Reason: Looks to be a dupe of https://review.opendev.org/#/c/518119

Change abandoned by Eric Fried (<email address hidden>) on branch: master
Review: https://review.opendev.org/518119
Reason: I believe this is covered by [1]. If not, please feel free to restore and update to pass tests.

[1] https://review.opendev.org/#/c/685181/

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers