serial console in Horizon is broken by origin header verification

Bug #1434611 reported by Markus Zoeller (markus_z)
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
High
Andrew Laski
OpenStack Dashboard (Horizon)
Invalid
Undecided
Aaron Sahlin

Bug Description

Issue
=====

The console tab in Horizon doesn't show the console of the instance
anymore if the "serial console" feature is enabled.

Steps to reproduce
==================

* Enable "serial console" feature in "nova.conf"
  [Default]
  vnc_enabled=False
  [serial_console]
  enabled=True
  base_url=ws://<host-ip>:6083
* Launch an instance
* Open to the "console" tab of that instance

Expected behavior
=================

The login prompt from the instance is shown via serial console
connection.

Actual behavior
===============

* Black screen + "Status: Closed"
* The traceback shows a validation error [1].
* If "base_url" is NOT set in "nova.conf" the console will still not
   connect, but nothing will be logged.

Possible root causes
--------------------

This validation logic was introduced with [2]. The unit seems to be
based on the assumption that the serial console uses "https" [3]. AFAIK
the supposed protocol is "ws".

Logs & Env.
===========

Manually added debug log:
/opt/stack/screen-logs/screen-n-sproxy.log:2015-03-20 14:49:14.081
DEBUG nova.console.websocketproxy
[req-fae1b332-5b8c-4253-a1ec-5306c56c1f3a None None]
origin: http, expected: ws
verify_origin_proto /opt/stack/nova/nova/console/websocketproxy.py:65

Nova version
------------

/opt/stack/nova$ git log --oneline -n5
90ee915 Merge "Add api microvesion unit test case for wsgi.action"
7885b74 Merge "Remove db layer hard-code permission checks for [...]
416f310 Merge "Remove db layer hard-code permission checks for [...]
ecb306b Merge "Remove db layer hard-code permission checks for [...]
6efc8ad Merge "libvirt: don't allow to resize down the default [...]

References
==========

[1] Traceback; nova.console.websocketproxy
    http://paste.openstack.org/show/193938/
[2] Gerrit; Websocket Proxy should verify Origin header
    https://review.openstack.org/#/c/163033/6
[3] test_websocketproxy.py module; "https" instead of "ws"
    https://github.com/openstack/nova/blob/master/nova/tests/unit/console/test_websocketproxy.py#L44

Tags: console
Revision history for this message
Markus Zoeller (markus_z) (mzoeller) wrote :

I added Horizon as affected project because in review [1] was mentioned that a possible client of the exposed API has to set the header correctly.

[1] https://review.openstack.org/#/c/163033/6

Revision history for this message
Markus Zoeller (markus_z) (mzoeller) wrote :

A workaround can be:

    --- a/nova/console/websocketproxy.py
    +++ b/nova/console/websocketproxy.py
    @@ -118,10 +120,11 @@ class NovaProxyRequestHandlerBase(object):
                 if expected_origin_hostname != origin_hostname:
                     detail = _("Origin header does not match this host.")
                     raise exception.ValidationError(detail=detail)
    - if not self.verify_origin_proto(connect_info['console_type'],
    - origin.scheme):
    - detail = _("Origin header protocol does not match this host.")
    - raise exception.ValidationError(detail=detail)
    + # mzoeller: bug https://bugs.launchpad.net/horizon/+bug/1434611
    + #if not self.verify_origin_proto(connect_info['console_type'],
    + # origin.scheme):
    + # detail = _("Origin header protocol does not match this host.")
    + # raise exception.ValidationError(detail=detail)

             self.msg(_('connect info: %s'), str(connect_info))
             host = connect_info['host']

Sean Dague (sdague)
Changed in nova:
status: New → Incomplete
Revision history for this message
Markus Zoeller (markus_z) (mzoeller) wrote :

@Sean Dague: Which additional data is needed to verify the behaviour?

Revision history for this message
Aaron Sahlin (asahlin) wrote :

The check made in https://review.openstack.org/#/c/163033/, breaks browser based serial consoles. I see in the patch where it has a special allowance to let non browser based serial console work.

Horizon added a browser serial console in Kilo https://review.openstack.org/#/c/144659, which is now broken. We need someone from Nova to look at that check and see what can be done.

Changed in horizon:
assignee: nobody → Aaron Sahlin (asahlin)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/169384

Changed in nova:
assignee: nobody → Andrew Laski (alaski)
status: Incomplete → In Progress
Revision history for this message
Tristan Cacqueray (tristan-cacqueray) wrote :

Not sure why the serial console base_url parameter does not use the "http://<ip>:608<port>/vnc_auto.html" format used by vnc and spice...

https://review.openstack.org/169384 fixed this use-case for me.

Andrew Laski (alaski)
Changed in nova:
milestone: none → kilo-rc1
importance: Undecided → High
Changed in horizon:
status: New → Invalid
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/169384
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=11d5ded6d557b0f087ab676a33f07349f5fd47cb
Submitter: Jenkins
Branch: master

commit 11d5ded6d557b0f087ab676a33f07349f5fd47cb
Author: Andrew Laski <email address hidden>
Date: Tue Mar 31 11:40:12 2015 -0400

    Expand Origin header check for serial console

    Serial consoles can use the ws or wss protocols which correspond to http
    and https respectively from a security perspective. The Origin header
    check failed has been expanded to allow for those protocols and to match
    appropriately with http/https when verifying the protocol security
    level.

    Change-Id: I7155607488495ba70e6edd550b12abc21b3e69c4
    Closes-Bug: 1434611

Changed in nova:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in nova:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: kilo-rc1 → 2015.1.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.