Add novnc console support in favor of spice

[Bjoern]

This is a resubmission

Currently the spice console is plagued by too many open issues and we should use the novnc as default console back-end while having spice available as a future option.
The spice issues primarily manifest into mouse cursor and cursor issues inside GFX enabled consoles/displays like Windows display
or Linux consoles which have frame buffer (FB) enabled.

* No Windows 2008/2012 display driver support

  Windows is using a ordinary Basic Microsoft Video adapter driver
  No screen acceleration provided by spice is supported that way.
  That especially show into console issues once the guest resizes the screen, for example going into recovery mode.

* Windows 2008/2012 mouse cursor issues

  The windows cursor inside the spice console is virtually unusable.
  X/Y and acceleration coordinates between guest and spice server are deviating more and more, to a point
  which makes it impossible to use the mouse.

* CTRL+ALT+DEL button missing

  This has been fixed in OSAD per #1423669, the proposed fix never made it into the spice-html5 proxy upstream project
  This patch was originally submitted by RedHat.

* Display issues with newer Linux OS like centos or RHEL7 referred to 'GFX/FB (frame buffer) consoles'

 This has been addressed in a workaround published inside the RPC release notes/documentation

* QXL driver issues with the last tested version 0.11

  I did got to manage the installation of this driver but then it rotated the display 180deg which is related to a issue
  inside the spice-html5 proxy again. A found workaround fixed this issue. Resolution resets or display accelleration was not tested
  since it did not improve the most pressing issue in regards to the mouse coordinates

* No vdagent/vdservice support in nova

  I was not able to locate any support within Nova, but even when I enabled it manually it did not improve the situation.

Other references :
https://bugzilla.redhat.com/show_bug.cgi?id=895356 : WDDM Display Only Driver for windows 8/2012 guest

Wararound for CTRL+ALT+DEL button missing :
===========================================
This has been fixed by OSAD in a post install patch via https://bugs.launchpad.net/openstack-ansible/+bug/1423669

Woraround for GFX/FB (frame buffer) consoles issues :
=====================================================

spice console issue with CentOS 7/RHEL. Here the instructions:

Please add inside the /etc/default/grub config file the
option "nofb nomodereset" to the GRUB_CMDLINE_LINUX variable and regenerate the grub2 config

Tested config /etc/default/grub
-------------------------------
GRUB_TIMEOUT=1
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL="serial console"
GRUB_SERIAL_COMMAND="serial"
GRUB_CMDLINE_LINUX="console=ttyS0 console=tty0 crashkernel=auto vconsole.keymap=us nofb nomodereset"
GRUB_DISABLE_RECOVERY="true"

Rebuilt grub2 config
--------------------
grub2-mkconfig -o /boot/grub2/grub.cfg

Woraround for QXL driver issues with the last tested version 0.11 :
===================================================================

I got the qxl wddm driver 0.11 working in spice using the latest spice-html5 proxy (http://anongit.freedesktop.org/git/spice/spice-html5.git)
and a patch slightly changed I got from http://lists.freedesktop.org/archives/spice-devel/2013-January/012045.html :

diff --git a/display.js b/display.js
index 814ada6..ee197c8 100644
--- a/display.js
+++ b/display.js
@@ -284,9 +284,6 @@ SpiceDisplayConn.prototype.process_channel_message = function(msg)
                     return false;
                 }

- if (draw_copy.data.src_bitmap.lz_rgb.top_down != 1)
- this.log_warn("FIXME: Implement non top down support for lz_rgb");
-
                 var source_img = convert_spice_lz_to_web(canvas.context,
                                             draw_copy.data.src_bitmap.lz_rgb);
                 if (! source_img)
diff --git a/lz.js b/lz.js
index 4292eac..758f7d1 100644
--- a/lz.js
+++ b/lz.js
@@ -141,6 +141,27 @@ function lz_rgb32_decompress(in_buf, at, out_buf, type, default_alpha)
     return encoder - 1;
 }

+function flip_image_data(img)
+{
+ var w = img.width;
+ var h = img.height;
+ var temp_w = w;
+ var temp_h = h;
+ var buff = new Uint8Array(img.width * img.height * 4);
+ while (temp_h--)
+ {
+ while (temp_w--)
+ {
+ buff[(temp_h * w + temp_w) * 4] = img.data[((h - temp_h - 1) * w + temp_w) * 4];
+ buff[(temp_h * w + temp_w) * 4 + 1] = img.data[((h - temp_h -1) * w + temp_w) * 4 + 1];
+ buff[(temp_h * w + temp_w) * 4 + 2] = img.data[((h - temp_h -1) * w + temp_w) * 4 + 2];
+ buff[(temp_h * w + temp_w) * 4 + 3] = img.data[((h - temp_h -1) * w + temp_w) * 4 + 3];
+ }
+ temp_w = w;
+ }
+ img.data.set(buff);
+}
+
 function convert_spice_lz_to_web(context, lz_image)
 {
     var at;
@@ -150,6 +171,9 @@ function convert_spice_lz_to_web(context, lz_image)
         var ret = context.createImageData(lz_image.width, lz_image.height);

         at = lz_rgb32_decompress(u8, 0, ret.data, LZ_IMAGE_TYPE_RGB32, lz_image.type != LZ_IMAGE_TYPE_RGBA);
+ if (! lz_image.top_down)
+ flip_image_data(ret);
+
         if (lz_image.type == LZ_IMAGE_TYPE_RGBA)
             lz_rgb32_decompress(u8, at, ret.data, LZ_IMAGE_TYPE_RGBA, false);
     }

Lastly this workaround does not fix the mouse acceleration issues, it fixes only the screen rotation issues inside

[Jesse Klint (jesse-klint)]

RE: Display issues with Centos/RHEL 7. The issue does not present itself in a vanilla Ubuntu build of Icehouse. I was able to test in my home lab without the issue manifesting itself. After some digging, the only significant difference in packages/configuration was for websockify. Ubuntu backported several fixes to correct zombie processes (which the customer is also experiencing). So it seems that we should be able to tweak the requirements file to pull in websockify 0.6.0 which has the zombie patch. Or simply use the Ubuntu package.

We, however, were not able to test installing the patched version to see if it actually resolves the issue as the customer is upgrading to Juno this week. Perhaps we can test shortly before upgrading? They were fine with just waiting for the upgrade which should resolve the issue as it pulls in the patched version.

http://changelogs.ubuntu.com/changelogs/pool/universe/w/websockify/websockify_0.5.1+dfsg1-3ubuntu0.14.04.1/changelog
https://bugs.launchpad.net/ubuntu/+source/websockify/+bug/1350352

[Bjoern (bjoern-t)]

FYI. The spice html client is still a prototype client, as the project still refers to at http://www.spice-space.org/page/Html5:

 HTML5 Client

Spice has a prototype Web client which runs entirely within a modern browser. It is limited in function, a bit slow, and lacks support for many features of Spice (audio, video, agents just to name a few).

[Bjoern (bjoern-t)]

we have a workaround for the spice console issue with CentOS 7/RHEL. Here the instructions:

Please add inside the /etc/default/grub config file the
option "nofb nomodereset" to the GRUB_CMDLINE_LINUX variable and regenerate the grub2 config

Tested config /etc/default/grub
-------------------------------
GRUB_TIMEOUT=1
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL="serial console"
GRUB_SERIAL_COMMAND="serial"
GRUB_CMDLINE_LINUX="console=ttyS0 console=tty0 crashkernel=auto vconsole.keymap=us nofb nomodereset"
GRUB_DISABLE_RECOVERY="true"

Rebuilt grub2 config
--------------------
grub2-mkconfig -o /boot/grub2/grub.cfg

[Andy McCrae (andrew-mccrae)]

I would assume these are grub settings for the CentOS 7/RHEL servers - in which case we can't really fix this within os-ansible-deployment.

[Bjoern (bjoern-t)]

FYI, I got the qxl wddm driver 0.11 working in spice using the latest spice-html5 proxy (http://anongit.freedesktop.org/git/spice/spice-html5.git) and a patch slightly changed I got from http://lists.freedesktop.org/archives/spice-devel/2013-January/012045.html :

diff --git a/display.js b/display.js
index 814ada6..ee197c8 100644
--- a/display.js
+++ b/display.js
@@ -284,9 +284,6 @@ SpiceDisplayConn.prototype.process_channel_message = function(msg)
                     return false;
                 }

- if (draw_copy.data.src_bitmap.lz_rgb.top_down != 1)
- this.log_warn("FIXME: Implement non top down support for lz_rgb");
-
                 var source_img = convert_spice_lz_to_web(canvas.context,
                                             draw_copy.data.src_bitmap.lz_rgb);
                 if (! source_img)
diff --git a/lz.js b/lz.js
index 4292eac..758f7d1 100644
--- a/lz.js
+++ b/lz.js
@@ -141,6 +141,27 @@ function lz_rgb32_decompress(in_buf, at, out_buf, type, default_alpha)
     return encoder - 1;
 }

+function flip_image_data(img)
+{
+ var w = img.width;
+ var h = img.height;
+ var temp_w = w;
+ var temp_h = h;
+ var buff = new Uint8Array(img.width * img.height * 4);
+ while (temp_h--)
+ {
+ while (temp_w--)
+ {
+ buff[(temp_h * w + temp_w) * 4] = img.data[((h - temp_h - 1) * w + temp_w) * 4];
+ buff[(temp_h * w + temp_w) * 4 + 1] = img.data[((h - temp_h -1) * w + temp_w) * 4 + 1];
+ buff[(temp_h * w + temp_w) * 4 + 2] = img.data[((h - temp_h -1) * w + temp_w) * 4 + 2];
+ buff[(temp_h * w + temp_w) * 4 + 3] = img.data[((h - temp_h -1) * w + temp_w) * 4 + 3];
+ }
+ temp_w = w;
+ }
+ img.data.set(buff);
+}
+
 function convert_spice_lz_to_web(context, lz_image)
 {
     var at;
@@ -150,6 +171,9 @@ function convert_spice_lz_to_web(context, lz_image)
         var ret = context.createImageData(lz_image.width, lz_image.height);

         at = lz_rgb32_decompress(u8, 0, ret.data, LZ_IMAGE_TYPE_RGB32, lz_image.type != LZ_IMAGE_TYPE_RGBA);
+ if (! lz_image.top_down)
+ flip_image_data(ret);
+
         if (lz_image.type == LZ_IMAGE_TYPE_RGBA)
             lz_rgb32_decompress(u8, at, ret.data, LZ_IMAGE_TYPE_RGBA, false);
     }

[Kevin Carter (kevin-carter)]

This issue is not an OSAD thing to fix as such its been marked as "wont fix". If the community would like to have novnc support a spec will need to be created to add novnc support. At this time we will not be "replacing" spice.

[Bjoern (bjoern-t)]

I disagree this is an OSAD issue, implementing a broken, half developed console.
VNC should have been standard all along with spice as option. So please go ahead and spec we still need it.

[Kevin Carter (kevin-carter)]

The spice console works fine as a simple console. Being that the issues you've reported is not reproducible and or are specific to images that you've chosen to use this is not a bug but rather a feature request. If you would like to have novnc support a spec will need to be created to add novnc to master which may be backported to kilo. As I've stated, at this time we will not be "replacing" spice.

[Bjoern (bjoern-t)]

The spice console does not even work in GFX consoles with Linux. As I stated the issues are really easy to reproduce if you use a GFX linux console or windows guest.

[Kevin Carter (kevin-carter)]

You don't mention anything GFX + linux anywhere in this bug. Everything that you've spoken about so far has been issues with CentOS7 and Windows 2012. Windows drivers can be found here [ http://www.spice-space.org/download.html ] (if theres something wrong with the drivers raise an issue upstream), the CentOS7 issues are grub related and not something that we can or will fix for you, and the only thing that the horizon console should be used for is when SSH/RDP is otherwise not available (like in the case of a windows password setup/creation). All that aside, as I've repeatedly now, at this time we will not be "replacing" spice and If you would like to have novnc support a spec will need to be created to add novnc to master which may be backported to kilo.

[Bjoern (bjoern-t)]

To be precise the gfx console issue for Linux refers to the Linux terminal, once you boot the console with the frame buffer mode.
That is standard for RedHat based systems. The workaround I posted further up inside the ticket will get around this issue (grub settings). FYI, the spice drivers from http://www.spice-space.org/download.html are not Windows 8/2012 compatible and have not been updated since 2013 anymore. We will discuss this issue offline.

[Kevin Carter (kevin-carter)]

If you would like to discuss this further please post this to the mailing list and or add it to our meeting agenda found here [ "https://wiki.openstack.org/wiki/Meetings/openstack-ansible#Agenda_for_next_meeting" ]. If you have a solution in mind please create a spec if you do not but have a need for something to be done please lobby for a spec to be created from within the community via the mailing list or during one of our regularly scheduled community meetings as found here [ "https://wiki.openstack.org/wiki/Meetings/openstack-ansible#Community_Meeting" ].

[Bjoern (bjoern-t)]

Resubmission in progress

[Jesse Pretorius (jesse-pretorius)]

Removing milestone. If someone picks this up then it can be targeted.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/228197
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible/commit/?id=0681e25a5269e0b602207f10f72d0278769e85c8
Submitter: Jenkins
Branch: master

commit 0681e25a5269e0b602207f10f72d0278769e85c8
Author: Kevin Carter <email address hidden>
Date: Fri Sep 25 20:50:42 2015 -0500

    Add novnc console support

    This change adds in support for the novnc console type in Nova.

    * The change adds in a few new variables to the defaults which allow
      for the novnc console to be configued.
    * A port entry was added to haproxy to support the console type.
    * noVNC is being installed from source in the nova_console container.
      The git repo has been added to the openstack_other.yml repo-package file
      which allows for the repo to be cloned into the repo containers and then
      distributed out where needed from within the environment.

    Closes-Bug: 1428833

    Change-Id: I221557aad77bf266b4e2fae23007ffa210aa1f75
    Signed-off-by: Kevin Carter <email address hidden>

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/232657
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible/commit/?id=f3f56a7af3a7d93987184395edb99f68dd101a2b
Submitter: Jenkins
Branch: kilo

commit f3f56a7af3a7d93987184395edb99f68dd101a2b
Author: Kevin Carter <email address hidden>
Date: Fri Sep 25 20:50:42 2015 -0500

    Add novnc console support

    This change adds in support for the novnc console type in Nova.

    * The change adds in a few new variables to the defaults which allow
      for the novnc console to be configued.
    * A port entry was added to haproxy to support the console type.
    * noVNC is being installed from source in the nova_console container.
      The git repo has been added to the openstack_other.yml repo-package file
      which allows for the repo to be cloned into the repo containers and then
      distributed out where needed from within the environment.

    This cherry-pick was modified to remove the "vnc" section because that
    configuration option within nova was not backported to kilo in upstream
    nova.

    Closes-Bug: 1428833

    Change-Id: I221557aad77bf266b4e2fae23007ffa210aa1f75
    Signed-off-by: Kevin Carter <email address hidden>
    (cherry picked from commit 0681e25a5269e0b602207f10f72d0278769e85c8)
    Signed-off-by: Kevin Carter <email address hidden>

[Davanum Srinivas (DIMS) (dims-v)]

This issue was fixed in the openstack/openstack-ansible 11.2.11 release.

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/openstack-ansible 11.2.12 release.

[Davanum Srinivas (DIMS) (dims-v)]

This issue was fixed in the openstack/openstack-ansible 11.2.14 release.

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/openstack-ansible 11.2.15 release.