vnc console fail when changing NovaLibvirtNetwork to e.g. ctlplane

[Martin Schuppert]

Deployment with the following THT param make the vnc console to fail:

parameter_defaults:
    ServiceNetMap:
        NovaLibvirtNetwork: ctlplane

Steps to Reproduce:
1. Deploy with THT parameter:
    ServiceNetMap:
        NovaLibvirtNetwork: ctlplane
2. Run tempest test tempest.api.compute.servers.test_novnc.NoVNCConsoleTestJSON.test_novnc
3. Test fails

Actual results:
       Body: {"os-getVNCConsole": {"type": "novnc"}}
    Response - Headers: {'status': '200', u'content-length': '144', 'content-location': 'https://overcloud.redhat.local:13774/v2.1/servers/21a9b536-1162-43d5-8e38-4f4d68bbf788/action', u'x-compute-request-id': 'req-80c6f4bd-9aab-4c50-8db8-d46da77d930e', u'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding', u'server': 'Apache', u'openstack-api-version': 'compute 2.1', u'connection': 'close', u'x-openstack-nova-api-version': '2.1', u'date': 'Wed, 16 Sep 2020 14:38:46 GMT', u'content-type': 'application/json', u'x-openstack-request-id': 'req-80c6f4bd-9aab-4c50-8db8-d46da77d930e'}
        Body: {"console": {"url": "https://overcloud.redhat.local:13080/vnc_auto.html?path=%3Ftoken%3D885cd4b8-3c0a-487c-91b8-33fe326bc9f1", "type": "novnc"}}
}}}

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/tempest/api/compute/servers/test_novnc.py", line 188, in test_novnc
    self._validate_rfb_negotiation()
  File "/usr/lib/python2.7/site-packages/tempest/api/compute/servers/test_novnc.py", line 91, in _validate_rfb_negotiation
    'Token must be invalid because the connection '
  File "/usr/lib/python2.7/site-packages/unittest2/case.py", line 672, in assertFalse
    raise self.failureException(msg)
AssertionError: True is not false : Token must be invalid because the connection closed.

[Martin Schuppert (mschuppert)]

the issue is that we have NovaVncProxyNetwork [1], NovaLibvirtNetwork [2] and NovaApiNetwork [3] being used for vnc configuration. We should move to use one network for vnc configuration, probably the NovaLibvirtNetwork as this is also used in couple of places, e.g. live migration. If one of the networks get changed from internal_api the service configuration between libvirt, nova-compute and novnc proxy gets inconsistent and the console is broken as seen from the tempest test.

[1] https://github.com/openstack/tripleo-heat-templates/blob/master/deployment/nova/nova-compute-container-puppet.yaml#L1102
[2] https://github.com/openstack/tripleo-heat-templates/blob/master/deployment/nova/nova-libvirt-container-puppet.yaml#L441
[3] https://github.com/openstack/tripleo-heat-templates/blob/master/deployment/nova/nova-vnc-proxy-container-puppet.yaml#L256

[Martin Schuppert (mschuppert)]

https://review.opendev.org/c/openstack/tripleo-heat-templates/+/778617

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/786082
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/5171cd3d7a68c51eb6ee0f5a9d9b8e621e941f23
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 5171cd3d7a68c51eb6ee0f5a9d9b8e621e941f23
Author: Ade Lee <email address hidden>
Date: Tue Apr 13 13:14:15 2021 -0400

    Fix NovaVncProxyNetwork removal

    In commit [1], NovaVncProxyNetwork was removed. Unfortunately, the
    patch to backport [2] was incorrectly merged and some references to
    NovaVncProxyNetwork were not removed and renamed to NoveLibvirtNetwork.

    This patch fixes those references.

    [1] https://review.opendev.org/c/openstack/tripleo-heat-templates/+/778617
    [2] https://review.opendev.org/c/openstack/tripleo-heat-templates/+/781077
    Related-Bug: #1917719
    Change-Id: Id596898787a3ad16b87116e80a1c31902146168b

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/786217

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/786218

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Ade Lee <email address hidden>" on branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/786217
Reason: relevant patch to remove NovaVncroxyNetwork not yet merged

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Ade Lee <email address hidden>" on branch: stable/train
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/786218
Reason: relevant patch to remove NovaVncroxyNetwork not yet merged

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/781612
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/124419ca7a382a9a0eb5dacb9436a0416d5fd513
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit 124419ca7a382a9a0eb5dacb9436a0416d5fd513
Author: Martin Schuppert <email address hidden>
Date: Thu Mar 4 08:26:58 2021 +0100

    Use single NovaLibvirtNetwork to configure instance console components

    Nova vnc configuration right now uses NovaVncProxyNetwork,
    NovaLibvirtNetwork and NovaApiNetwork to configure the different
    components (novnc proxy, nova-compute and libvirt) for vnc.
    If one of the networks get changed from internal_api, the service
    configuration between libvirt, nova-compute and novnc proxy gets
    inconsistent and the console is broken.
    This changed to just use NovaLibvirtNetwork for configuring the vnc
    endpoints and removes NovaVncProxyNetwork completely.

    Conflicts:
     deployment/nova/nova-vnc-proxy-container-puppet.yaml

    Added missing change from https://review.opendev.org/c/openstack/tripleo-heat-templates/+/786082

    Depends-On: https://review.opendev.org/c/openstack/puppet-tripleo/+/781611

    Change-Id: Icef2481b65b41b524ad44eeecfbee4451006e1d2
    Closes-Bug: #1917719
    (cherry picked from commit dadf71fcae73825888b8ed4850de5a6ecf2fc29d)
    (cherry picked from commit 05b191d3e6254fcd6ded4fd8161456f3d5d89632)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/781614
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/860a68a4a9e31df4c9224833b094d1ef8173c0f4
Submitter: "Zuul (22348)"
Branch: stable/train

commit 860a68a4a9e31df4c9224833b094d1ef8173c0f4
Author: Martin Schuppert <email address hidden>
Date: Thu Mar 4 08:26:58 2021 +0100

    Use single NovaLibvirtNetwork to configure instance console components

    Nova vnc configuration right now uses NovaVncProxyNetwork,
    NovaLibvirtNetwork and NovaApiNetwork to configure the different
    components (novnc proxy, nova-compute and libvirt) for vnc.
    If one of the networks get changed from internal_api, the service
    configuration between libvirt, nova-compute and novnc proxy gets
    inconsistent and the console is broken.
    This changed to just use NovaLibvirtNetwork for configuring the vnc
    endpoints and removes NovaVncProxyNetwork completely.

    Conflicts:
     deployment/nova/nova-vnc-proxy-container-puppet.yaml

    Added missing change from https://review.opendev.org/c/openstack/tripleo-heat-templates/+/786082

    Depends-On: https://review.opendev.org/c/openstack/puppet-tripleo/+/781613

    Change-Id: Icef2481b65b41b524ad44eeecfbee4451006e1d2
    Closes-Bug: #1917719
    (cherry picked from commit dadf71fcae73825888b8ed4850de5a6ecf2fc29d)
    (cherry picked from commit 05b191d3e6254fcd6ded4fd8161456f3d5d89632)
    (cherry picked from commit 6c80882f9f52a442400646785def50f5bb8ec4ee)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 14.1.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 13.3.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 11.6.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 12.4.4 release.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/798301

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/798301
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/9da8a405952623e42f52e13ac948da1e57cf6906
Submitter: "Zuul (22348)"
Branch: master

commit 9da8a405952623e42f52e13ac948da1e57cf6906
Author: Martin Schuppert <email address hidden>
Date: Mon Jun 28 15:06:58 2021 +0200

    Remove NovaVncProxyNetwork from ServiceNetMap

    In Icef2481b65b41b524ad44eeecfbee4451006e1d2 we moved to use
    NovaLibvirtNetwork as the single network to configure instance console
    components.
    Due to how hieradata is being rendered via tripleo-hieradata, the
    removed NovaVncProxyNetwork resulted in no-vnc-proxy service to
    fall back to the control plane network because NovaVncProxyNetwork
    was removed.
    NovaVncProxyNetwork was then reintroduced with
    I595294956a4a7a0e2280c685ac1d2543edbc32f2 but as a side effect
    again introduced the inconsistency on the console configuration.

    This patch removes the NovaVncProxyNetwork from ServiceNetMap
    but set it as part of service_net_map.j2.yaml and keep it in
    sync with NovaLibvirtNetwork.

    Change-Id: I08fa1ee947574e6fac24885b17706a22054fcba9
    Closes-Bug: #1917719

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/798805

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/798832

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/798833

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/798834

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/798832
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/d18c6e92830bb7f6d8d62204747130678aa5a61c
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit d18c6e92830bb7f6d8d62204747130678aa5a61c
Author: Martin Schuppert <email address hidden>
Date: Mon Jun 28 15:06:58 2021 +0200

    Remove NovaVncProxyNetwork from ServiceNetMap

    In Icef2481b65b41b524ad44eeecfbee4451006e1d2 we moved to use
    NovaLibvirtNetwork as the single network to configure instance console
    components.
    Due to how hieradata is being rendered via tripleo-hieradata, the
    removed NovaVncProxyNetwork resulted in no-vnc-proxy service to
    fall back to the control plane network because NovaVncProxyNetwork
    was removed.
    NovaVncProxyNetwork was then reintroduced with
    I595294956a4a7a0e2280c685ac1d2543edbc32f2 but as a side effect
    again introduced the inconsistency on the console configuration.

    This patch removes the NovaVncProxyNetwork from ServiceNetMap
    but set it as part of service_net_map.j2.yaml and keep it in
    sync with NovaLibvirtNetwork.

    Conflicts:
        network/service_net_map.j2.yaml
        overcloud-resource-registry-puppet.j2.yaml

    Change-Id: I08fa1ee947574e6fac24885b17706a22054fcba9
    Closes-Bug: #1917719
    (cherry picked from commit 9da8a405952623e42f52e13ac948da1e57cf6906)
    (cherry picked from commit 2f0689d8d7e4078c4e9c025c3deb9b9e0a638d05)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/798805
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/2f0689d8d7e4078c4e9c025c3deb9b9e0a638d05
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 2f0689d8d7e4078c4e9c025c3deb9b9e0a638d05
Author: Martin Schuppert <email address hidden>
Date: Mon Jun 28 15:06:58 2021 +0200

    Remove NovaVncProxyNetwork from ServiceNetMap

    In Icef2481b65b41b524ad44eeecfbee4451006e1d2 we moved to use
    NovaLibvirtNetwork as the single network to configure instance console
    components.
    Due to how hieradata is being rendered via tripleo-hieradata, the
    removed NovaVncProxyNetwork resulted in no-vnc-proxy service to
    fall back to the control plane network because NovaVncProxyNetwork
    was removed.
    NovaVncProxyNetwork was then reintroduced with
    I595294956a4a7a0e2280c685ac1d2543edbc32f2 but as a side effect
    again introduced the inconsistency on the console configuration.

    This patch removes the NovaVncProxyNetwork from ServiceNetMap
    but set it as part of service_net_map.j2.yaml and keep it in
    sync with NovaLibvirtNetwork.

    Conflicts:
        network/service_net_map.j2.yaml
        overcloud-resource-registry-puppet.j2.yaml

    Change-Id: I08fa1ee947574e6fac24885b17706a22054fcba9
    Closes-Bug: #1917719
    (cherry picked from commit 9da8a405952623e42f52e13ac948da1e57cf6906)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/798833
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/de54e5ab630e01f8ac4c5543745d167922720c54
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit de54e5ab630e01f8ac4c5543745d167922720c54
Author: Martin Schuppert <email address hidden>
Date: Mon Jun 28 15:06:58 2021 +0200

    Remove NovaVncProxyNetwork from ServiceNetMap

    In Icef2481b65b41b524ad44eeecfbee4451006e1d2 we moved to use
    NovaLibvirtNetwork as the single network to configure instance console
    components.
    Due to how hieradata is being rendered via tripleo-hieradata, the
    removed NovaVncProxyNetwork resulted in no-vnc-proxy service to
    fall back to the control plane network because NovaVncProxyNetwork
    was removed.
    NovaVncProxyNetwork was then reintroduced with
    I595294956a4a7a0e2280c685ac1d2543edbc32f2 but as a side effect
    again introduced the inconsistency on the console configuration.

    This patch removes the NovaVncProxyNetwork from ServiceNetMap
    but set it as part of service_net_map.j2.yaml and keep it in
    sync with NovaLibvirtNetwork.

    Conflicts:
        network/service_net_map.j2.yaml
        overcloud-resource-registry-puppet.j2.yaml

    Change-Id: I08fa1ee947574e6fac24885b17706a22054fcba9
    Closes-Bug: #1917719
    (cherry picked from commit 9da8a405952623e42f52e13ac948da1e57cf6906)
    (cherry picked from commit 2f0689d8d7e4078c4e9c025c3deb9b9e0a638d05)
    (cherry picked from commit d18c6e92830bb7f6d8d62204747130678aa5a61c)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/798834
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/6f2d2d06d0eac032ee99a2841aca529a9ab3bebd
Submitter: "Zuul (22348)"
Branch: stable/train

commit 6f2d2d06d0eac032ee99a2841aca529a9ab3bebd
Author: Martin Schuppert <email address hidden>
Date: Mon Jun 28 15:06:58 2021 +0200

    Remove NovaVncProxyNetwork from ServiceNetMap

    In Icef2481b65b41b524ad44eeecfbee4451006e1d2 we moved to use
    NovaLibvirtNetwork as the single network to configure instance console
    components.
    Due to how hieradata is being rendered via tripleo-hieradata, the
    removed NovaVncProxyNetwork resulted in no-vnc-proxy service to
    fall back to the control plane network because NovaVncProxyNetwork
    was removed.
    NovaVncProxyNetwork was then reintroduced with
    I595294956a4a7a0e2280c685ac1d2543edbc32f2 but as a side effect
    again introduced the inconsistency on the console configuration.

    This patch removes the NovaVncProxyNetwork from ServiceNetMap
    but set it as part of service_net_map.j2.yaml and keep it in
    sync with NovaLibvirtNetwork.

    Conflicts:
        network/service_net_map.j2.yaml
        overcloud-resource-registry-puppet.j2.yaml

    Change-Id: I08fa1ee947574e6fac24885b17706a22054fcba9
    Closes-Bug: #1917719
    (cherry picked from commit 9da8a405952623e42f52e13ac948da1e57cf6906)
    (cherry picked from commit 2f0689d8d7e4078c4e9c025c3deb9b9e0a638d05)
    (cherry picked from commit d18c6e92830bb7f6d8d62204747130678aa5a61c)
    (cherry picked from commit e50a10a4bf914bbfbcbe9e15fe4d2d07aa7b71b5)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 14.2.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 13.4.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 12.4.5 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 15.1.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates train-eol release.