[SRU] config OPENSTACK_INSTANCE_RETRIEVE_IP_ADDRESSES does not apply to instance detail page

[Rodrigo Barbieri]

Setting the config option OPENSTACK_INSTANCE_RETRIEVE_IP_ADDRESSES to False successfully allows skipping neutron calls when loading the instance list page, therefore speeding up page loading. However, when clicking on an instance and loading the instance details page it still makes the neutron calls, taking a very long time.

The usage of the config option in the code could be adjusted to also be used when loading the instance details page, thus speeding up the page loading there as well.

===============
SRU Description
===============

[Impact]

Environments that have too many neutron (networking) ports are
very slow to load the instance list and instance detail pages.

The existing config OPENSTACK_INSTANCE_RETRIEVE_IP_ADDRESSES [1]
can be set to False for not retrieving/displaying IP addresses
(which requires the expensive/slow calls for neutron port list).

That does speed up the loading of the _instance list_ page,
but it's not applied to the _single instance_ detail page,
which remains slow.

By applying the config option when loading the instance detail
page as well, we speed up instance detail page loading and we
have minimal side effects / behavior changes, which are already
the same seen when displaying the instance list anyway (see [1]):

- IP addresses are not included in the detail page
  (this is aligned with the option's desired goal).

- Floating IP addresses (if used/available in the deployment)
  may take a while to be visible, but a page reload helps [1]
  (and users were already be subject to this in the list page):
  """
  Note that when disabling the query to neutron it takes some time
  until associated floating IPs are visible in the project instance
  table and users may reload the table to check them.
  """

This admittedly introduces a behavior change, however in this
case it seems arguably reasonable/acceptable for some reasons:

- The _default behavior_ does not change, as the new change
  is gated by the opt-in setting of config option to False.

- The _opt-in behavior_ change (once option is set to False)
  is aligned with the _existing_ behavior/goal of that option
  (i.e., not to retrieve/display IP addresses _somewhere_,
  just _extending_ it from instance _list_ to _details_ too).

- Users opt into that option for it _to address the issue_
  of slowness in Horizon when looking at instances (VMs),
  but it actually _does not address it_ fully -- i.e., one
  page (list) is addressed, but the other (details) is not.

  This patch/change improves the behavior/does achieve the
  intended goal (address slowness) in the details page too.

- This change is already present in upstream and Noble LTS,
  so users would eventually get to it during cloud upgrades.

[Test case]

1. Setting up the env

1a. Deploy openstack env with horizon/openstack-dashboard

1b. Declare and set OPENSTACK_INSTANCE_RETRIEVE_IP_ADDRESSES to False in /etc/openstack-dashboard/local_settings.py and restart apache2

2. Prepare to reproduce the bug

2a. Create a single VM successfully

2b. As we cannot easily create enough ports in the lab to replicate the slowness, we will rely on the message being present in the logs. Therefore, at this step we enable debug in horizon to see the messages. Set DEBUG to True in /etc/openstack-dashboard/local_settings.py and restart apache2.

3. Reproducing the bug

3a. Load the instance list page and verify that the following messages are not present in the logs:

GET /v2.0/floatingips?port_id=...
GET /v2.0/ports?tenant_id=...
GET /v2.0/networks?id=...
GET /v2.0/subnets

3b. Click on the instance to load the detail page and verify that the following messages ARE present in the logs:

GET /v2.0/floatingips?port_id=...
GET /v2.0/ports?tenant_id=...
GET /v2.0/networks?id=...
GET /v2.0/subnets

5. Install package that contains the fixed code

6. Confirm fix

6a. Repeat step 3a.

6b. Click on the instance to load the detail page and verify that the following messages are NOT present in the logs:

GET /v2.0/floatingips?port_id=...
GET /v2.0/ports?tenant_id=...
GET /v2.0/networks?id=...
GET /v2.0/subnets

[Where problems could occur]

The code has tested in upstream CI (without the addition of bug-specific functional tests) from master(Caracal) to stable/zed without any issue captured. Side effects documented at [1]. The code itself is a simple 2-liner with minimal to none chance of regression due to narrow scope of code change impact.

[Other Info]

None.

[1] https://github.com/openstack/horizon/blob/2b03b44f3adeea7e7a8aaab7777cccfa00614301/doc/source/configuration/settings.rst#openstack_instance_retrieve_ip_addresses

[Rodrigo Barbieri (rodrigo-barbieri2010)]

Patches already merged

https://review.opendev.org/#/q/I3e71a208a1c7212e168d63a259f2adddf27dbabf

[Rodrigo Barbieri (rodrigo-barbieri2010)]

Fix has merged in Caracal/Noble cycle, has been backported upstream to Bobcat, Antelope and Zed. It needs to have SRU'ed back to Ussuri.

Bobcat, Antelope and Zed could have Point Releases (as long as there one new tag upstream), but for Yoga, Xena, Wallaby, Victoria and Ussuri it will necessary to merge the diff directly into SRU code.

For simplicity, I believe it will be better to not wait for Point releases of Bobcat, Antelope and Zed because just this "waiting for a new tag" period could take months and all the other releases that don't have point releases would have to wait on that to get their SRU started

[Rodrigo Barbieri (rodrigo-barbieri2010)]

As I was creating the debdiffs, I faced some conflicts (just unit tests though) in Zed due to the tag being very old. I pinged the horizon PTL and he updated the hash for the next tag. Apparently we could have a new zed point release very soon [1].

I am attaching all debdiffs either way.

[1] https://review.opendev.org/c/openstack/releases/+/906842

[Rodrigo Barbieri (rodrigo-barbieri2010)]

The tag 23.0.1 for zed has been created, so I deleted the jammy-zed attachment

[Rodrigo Barbieri (rodrigo-barbieri2010)]

I have also just fixed bug LP#1728031 and backporting that all the way to yoga, to it is best to wait for that and combine with a single SRU.

[Rodrigo Barbieri (rodrigo-barbieri2010)]

actually, this one needs to be SRU'ed back to Ussuri, while LP#1728031 is up to yoga

[Rodrigo Barbieri (rodrigo-barbieri2010)]

SRU for bug LP#1728031 is now ready as well so it can be picked up and merged with this one

[Mauricio Faria de Oliveira (mfo)]

Adding bug tasks for Noble/Oracular as Fix Released, for clarity, per comment #2 and checks below.

$ git log --oneline -1 95089025fda7c8cce6f7195e2a63f7f09efc9e0a
95089025fda7 Extend configurable skippability of neutron calls to project instance detail

$ git describe --contains 95089025fda7
24.0.0~11^2

$ rmadison -a source horizon
...
 horizon | 4:24.0.0-0ubuntu1 | noble | source
 horizon | 4:24.0.0-0ubuntu1 | oracular | source

OK.
Double check:

$ pull-lp-source horizon noble
Found horizon 4:24.0.0-0ubuntu1 in noble
...

$ grep -A2 'def _update_addresses' horizon-24.0.0/openstack_dashboard/dashboards/project/instances/views.py
    def _update_addresses(self, instance):
        if not settings.OPENSTACK_INSTANCE_RETRIEVE_IP_ADDRESSES:
            return

[Mauricio Faria de Oliveira (mfo)]

Slightly reformatted the Impact section and clarified the behavior change/extension with the opt-in config.

[Mauricio Faria de Oliveira (mfo)]

Uploaded to Mantic.
Build-tested on PPA [1] with -proposed enabled.

[1] https://launchpad.net/~mfo/+archive/ubuntu/horizon-lp1728031-lp2054799-lp2055409/

[Mauricio Faria de Oliveira (mfo)]

Uploaded to Jammy.
Build-tested on PPA [1] with -proposed enabled.

[1] https://launchpad.net/~mfo/+archive/ubuntu/horizon-lp1728031-lp2054799-lp2055409/

[Mauricio Faria de Oliveira (mfo)]

Uploaded to Focal.
Build-tested on PPA [1] with -proposed enabled.

[1] https://launchpad.net/~mfo/+archive/ubuntu/horizon-lp1728031-lp2054799-lp2055409/

[Andreas Hasenack (ahasenack)]

Hello Rodrigo, or anyone else affected,

Accepted horizon into mantic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/horizon/4:23.3.0-0ubuntu1.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-mantic to verification-done-mantic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-mantic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Andreas Hasenack (ahasenack)]

Hello Rodrigo, or anyone else affected,

Accepted horizon into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/horizon/4:22.1.1-0ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Andreas Hasenack (ahasenack)]

Hello Rodrigo, or anyone else affected,

Accepted horizon into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/horizon/3:18.3.5-0ubuntu2.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted horizon (4:23.3.0-0ubuntu1.2) for mantic have finished running.
The following regressions have been reported in tests triggered by the package:

freezer-web-ui/unknown (arm64)
heat-dashboard/unknown (arm64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/mantic/update_excuses.html#horizon

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted horizon (4:22.1.1-0ubuntu1.1) for jammy have finished running.
The following regressions have been reported in tests triggered by the package:

heat-dashboard/unknown (arm64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/jammy/update_excuses.html#horizon

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!