Bug #2037339 “f/antelope: stx-openstack: `stx-openstackclients` ...” : Bugs : StarlingX

Luan Nunes Utimura (lutimura) on 2023-09-25

tags:	added: stx.9.0 stx.distro.openstack
description:	updated
Changed in starlingx:
assignee:	nobody → Luan Nunes Utimura (lutimura)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-10: Fix merged to root (f/antelope)

#1

Download full text (3.7 KiB)

Reviewed: https://review.opendev.org/c/starlingx/root/+/896277
Committed: https://opendev.org/starlingx/root/commit/5fa60a542120d7dd510aabae237c18839127705e
Submitter: "Zuul (22348)"
Branch: f/antelope

commit 5fa60a542120d7dd510aabae237c18839127705e
Author: Luan Nunes Utimura <email address hidden>
Date: Fri Sep 22 13:18:37 2023 -0300

Add sources.list for non-default layers

    After reestructuring `deb-local-binary` into multiple binary
    repositories, one for each layer, the image build now has access to
    three repositories by default:

      * deb-local-binary;
      * deb-local-binary-containers;
      * deb-local-build.

    Although these repositories are sufficient for the vast majority of
    images, there are some, such as `stx-openstackclients`, that require
    dependencies that are found in another repository:
    `deb-local-binary-openstack`.

This image, as the name suggests, contains all the OpenStack clients
currently supported by the StarlingX OpenStack application.

    As one can see in [1], although most clients are installed in the form
    of pip packages, i.e., through wheel files collected from the upstream
    indexes, there are two clients that are installed in the form of
    distribution packages:

* python3-cinderclient;
* python3-openstackclient.

    The reason why they are installed as distribution packages is because
    they have patches that are specific to StarlingX OpenStack -- [2] and
    [3] -- and also because the build procedure can easily identify them in
    `deb-local-build` and use the patched versions as expected by us (of
    course, it they have been built previously).

The problem, however, is that although this image is currently being
built, there is a detail in it that can cause problems at run time.

    Despite the build locating (and using) our patched packages, when it
    searches for their dependencies, it always ends up using binaries from
    repositories `deb-local-binary`, `deb-local-binary-containers` or
    `deb-local-build`.

    Which means that, if we have a dependency on
    `deb-local-binary-openstack` meant for OpenStack Antelope, and the same
    dependency on `deb-local-binary` meant for OpenStack Victoria, the image
    build will use the latter, for the simple fact that it do not consider
    the `openstack` layer (and, therefore, `deb-local-binary-openstack`).

    To get around this problem, this change seeks to add a `sources.list`
    for each build layer. By default, they are all disabled. However, if
    necessary, they can be enabled and used in image builds based on Docker
    or LOCI.

    Note: This change is the prerequisite of [4].
          The test plan with the `stx-openstackclients` image will be done
          there.

    [1] https://opendev.org/starlingx/openstack-armada-app/src/branch/f/antelope/upstream/openstack/python-openstackclient/debian/stx-openstackclient.stable_docker_image#L26-L27
    [2] https://opendev.org/starlingx/openstack-armada-app/src/branch/f/antelope/upstream/openstack/python-cinderclient/debian/patches
    [3] https://opend...

Reviewed:  https://review.opendev.org/c/starlingx/root/+/896277
Committed: https://opendev.org/starlingx/root/commit/5fa60a542120d7dd510aabae237c18839127705e
Submitter: "Zuul (22348)"
Branch:    f/antelope

commit 5fa60a542120d7dd510aabae237c18839127705e
Author: Luan Nunes Utimura <LuanNunes.Utimura@windriver.com>
Date:   Fri Sep 22 13:18:37 2023 -0300

Add sources.list for non-default layers
    
    After reestructuring `deb-local-binary` into multiple binary
    repositories, one for each layer, the image build now has access to
    three repositories by default:
    
      * deb-local-binary;
      * deb-local-binary-containers;
      * deb-local-build.
    
    Although these repositories are sufficient for the vast majority of
    images, there are some, such as `stx-openstackclients`, that require
    dependencies that are found in another repository:
    `deb-local-binary-openstack`.
    
    This image, as the name suggests, contains all the OpenStack clients
    currently supported by the StarlingX OpenStack application.
    
    As one can see in [1], although most clients are installed in the form
    of pip packages, i.e., through wheel files collected from the upstream
    indexes, there are two clients that are installed in the form of
    distribution packages:
    
      * python3-cinderclient;
      * python3-openstackclient.
    
    The reason why they are installed as distribution packages is because
    they have patches that are specific to StarlingX OpenStack -- [2] and
    [3] -- and also because the build procedure can easily identify them in
    `deb-local-build` and use the patched versions as expected by us (of
    course, it they have been built previously).
    
    The problem, however, is that although this image is currently being
    built, there is a detail in it that can cause problems at run time.
    
    Despite the build locating (and using) our patched packages, when it
    searches for their dependencies, it always ends up using binaries from
    repositories `deb-local-binary`, `deb-local-binary-containers` or
    `deb-local-build`.
    
    Which means that, if we have a dependency on
    `deb-local-binary-openstack` meant for OpenStack Antelope, and the same
    dependency on `deb-local-binary` meant for OpenStack Victoria, the image
    build will use the latter, for the simple fact that it do not consider
    the `openstack` layer (and, therefore, `deb-local-binary-openstack`).
    
    To get around this problem, this change seeks to add a `sources.list`
    for each build layer. By default, they are all disabled. However, if
    necessary, they can be enabled and used in image builds based on Docker
    or LOCI.
    
    Note: This change is the prerequisite of [4].
          The test plan with the `stx-openstackclients` image will be done
          there.
    
    [1] https://opendev.org/starlingx/openstack-armada-app/src/branch/f/antelope/upstream/openstack/python-openstackclient/debian/stx-openstackclient.stable_docker_image#L26-L27
    [2] https://opendev.org/starlingx/openstack-armada-app/src/branch/f/antelope/upstream/openstack/python-cinderclient/debian/patches
    [3] https://opendev.org/starlingx/openstack-armada-app/src/branch/f/antelope/upstream/openstack/python-openstackclient/debian/patches
    [4] https://review.opendev.org/c/starlingx/openstack-armada-app/+/896462
    
    Test Plan:
    PASS - Build `stx-debian` base image
    PASS - Build all container images with --clean and confirm that:
           1. All container images were built successfully;
           2. All container images were removed successfully.
    
    Partial-Bug: 2037339
    
    Change-Id: I36d1af990afad9fd69b723bf3a13b88673dd08ad
    Signed-off-by: Luan Nunes Utimura <LuanNunes.Utimura@windriver.com>

tags:

added: in-f-antelope

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-10: Fix merged to openstack-armada-app (f/antelope)

#2

Reviewed: https://review.opendev.org/c/starlingx/openstack-armada-app/+/896462
Committed: https://opendev.org/starlingx/openstack-armada-app/commit/418a7bcb6e21d64cc54e4ddda16a377a483d7e0b
Submitter: "Zuul (22348)"
Branch: f/antelope

commit 418a7bcb6e21d64cc54e4ddda16a377a483d7e0b
Author: Luan Nunes Utimura <email address hidden>
Date: Mon Sep 25 15:34:50 2023 -0300

stx-openstackclients: Add openstack in DIST_REPOS

    After introducing a new set of layer-specific `sources.list` files in
    [1], we can now update the `DIST_REPOS` of this image to additionally
    use the `openstack` layer binary repository when searching for
    dependencies of packages listed in `DIST_PACKAGES`.

This ensures that OpenStack Antelope packages will be built *without* OpenStack Victoria dependencies.

[1] https://review.opendev.org/c/starlingx/root/+/896277

    Test Plan:
    PASS - Build `stx-debian` base image
    PASS - Build `stx-openstackclients` container image and confirm that
           OpenStack Antelope dependencies were used in the build of
           `python3-cinderclient` and `python3-openstackclient`

Depends-On: https://review.opendev.org/c/starlingx/root/+/896277

Closes-Bug: 2037339

Change-Id: If94d5281c5c2633b59a6aaf39f0881fb5cec61bb
Signed-off-by: Luan Nunes Utimura <email address hidden>

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-11: Fix proposed to root (master)

#3

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/root/+/897944

Changed in starlingx:
status:	New → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-11: Fix proposed to openstack-armada-app (master)

#4

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/openstack-armada-app/+/897946

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-23: Fix merged to root (master)

#5

Download full text (3.8 KiB)

Reviewed: https://review.opendev.org/c/starlingx/root/+/897944
Committed: https://opendev.org/starlingx/root/commit/1f196aa31057577ef13a496866532a28ffd68092
Submitter: "Zuul (22348)"
Branch: master

commit 1f196aa31057577ef13a496866532a28ffd68092
Author: Luan Nunes Utimura <email address hidden>
Date: Fri Sep 22 13:18:37 2023 -0300

Add sources.list for non-default layers

    After reestructuring `deb-local-binary` into multiple binary
    repositories, one for each layer, the image build now has access to
    three repositories by default:

      * deb-local-binary;
      * deb-local-binary-containers;
      * deb-local-build.

    Although these repositories are sufficient for the vast majority of
    images, there are some, such as `stx-openstackclients`, that require
    dependencies that are found in another repository:
    `deb-local-binary-openstack`.

This image, as the name suggests, contains all the OpenStack clients
currently supported by the StarlingX OpenStack application.

    As one can see in [1], although most clients are installed in the form
    of pip packages, i.e., through wheel files collected from the upstream
    indexes, there are two clients that are installed in the form of
    distribution packages:

* python3-cinderclient;
* python3-openstackclient.

    The reason why they are installed as distribution packages is because
    they have patches that are specific to StarlingX OpenStack -- [2] and
    [3] -- and also because the build procedure can easily identify them in
    `deb-local-build` and use the patched versions as expected by us (of
    course, it they have been built previously).

The problem, however, is that although this image is currently being
built, there is a detail in it that can cause problems at run time.

    Despite the build locating (and using) our patched packages, when it
    searches for their dependencies, it always ends up using binaries from
    repositories `deb-local-binary`, `deb-local-binary-containers` or
    `deb-local-build`.

    Which means that, if we have a dependency on
    `deb-local-binary-openstack` meant for OpenStack Antelope, and the same
    dependency on `deb-local-binary` meant for OpenStack Victoria, the image
    build will use the latter, for the simple fact that it do not consider
    the `openstack` layer (and, therefore, `deb-local-binary-openstack`).

    To get around this problem, this change seeks to add a `sources.list`
    for each build layer. By default, they are all disabled. However, if
    necessary, they can be enabled and used in image builds based on Docker
    or LOCI.

    Note: This change is the prerequisite of [4].
          The test plan with the `stx-openstackclients` image will be done
          there.

    [1] https://opendev.org/starlingx/openstack-armada-app/src/branch/f/antelope/upstream/openstack/python-openstackclient/debian/stx-openstackclient.stable_docker_image#L26-L27
    [2] https://opendev.org/starlingx/openstack-armada-app/src/branch/f/antelope/upstream/openstack/python-cinderclient/debian/patches
    [3] https://opendev.o...

Reviewed:  https://review.opendev.org/c/starlingx/root/+/897944
Committed: https://opendev.org/starlingx/root/commit/1f196aa31057577ef13a496866532a28ffd68092
Submitter: "Zuul (22348)"
Branch:    master

commit 1f196aa31057577ef13a496866532a28ffd68092
Author: Luan Nunes Utimura <LuanNunes.Utimura@windriver.com>
Date:   Fri Sep 22 13:18:37 2023 -0300

Add sources.list for non-default layers
    
    After reestructuring `deb-local-binary` into multiple binary
    repositories, one for each layer, the image build now has access to
    three repositories by default:
    
      * deb-local-binary;
      * deb-local-binary-containers;
      * deb-local-build.
    
    Although these repositories are sufficient for the vast majority of
    images, there are some, such as `stx-openstackclients`, that require
    dependencies that are found in another repository:
    `deb-local-binary-openstack`.
    
    This image, as the name suggests, contains all the OpenStack clients
    currently supported by the StarlingX OpenStack application.
    
    As one can see in [1], although most clients are installed in the form
    of pip packages, i.e., through wheel files collected from the upstream
    indexes, there are two clients that are installed in the form of
    distribution packages:
    
      * python3-cinderclient;
      * python3-openstackclient.
    
    The reason why they are installed as distribution packages is because
    they have patches that are specific to StarlingX OpenStack -- [2] and
    [3] -- and also because the build procedure can easily identify them in
    `deb-local-build` and use the patched versions as expected by us (of
    course, it they have been built previously).
    
    The problem, however, is that although this image is currently being
    built, there is a detail in it that can cause problems at run time.
    
    Despite the build locating (and using) our patched packages, when it
    searches for their dependencies, it always ends up using binaries from
    repositories `deb-local-binary`, `deb-local-binary-containers` or
    `deb-local-build`.
    
    Which means that, if we have a dependency on
    `deb-local-binary-openstack` meant for OpenStack Antelope, and the same
    dependency on `deb-local-binary` meant for OpenStack Victoria, the image
    build will use the latter, for the simple fact that it do not consider
    the `openstack` layer (and, therefore, `deb-local-binary-openstack`).
    
    To get around this problem, this change seeks to add a `sources.list`
    for each build layer. By default, they are all disabled. However, if
    necessary, they can be enabled and used in image builds based on Docker
    or LOCI.
    
    Note: This change is the prerequisite of [4].
          The test plan with the `stx-openstackclients` image will be done
          there.
    
    [1] https://opendev.org/starlingx/openstack-armada-app/src/branch/f/antelope/upstream/openstack/python-openstackclient/debian/stx-openstackclient.stable_docker_image#L26-L27
    [2] https://opendev.org/starlingx/openstack-armada-app/src/branch/f/antelope/upstream/openstack/python-cinderclient/debian/patches
    [3] https://opendev.org/starlingx/openstack-armada-app/src/branch/f/antelope/upstream/openstack/python-openstackclient/debian/patches
    [4] https://review.opendev.org/c/starlingx/openstack-armada-app/+/896462
    
    Test Plan:
    PASS - Build `stx-debian` base image
    PASS - Build all container images with --clean and confirm that:
           1. All container images were built successfully;
           2. All container images were removed successfully.
    
    Partial-Bug: 2037339
    
    Change-Id: I36d1af990afad9fd69b723bf3a13b88673dd08ad
    Signed-off-by: Luan Nunes Utimura <LuanNunes.Utimura@windriver.com>
    (cherry picked from commit 5fa60a542120d7dd510aabae237c18839127705e)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-23: Fix merged to openstack-armada-app (master)

#6

Reviewed: https://review.opendev.org/c/starlingx/openstack-armada-app/+/897946
Committed: https://opendev.org/starlingx/openstack-armada-app/commit/22a84736a3555c009e9f57bf225d423ebe175f1e
Submitter: "Zuul (22348)"
Branch: master

commit 22a84736a3555c009e9f57bf225d423ebe175f1e
Author: Luan Nunes Utimura <email address hidden>
Date: Mon Sep 25 15:34:50 2023 -0300

stx-openstackclients: Add openstack in DIST_REPOS

    After introducing a new set of layer-specific `sources.list` files in
    [1], we can now update the `DIST_REPOS` of this image to additionally
    use the `openstack` layer binary repository when searching for
    dependencies of packages listed in `DIST_PACKAGES`.

This ensures that OpenStack Antelope packages will be built *without* OpenStack Victoria dependencies.

[1] https://review.opendev.org/c/starlingx/root/+/896277

    Test Plan:
    PASS - Build `stx-debian` base image
    PASS - Build `stx-openstackclients` container image and confirm that
           OpenStack Antelope dependencies were used in the build of
           `python3-cinderclient` and `python3-openstackclient`

Depends-On: https://review.opendev.org/c/starlingx/root/+/897944

Closes-Bug: 2037339

    Change-Id: If94d5281c5c2633b59a6aaf39f0881fb5cec61bb
    Signed-off-by: Luan Nunes Utimura <email address hidden>
    (cherry picked from commit 418a7bcb6e21d64cc54e4ddda16a377a483d7e0b)

Changed in starlingx:
status:	In Progress → Fix Released

Ghada Khalil (gkhalil) on 2024-04-10

Changed in starlingx:
importance:	Undecided → Low

StarlingX

f/antelope: stx-openstack: `stx-openstackclients` is being built with dependencies on wrong versions

Bug Description

Other bug subscribers

Remote bug watches