Bug #1891416 “stx-platformclients docker image build doesn't inc...” : Bugs : StarlingX

Revision history for this message

Ghada Khalil (gkhalil) wrote on 2020-08-13: Re: Remote CLI “dcmanager fw-update-strategy show” returns an error when using image stx-platformclients:stx.5.0-v1.4.0

#1

As per Don Penney's investigation, this appears to be a docker build issue introduced some weeks ago. See email sent to stx-discuss:
http://lists.starlingx.io/pipermail/starlingx-discuss/2020-August/009415.html

summary:	- Remote CLI “dcmanager fw-update-strategy show” returns an error when use - image stx-platformclients:stx.5.0-v1.4.0 + Remote CLI “dcmanager fw-update-strategy show” returns an error when + using image stx-platformclients:stx.5.0-v1.4.0
tags:	added: stx.build
Changed in starlingx:
importance:	Undecided → High

Revision history for this message

Ghada Khalil (gkhalil) wrote on 2020-08-17:

#2

stx.5.0 / high priority - this needs to be addressed for that the platform-clients docker image can be updated with new content from the release. Note: This is also an issue for stx.4.0, but is less severe as it's not likely the image will need to be updated in that branch.

Changed in starlingx:
status:	New → Incomplete
status:	Incomplete → Opinion
status:	Opinion → Triaged
tags:	added: stx.5.0
Changed in starlingx:
assignee:	nobody → zhipeng liu (zhipengs)

Revision history for this message

zhipeng liu (zhipengs) wrote on 2020-08-18:

#3

It needs to patching cgtsclient,fmclient and distributedcloud_client to support py3 wheel build.
As talked with Chant(Chengde), he will take this task.

Thanks!
Zhipeng

YU CHENGDE (chant) on 2020-08-24

Changed in starlingx:
assignee:	zhipeng liu (zhipengs) → YU CHENGDE (chant)

Revision history for this message

Ghada Khalil (gkhalil) wrote on 2020-11-11:

#4

Is there an update on this issue?

Revision history for this message

YU CHENGDE (chant) wrote on 2020-11-11:

#5

flock_layered_build.log Edit (240.1 KiB, text/plain)

Modiy the spec for distributedcloud-client, fmclient, cgts-client

https://review.opendev.org/#/c/762341/
https://review.opendev.org/#/c/762340/
https://review.opendev.org/#/c/762342/

build-rpms successfully
build-iso successfully

Next step, I will deploy a distributed cloud, and try the command is workable or not

Revision history for this message

YU CHENGDE (chant) wrote on 2020-11-11:

#6

build_iso.log Edit (1.4 MiB, text/plain)

log form build-iso

Revision history for this message

Ghada Khalil (gkhalil) wrote on 2020-11-12:

#7

Adding the latest discussion on this topic from the mailing list archives:
http://lists.starlingx.io/pipermail/starlingx-discuss/2020-September/009544.html

Revision history for this message

Ghada Khalil (gkhalil) wrote on 2021-02-02:

#8

Is there an update on this?

Revision history for this message

Ghada Khalil (gkhalil) wrote on 2021-03-24:

#9

This is an issue with building the stx-platformclients docker image. Title is updated to better reflect the issue. As discussed with Frank Miller, assigning to the build team.

Changed in starlingx:
assignee:	YU CHENGDE (chant) → Davlet Panech (dpanech)
summary:	- Remote CLI “dcmanager fw-update-strategy show” returns an error when - using image stx-platformclients:stx.5.0-v1.4.0 + stx-platformclients docker image build doesn't include the correct + content

OpenStack Infra (hudson-openstack) on 2021-04-13

Changed in starlingx:
status:	Triaged → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2021-04-14: Fix merged to root (master)

#10

Reviewed: https://review.opendev.org/c/starlingx/root/+/785508
Committed: https://opendev.org/starlingx/root/commit/2bdac9ac3b09332604919c8e7d39c1676c106ae5
Submitter: "Zuul (22348)"
Branch: master

commit 2bdac9ac3b09332604919c8e7d39c1676c106ae5
Author: Davlet Panech <email address hidden>
Date: Thu Apr 8 15:17:44 2021 -0400

Build and use py2 wheels with py2 projects

Current build process can't handle python2-based projects because we
only build wheels based on Ussury (python3). This patch fixes this.

    - build-wheels/docker/stable-wheels-py2.cfg
    - build-wheels/docker/dev-wheels-py2.cfg
      Resurrected original cfg files from commit
      aa317323716a7f5bb844bb8867eeb8b01313b51e (before upgrade to Ussuri)
      and saved with -py2 suffix

- build-wheels/docker/centos-dockerfile
Install python2 packages in addition to python3

    - build-wheels/docker/docker-build-wheel.sh
      * Build with python3 or python2 depending on the environment
      * Suppress status message at the end because its better handled by the
        parent script (build-base-wheels.sh)

    - build-wheels/build-base-wheels.sh
      * Generate 2 wheel directories, "base" and "base-py2" at each
        invocation
      * Use the same builder image and docker script (with different
        environment) to generate wheels based on different cfg files.
        Save python3 wheels in "base" and python2 wheels in "base-py2"

    - build-wheels/build-wheel-tarball.sh
      * New command-line parameter --python2: generate *-wheels-py2.tar
        from "base-py2"
      * Use ussuri or train constraints depending on the presence of
        --python2

    - build-docker-images/build-stx-images.sh
      * Use *-wheels-py2.tar for python2/loci projects
      * Fail early on missing --wheels/--wheels-py2
      * Fail if multiple projects use the same LABEL
      * Remove support for config files

    - build-docker-images/docker-image-build-centos-dev.cfg
    - build-docker-images/docker-image-build-centos-stable.cfg
      Removed

    Change-Id: I2ca444f258a537ed2ba6f68206d32cf59e1802b4
    Partial-Bug: 1891416
    Signed-off-by: Davlet Panech <email address hidden>

Reviewed:  https://review.opendev.org/c/starlingx/root/+/785508
Committed: https://opendev.org/starlingx/root/commit/2bdac9ac3b09332604919c8e7d39c1676c106ae5
Submitter: "Zuul (22348)"
Branch:    master

commit 2bdac9ac3b09332604919c8e7d39c1676c106ae5
Author: Davlet Panech <davlet.panech@windriver.com>
Date:   Thu Apr 8 15:17:44 2021 -0400

Build and use py2 wheels with py2 projects
    
    Current build process can't handle python2-based projects because we
    only build wheels based on Ussury (python3). This patch fixes this.
    
    - build-wheels/docker/stable-wheels-py2.cfg
    - build-wheels/docker/dev-wheels-py2.cfg
      Resurrected original cfg files from commit
      aa317323716a7f5bb844bb8867eeb8b01313b51e (before upgrade to Ussuri)
      and saved with -py2 suffix
    
    - build-wheels/docker/centos-dockerfile
      Install python2 packages in addition to python3
    
    - build-wheels/docker/docker-build-wheel.sh
      * Build with python3 or python2 depending on the environment
      * Suppress status message at the end because its better handled by the
        parent script (build-base-wheels.sh)
    
    - build-wheels/build-base-wheels.sh
      * Generate 2 wheel directories, "base" and "base-py2" at each
        invocation
      * Use the same builder image and docker script (with different
        environment) to generate wheels based on different cfg files.
        Save python3 wheels in "base" and python2 wheels in "base-py2"
    
    - build-wheels/build-wheel-tarball.sh
      * New command-line parameter --python2: generate *-wheels-py2.tar
        from "base-py2"
      * Use ussuri or train constraints depending on the presence of
        --python2
    
    - build-docker-images/build-stx-images.sh
      * Use *-wheels-py2.tar for python2/loci projects
      * Fail early on missing --wheels/--wheels-py2
      * Fail if multiple projects use the same LABEL
      * Remove support for config files
    
    - build-docker-images/docker-image-build-centos-dev.cfg
    - build-docker-images/docker-image-build-centos-stable.cfg
      Removed
    
    Change-Id: I2ca444f258a537ed2ba6f68206d32cf59e1802b4
    Partial-Bug: 1891416
    Signed-off-by: Davlet Panech <davlet.panech@windriver.com>

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2021-04-15: Fix proposed to root (master)

#11

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/root/+/786486

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2021-04-15: Fix proposed to clients (master)

#12

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/clients/+/786490

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2021-04-16: Fix merged to root (master)

#13

Reviewed: https://review.opendev.org/c/starlingx/root/+/786486
Committed: https://opendev.org/starlingx/root/commit/52ca64eab3d3911cae010aa690621e3df82f6abe
Submitter: "Zuul (22348)"
Branch: master

commit 52ca64eab3d3911cae010aa690621e3df82f6abe
Author: Davlet Panech <email address hidden>
Date: Thu Apr 15 15:07:56 2021 -0400

Update stx-platformclients tag to stx.5.0-v1.4.1

    This commit updates the tag to pick up the patch that builds this image
    in python2, rather than python3 environment because stx-platformclients
    currently doesn't support python3.

    Change-Id: Ic8785d271f9ea7311708464a8cb75cc306ef209c
    Partial-Bug: 1891416
    Signed-off-by: Davlet Panech <email address hidden>

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2021-04-16: Fix merged to clients (master)

#14

Reviewed: https://review.opendev.org/c/starlingx/clients/+/786490
Committed: https://opendev.org/starlingx/clients/commit/0c2645d8ddc59d84dde75a558f3b1660b0fc3206
Submitter: "Zuul (22348)"
Branch: master

commit 0c2645d8ddc59d84dde75a558f3b1660b0fc3206
Author: Davlet Panech <email address hidden>
Date: Thu Apr 15 15:31:27 2021 -0400

Update stx-platformclients tag to stx.5.0-v1.4.1

    This commit updates the tag to pick up the patch that builds this image
    in python2, rather than python3 environment because stx-platformclients
    currently doesn't support python3.

    Change-Id: Ie254debfeee9c9a5cb1ec24a16b6075a0a326077
    Depends-On: https://review.opendev.org/c/starlingx/root/+/786486
    Depends-On: https://review.opendev.org/c/starlingx/clients/+/786504
    Closes-Bug: 1891416
    Signed-off-by: Davlet Panech <email address hidden>

Changed in starlingx:
status:	In Progress → Fix Released

Revision history for this message

Ghada Khalil (gkhalil) wrote on 2021-04-17:

#15

@Davlet, please cherrypick any commits that didn't make it to the r/stx.5.0 release branch

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2021-04-19: Fix proposed to root (r/stx.5.0)

#16

Fix proposed to branch: r/stx.5.0
Review: https://review.opendev.org/c/starlingx/root/+/786886

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2021-04-19: Fix proposed to clients (r/stx.5.0)

#17

Fix proposed to branch: r/stx.5.0
Review: https://review.opendev.org/c/starlingx/clients/+/786890

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2021-04-19: Fix merged to root (r/stx.5.0)

#18

Reviewed: https://review.opendev.org/c/starlingx/root/+/786886
Committed: https://opendev.org/starlingx/root/commit/b91c6ddf672f7eabacf46b5a63a81605fabb4d0e
Submitter: "Zuul (22348)"
Branch: r/stx.5.0

commit b91c6ddf672f7eabacf46b5a63a81605fabb4d0e
Author: Davlet Panech <email address hidden>
Date: Thu Apr 15 15:07:56 2021 -0400

Update stx-platformclients tag to stx.5.0-v1.4.1

    This commit updates the tag to pick up the patch that builds this image
    in python2, rather than python3 environment because stx-platformclients
    currently doesn't support python3.

    Change-Id: Ic8785d271f9ea7311708464a8cb75cc306ef209c
    Partial-Bug: 1891416
    Signed-off-by: Davlet Panech <email address hidden>
    (cherry picked from commit 52ca64eab3d3911cae010aa690621e3df82f6abe)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2021-04-19: Fix merged to clients (r/stx.5.0)

#19

Reviewed: https://review.opendev.org/c/starlingx/clients/+/786890
Committed: https://opendev.org/starlingx/clients/commit/6f82077370dd730dc3e7bf50febd0ee7408e5ee4
Submitter: "Zuul (22348)"
Branch: r/stx.5.0

commit 6f82077370dd730dc3e7bf50febd0ee7408e5ee4
Author: Davlet Panech <email address hidden>
Date: Thu Apr 15 15:31:27 2021 -0400

Update stx-platformclients tag to stx.5.0-v1.4.1

    This commit updates the tag to pick up the patch that builds this image
    in python2, rather than python3 environment because stx-platformclients
    currently doesn't support python3.

    Change-Id: Ie254debfeee9c9a5cb1ec24a16b6075a0a326077
    Closes-Bug: 1891416
    Depends-On: https://review.opendev.org/c/starlingx/root/+/786886
    Signed-off-by: Davlet Panech <email address hidden>
    (cherry picked from commit 0c2645d8ddc59d84dde75a558f3b1660b0fc3206)

Ghada Khalil (gkhalil) on 2021-04-19

tags:

added: in-r-stx50

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2021-05-19: Fix proposed to root (f/centos8)

#20

Fix proposed to branch: f/centos8
Review: https://review.opendev.org/c/starlingx/root/+/792232

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2021-05-27: Fix merged to root (f/centos8)

#21

Download full text (24.5 KiB)

Reviewed: https://review.opendev.org/c/starlingx/root/+/792232
Committed: https://opendev.org/starlingx/root/commit/7cb55d28111dbf7458ed3f01a5eda5c4dfdb124b
Submitter: "Zuul (22348)"
Branch: f/centos8

commit f457bd15b9ce0be512ec96abcb9b858b0bab2ea8
Author: Scott Little <email address hidden>
Date: Fri Apr 16 01:39:17 2021 -0400

Improved branching tools

    create_branches_and_tags.sh:
    - Update the .gitreview files in branched git repos.
    - When updating a manifest, add the ability to update and
      use the default revision field.
    - Create two levels of manifest lockdown, soft and hard.
      Soft lockdown only sets sha revisions on unbranched projects
      that lack a revision, or set the revision to master.
      Hard lockdown applies to all unbranched projects.

    push_branches_tags.sh:
    - opendev no longer accepts 'git push' for the delivery of
      new branches with updates. Instead we must now
      use separate commands to deliver the tag, the branch,
      and any updates.

    Closes-Bug: 1924762
    Signed-off-by: Scott Little <email address hidden>
    Change-Id: I6d669ddc80cc9b3cb9e72d65a64589dbccf43ae3

commit 0babd33b6d851dd11492c26e92c8a6ac2c2557de
Author: Rafael Jardim <email address hidden>
Date: Wed May 12 11:25:54 2021 -0300

Update stx-platformclients tag to stx.5.0-v1.4.3

This commit updates the image with the updated clients.

    Test:
    Some normal commands
    Commands related with https dcmanager that wasn't working
    System application-upload that wasn't working when executed
    from remote cli

    Closes-Bug: 1928233
    Closes-Bug: 1928231
    Signed-off-by: Rafael Jardim <email address hidden>
    Change-Id: I8a0d12f699336a4412be5ff3c73cfb8d59038780

commit a163d7723e659e89c37ec933d1b0f9aa638a6a73
Author: Cole Walker <email address hidden>
Date: Wed May 5 09:39:17 2021 -0400

Update image tag for notificationservice-base

Update image tag to stx.5.0-v1.0.4 for notificationservice-base

Closes-Bug: 1924201
Closes-Bug: 1924197

Signed-off-by: Cole Walker <email address hidden>
Change-Id: Id863c4f154cc0b39e30ee8986fc07f9856a22826

commit 84c45f5e3a241237887af9db89d8c0aa1f8923e0
Author: Charles Short <email address hidden>
Date: Sun May 2 12:04:04 2021 -0400

Fix wheels tarball generation

    In d7c5a54ab94bce6635b83d91a807d28f97836a81, django was dropped in favor
    of rfc3986. However the wheel was not added to the build-wheels
    generation which breaks the docker images. Also add the migrate wheel
    since it was mising as well. Add the required wheels in order to
    build the docker image properly.

    Test:
    - Build new centos-stable-wheels tarball
    - Build stx-keystone-api-proxy container

Closes-Bug: 1926795

Signed-off-by: Charles Short <email address hidden>
Change-Id: Ib6f0abfdcc82ca14f92ebc5b45fe8df961e804ee

commit 062353b6de7d4b0e017203c3e6086891bd6b9213
Author: Cole Walker <email address hidden>
Date: Mon May 3 09:48:54 2021 -0400

Update image tag for notificationclient-base...

Reviewed:  https://review.opendev.org/c/starlingx/root/+/792232
Committed: https://opendev.org/starlingx/root/commit/7cb55d28111dbf7458ed3f01a5eda5c4dfdb124b
Submitter: "Zuul (22348)"
Branch:    f/centos8

commit f457bd15b9ce0be512ec96abcb9b858b0bab2ea8
Author: Scott Little <scott.little@windriver.com>
Date:   Fri Apr 16 01:39:17 2021 -0400

Improved branching tools
    
    create_branches_and_tags.sh:
    - Update the .gitreview files in branched git repos.
    - When updating a manifest, add the ability to update and
      use the default revision field.
    - Create two levels of manifest lockdown, soft and hard.
      Soft lockdown only sets sha revisions on unbranched projects
      that lack a revision, or set the revision to master.
      Hard lockdown applies to all unbranched projects.
    
    push_branches_tags.sh:
    - opendev no longer accepts 'git push' for the delivery of
      new branches with updates.  Instead we must now
      use separate commands to deliver the tag, the branch,
      and any updates.
    
    Closes-Bug: 1924762
    Signed-off-by: Scott Little <scott.little@windriver.com>
    Change-Id: I6d669ddc80cc9b3cb9e72d65a64589dbccf43ae3

commit 0babd33b6d851dd11492c26e92c8a6ac2c2557de
Author: Rafael Jardim <rafaeljordao.jardim@windriver.com>
Date:   Wed May 12 11:25:54 2021 -0300

Update stx-platformclients tag to stx.5.0-v1.4.3
    
    This commit updates the image with the updated clients.
    
    Test:
    Some normal commands
    Commands related with https dcmanager that wasn't working
    System application-upload that wasn't working when executed
    from remote cli
    
    Closes-Bug: 1928233
    Closes-Bug: 1928231
    Signed-off-by: Rafael Jardim <rafaeljordao.jardim@windriver.com>
    Change-Id: I8a0d12f699336a4412be5ff3c73cfb8d59038780

commit a163d7723e659e89c37ec933d1b0f9aa638a6a73
Author: Cole Walker <cole.walker@windriver.com>
Date:   Wed May 5 09:39:17 2021 -0400

Update image tag for notificationservice-base
    
    Update image tag to stx.5.0-v1.0.4 for notificationservice-base
    
    Closes-Bug: 1924201
    Closes-Bug: 1924197
    
    Signed-off-by: Cole Walker <cole.walker@windriver.com>
    Change-Id: Id863c4f154cc0b39e30ee8986fc07f9856a22826

commit 84c45f5e3a241237887af9db89d8c0aa1f8923e0
Author: Charles Short <charles.short@windriver.com>
Date:   Sun May 2 12:04:04 2021 -0400

Fix wheels tarball generation
    
    In d7c5a54ab94bce6635b83d91a807d28f97836a81, django was dropped in favor
    of rfc3986. However the wheel was not added to the build-wheels
    generation which breaks the docker images. Also add the migrate wheel
    since it was mising as well. Add the required wheels in order to
    build the docker image properly.
    
    Test:
    - Build new centos-stable-wheels tarball
    - Build stx-keystone-api-proxy container
    
    Closes-Bug: 1926795
    
    Signed-off-by: Charles Short <charles.short@windriver.com>
    Change-Id: Ib6f0abfdcc82ca14f92ebc5b45fe8df961e804ee

commit 062353b6de7d4b0e017203c3e6086891bd6b9213
Author: Cole Walker <cole.walker@windriver.com>
Date:   Mon May 3 09:48:54 2021 -0400

Update image tag for notificationclient-base
    
    Update image tag to stx.5.0-v1.0.4 for notificationclient-base
    
    Closes-Bug: 1924198
    Closes-Bug: 1926532
    
    Signed-off-by: Cole Walker <cole.walker@windriver.com>
    Change-Id: Ie09ae4f28864aa6ee8a549bd15be5645519df7b9

commit 33895c29224792ad1845d8f9bfa37ee484899114
Author: Don Penney <don.penney@windriver.com>
Date:   Tue Apr 20 19:31:53 2021 -0400

Add MIRROR_LOCAL support to build-stx-images.sh
    
    As the loci build runs in a container, it is unable to directly use a
    PROJECT_REPO ref that requires ssh keys to be setup. In order to
    support this, a MIRROR_LOCAL option is added to loci image directives
    files.
    
    If the image directives file has MIRROR_LOCAL=yes, the build utility
    will first setup a bare clone of the PROJECT_REPO in the local
    workspace and instead pass a reference to that to the loci build.
    
    This functionality can be used by developers to test an image build
    using a private forked repo that requires an ssh key to access, for
    example. This should not be used in the formal build, as CENGN and
    other developers are unlikely to have the required access.
    
    Note: This requires that the build server in use has lighttpd server
    setup with external access, as the container cannot access localhost.
    
    Change-Id: Ibd260fbb47d2bece4dc27e1cf1c026fb5cd5ff0f
    Story: 2003907
    Task: 42364
    Co-Authored-By: Scott Little <scott.little@windriver.com>
    Signed-off-by: Don Penney <don.penney@windriver.com>

commit e16bd7d01b14862b08f8c58751102afddbe0eb82
Author: Rafael Jardim <rafaeljordao.jardim@windriver.com>
Date:   Tue Apr 20 13:52:13 2021 -0300

Update stx-platformclients tag to stx.5.0-v1.4.2
    
    This commit updates the image with the updated clients
    cgts-client and distributed-cloud-client to execute both
    in python2 and python3.
    
    Story: 2007106
    Task: 42314
    Signed-off-by: Rafael Jardim <rafaeljordao.jardim@windriver.com>
    Change-Id: Ia037a8e437ffd371bd20558eff716b2678a7d171

commit 0178b539ed092f2b78bb09c6c35056676419c7e7
Author: Giana Francisco <francisco.giana@windriver.com>
Date:   Wed Apr 21 09:33:40 2021 -0300

build-img: build PATCH.img-addon as REL and non-RR patch
    
    Added parameters --status REL and --reboot-required N when
    PATCH.img-addon is created from build-img script.
    In order to allow to sw-patch commits the PATCH.img-addon.patch
    
    Story: 2007858
    Task: 42316
    
    Signed-off-by: Giana Francisco <francisco.giana@windriver.com>
    Change-Id: Ief73924e52f364e70f7ad2713480536c953c055b

commit 26aba37d117600a4f4553d833f58df438548ae01
Author: Thiago Brito <thiago.brito@windriver.com>
Date:   Fri Apr 16 11:07:32 2021 -0400

Fixing dev stream py3 wheels
    
    Some wheels on the py3 files were still pointing towards the py2
    versions, some were outdated with the wheels used on the stable
    build, so fixing this so we can get a proper build from the dev
    stream.
    
    Signed-off-by: Thiago Brito <thiago.brito@windriver.com>
    Change-Id: I2b69d9afa06d22441d8270c25dbb0a3cd08a4b99

commit 52ca64eab3d3911cae010aa690621e3df82f6abe
Author: Davlet Panech <davlet.panech@windriver.com>
Date:   Thu Apr 15 15:07:56 2021 -0400

Update stx-platformclients tag to stx.5.0-v1.4.1
    
    This commit updates the tag to pick up the patch that builds this image
    in python2, rather than python3 environment because stx-platformclients
    currently doesn't support python3.
    
    Change-Id: Ic8785d271f9ea7311708464a8cb75cc306ef209c
    Partial-Bug: 1891416
    Signed-off-by: Davlet Panech <davlet.panech@windriver.com>

commit 2bdac9ac3b09332604919c8e7d39c1676c106ae5
Author: Davlet Panech <davlet.panech@windriver.com>
Date:   Thu Apr 8 15:17:44 2021 -0400

Build and use py2 wheels with py2 projects
    
    Current build process can't handle python2-based projects because we
    only build wheels based on Ussury (python3). This patch fixes this.
    
    - build-wheels/docker/stable-wheels-py2.cfg
    - build-wheels/docker/dev-wheels-py2.cfg
      Resurrected original cfg files from commit
      aa317323716a7f5bb844bb8867eeb8b01313b51e (before upgrade to Ussuri)
      and saved with -py2 suffix
    
    - build-wheels/docker/centos-dockerfile
      Install python2 packages in addition to python3
    
    - build-wheels/docker/docker-build-wheel.sh
      * Build with python3 or python2 depending on the environment
      * Suppress status message at the end because its better handled by the
        parent script (build-base-wheels.sh)
    
    - build-wheels/build-base-wheels.sh
      * Generate 2 wheel directories, "base" and "base-py2" at each
        invocation
      * Use the same builder image and docker script (with different
        environment) to generate wheels based on different cfg files.
        Save python3 wheels in "base" and python2 wheels in "base-py2"
    
    - build-wheels/build-wheel-tarball.sh
      * New command-line parameter --python2: generate *-wheels-py2.tar
        from "base-py2"
      * Use ussuri or train constraints depending on the presence of
        --python2
    
    - build-docker-images/build-stx-images.sh
      * Use *-wheels-py2.tar for python2/loci projects
      * Fail early on missing --wheels/--wheels-py2
      * Fail if multiple projects use the same LABEL
      * Remove support for config files
    
    - build-docker-images/docker-image-build-centos-dev.cfg
    - build-docker-images/docker-image-build-centos-stable.cfg
      Removed
    
    Change-Id: I2ca444f258a537ed2ba6f68206d32cf59e1802b4
    Partial-Bug: 1891416
    Signed-off-by: Davlet Panech <davlet.panech@windriver.com>

commit 986e68b992314f1e1e9fdad9b2c3bc0a85747c5f
Author: Carmen Rata <carmen.rata@windriver.com>
Date:   Wed Apr 7 22:28:27 2021 -0400

Add openscap rpms to image.inc
    
    In order to be up-to-date with openscap tools used to scan
    for security violations we need to update the openscap rpms
    from:
    openscap-scanner-1.2.17-2.el7.x86_64
    openscap-1.2.17-2.el7.x86_64
    to:
    openscap-1.2.17-13.el7_9.x86_64.rpm
    openscap-scanner-1.2.17-13.el7_9.x86_64.rpm
    Added the 2 openscap rpms to image.inc to include them in
    the iso.
    
    Story: 2008668
    Task: 42232
    Depends-On: https://review.opendev.org/c/starlingx/tools/+/784801
    
    Signed-off-by: Carmen Rata <carmen.rata@windriver.com>
    Change-Id: Id775dd299b35c085ef7f0762b778b8832c1fa68f

commit 3ef97d279c527d1c1a79b95aeebcb6c7e00a863d
Author: Davlet Panech <davlet.panech@windriver.com>
Date:   Thu Mar 25 18:21:51 2021 -0400

centos7: delete yum.pid after yum-builddep
    
    Pre-dnf yum-builddep leaves a stale yum.pid file behind with its own
    process ID. If that PID happens to be reused and match an existing
    process, a subsequent yum & co invocation hangs.
    
    Solution: create a wrapper script that deletes the pid file if
    necessary.
    
    Change-Id: I821643f576645d78ab1c29cdccefa12740bbc12f
    Closes-Bug: 1920805
    Signed-off-by: Davlet Panech <davlet.panech@windriver.com>

commit 290b3ea8d2b91a3eb4845e9df856119a13ecb2a9
Author: Davlet Panech <davlet.panech@windriver.com>
Date:   Thu Mar 25 09:57:30 2021 -0400

Revert "centos7: delete yum.lock after yum-builddep"
    
    This reverts commit 90aa450e427e21e6dbe3fc230c089e909ad19f29.
    
    The original commit breaks the build-iso script.
    
    Change-Id: I73c9bc8e502c8401922d107b0b78cd88511a2a4b
    Signed-off-by: Davlet Panech <davlet.panech@windriver.com>

commit 90aa450e427e21e6dbe3fc230c089e909ad19f29
Author: Davlet Panech <davlet.panech@windriver.com>
Date:   Mon Mar 22 16:13:06 2021 -0400

centos7: delete yum.lock after yum-builddep
    
    Pre-dnf yum-builddep leaves a stale yum.pid file behind with its own
    process ID. If that PID happens to be reused and match an existing
    process, a subsequent yum & co invocation hangs.
    
    Solution: create a wrapper script that deletes the pid file if
    necessary.
    
    Change-Id: Iee5c6a092835103a5dc52979345fa84e8a36c084
    Closes-Bug: 1920805
    Signed-off-by: Davlet Panech <davlet.panech@windriver.com>

commit 8cea0ccceb90de4005e556ca2b6929b259f572c0
Author: Jose Infanzon <jose.infanzon@windriver.com>
Date:   Tue Mar 16 14:57:38 2021 -0400

Update stx-fm-trap-subagent tag
    
    Update yaml file with a new tag for stx-fm-trap-subagent,
    due to a new commit delivered to support IPv6
    
    Story: 2008132
    Task: 42063
    Signed-off-by: Jose Infanzon <jose.infanzon@windriver.com>
    Change-Id: Id8a12b4c68b887ff153490af5e8084d7ee5224fe

commit fcba6679ab63cf2b384489e10e9ca244952ec6c4
Author: Cole Walker <cole.walker@windriver.com>
Date:   Mon Mar 15 13:10:59 2021 -0400

Update notificationclient-base image tag
    
    Update the notificationclient-base image tag to stx.5.0-v1.0.3
    
    Story: 2008529
    Task: 42054
    
    Signed-off-by: Cole Walker <cole.walker@windriver.com>
    Change-Id: I6312ecddf82445b8ecb7a16e059a29064eaa289d

commit 78be59c758945cb10c1e5569669f8764603eccc8
Author: Scott Little <scott.little@windriver.com>
Date:   Wed Mar 3 11:02:43 2021 -0500

Fix memory overcommit that caused OOM killer
    
    Parallel package builds use large ramdisks.
    It's important not to commit too much memory to these
    ram disks, or we may push the system into memory exhaustion.
    At that stage the Kernel will invoke the OOM killer,
    It will likely select our build, or worse someone else's build,
    to sacrifice.
    
    The current algorithm only considers free memory at the instant
    the parallel build starts.  It does not consider how many
    other builds are in flight, but might not have allocated their
    ramdisk yet.  The other build intends to use the memory, we see
    the memory as free and try to use the same memory.
    
    Solution is to consider total memory, and number of builds
    already running or which might foreseeably start in the near future
    (share factor) to derive an alternate estimate of memory available.
    We then allocate the lesser amount.
    
    Also fixed some issues with cleaning up of child processes when
    a newer mockchain-parallel is in use.
    
    Closes-Bug: 1917525
    Signed-off-by: Scott Little <scott.little@windriver.com>
    Change-Id: Iab178c6f9acbd5a209d66d0da21f367911f34905

commit d4409383d4fd1d5fb9656f61548547c2784836d4
Author: Babak Sarashki <babak.sarashki@windriver.com>
Date:   Fri Feb 12 00:18:46 2021 +0000

build-tools: include ICE driver for E810 in the installer
    
    This commit adds ice driver (version 1.2.1) to iso. This is
    needed for network boot.
    
    Story: 2008436
    Task: 41398
    
    Depends-On: https://review.opendev.org/c/starlingx/kernel/+/776262
    
    Signed-off-by: Babak Sarashki <babak.sarashki@windriver.com>
    Change-Id: I233174df1da5b2c86e6684289663bec775e049eb

commit 0972ffe2464293657b717ae84bc1fef890f7ffc1
Author: Scott Little <scott.little@windriver.com>
Date:   Fri Feb 19 11:40:12 2021 -0500

mock 2.7 requires additional config options
    
    Mock 2.7 defaults to using dnf.
    
    The centos 7 mock config must provide additional
    arguements to ensure a sane yum environment.  With
    out them it will try to use dnf.
    
    Closes-Bug: 1916275
    Signed-off-by: Scott Little <scott.little@windriver.com>
    Change-Id: I4801accf05b68e1e0bdf8975d1b4d989fd0a303f

commit 1d669071f928d5d1a63df4636176ea4ba71667f9
Author: Scott Little <scott.little@windriver.com>
Date:   Fri Feb 19 11:40:12 2021 -0500

Support mock >= 2.7
    
    Starting in Mock 2.7, mock's util.py was split,
    and a function we need was moved to the new config.py.
    
    Closes-Bug: 1916275
    Change-Id: I4be183a855ca1fd8c18b67c63a0d69585598c2b3
    Signed-off-by: Scott Little <scott.little@windriver.com>

commit c19642ada5e54bd919abc26bdd33f4ecd469b697
Author: Steven Webster <steven.webster@windriver.com>
Date:   Wed Feb 17 13:40:23 2021 -0500

Update k8-cni-sriov image to stx.5.0-v2.6-7-gb18123d8
    
    This commit uprevs the SR-IOV CNI image to pick up a few bug
    fixes.  Specifically, this commit will allow rate-limiting
    configuration on a VF to be retained after the VF has been
    used by a pod (and pod subsequently deleted).
    
    Testing:
    
    NICs:
    Ethernet Controller X710 for 10GbE SFP+
    Mellanox MT27700 Family [ConnectX-4]
    
    Functional:
    Connectivity testing (kernel + DPDK)
    Devices allocated appropriately to pod
    Rate-limiting information retained after pod deletion
    
    Partial-Bug: #1915951
    
    Signed-off-by: Steven Webster <steven.webster@windriver.com>
    Change-Id: I3eff245392312f969aa23d162c7d45f75f598afe

commit e6454552ae4f1c7e240c54babe19539f788595d2
Author: Cole Walker <cole.walker@windriver.com>
Date:   Wed Feb 17 16:22:39 2021 -0500

Update image tags for ptp-notification
    
    Update the image tags for:
    notificationservice-base
    notificationclient-base
    
    Story: 2008529
    Task: 41768
    
    Signed-off-by: Cole Walker <cole.walker@windriver.com>
    Change-Id: Ia12d582d38121af8bb482c0150c5fadbbf4f464c

commit a595a39242d37cfd3c228f8fdbd1396b666a8e1d
Author: Cole Walker <cole.walker@windriver.com>
Date:   Thu Feb 4 13:03:18 2021 -0500

Add ptp-notification images to tag management
    
    Includes tags for 3 images built in the
    ptp-notification-armada-app repo:
    
    notificationservice-base
    notificationclient-base
    locationservice-base
    
    Story: 2008529
    Task: 41768
    
    Signed-off-by: Cole Walker <cole.walker@windriver.com>
    Change-Id: I25a1973bfd1b78787001a7427ed464c36bfacca4

commit 71d5ae9df9268be2803e2a45277e65c7c84c1fdb
Author: Scott Little <scott.little@windriver.com>
Date:   Tue Feb 9 13:12:06 2021 -0500

build-img fails if not run under $MY_WORKSPACE
    
    Build-img may need to generate a patch that is to be applied to the
    image being produced.  The patch build tool drops the patch into
    the current directory.  No explicit working directory was set.
    A subsequent step tries to apply the patch, but assumes it is in
    $MY_WORKSPACE.
    
    We want all build artifacts to be placed under $MY_WORKSPACE, so
    set $MY_WORKSPACE as the current directory before creating the patch.
    
    Closes-Bug: 1915182
    Signed-off-by: Scott Little <scott.little@windriver.com>
    Change-Id: Ie2fbafae7a0ec90fc11e4af3d67dd15aec27c909

commit 395308f881f90c8beb94a82844ee4bcebc560e4f
Author: Cole Walker <cole.walker@windriver.com>
Date:   Thu Feb 4 13:53:25 2021 -0500

Update .gitignore with /ptp-notification-armada-app
    
    Required to keep repo status and git status commands clean
    
    Story: 2008529
    Task: 41770
    
    Signed-off-by: Cole Walker <cole.walker@windriver.com>
    Change-Id: Ib00800c61cafcb174027c4c936ef000eb59c009e

commit 0aec356f26c745d100f12081ac88f51353e41092
Author: Teresa Ho <teresa.ho@windriver.com>
Date:   Tue Jan 19 12:23:25 2021 -0500

Update tag for stx-oidc-client to stx.5.0-v1.0.4
    
    Closes-Bug: 1911244
    
    Signed-off-by: Teresa Ho <teresa.ho@windriver.com>
    Change-Id: I0f49f13dd3e111d436ef6b32d8576136faad53bb

commit 023323eb44f2cb0903853e05938ff8b1ccb22051
Author: Melissa Wang <melissa.wang@windriver.com>
Date:   Fri Jan 8 11:13:55 2021 -0500

build-img: add option to include additional rpms
    
    This change adds the option to add new packages to the qcow2 image
    after the iso has been built.
    
    Story: 2007858
    Task: 41557
    
    Change-Id: I7a75df2b62f56899ee4b1b69e360e07b09e1e391
    Signed-off-by: Melissa Wang <melissa.wang@windriver.com>

commit e6022190a0dd04bac3a350db549d5666d5f43d23
Author: Chen, Haochuan Z <haochuan.z.chen@intel.com>
Date:   Fri Jan 8 10:10:56 2021 +0800

Add ceph performance tuning tool in dev image
    
    Story: 2008497
    Task: 41554
    
    Change-Id: I705189b560c513ef2aaa06003a9b77518f39d28d
    Signed-off-by: Chen, Haochuan Z <haochuan.z.chen@intel.com>

commit fe4f9a9b88b8d93e5623888c134edef132e148c7
Author: Gonzalo Gallardo <gonzalo.gallardo@windriver.com>
Date:   Tue Jan 5 12:31:03 2021 -0300

Add SNMP images to tag management
    
    Update yaml file to provide information about the source images of SNMP
    app and its references for static tags.
    
    Story: 2008132
    Task: 41527
    Signed-off-by: Gonzalo Gallardo <gonzalo.gallardo@windriver.com>
    Change-Id: I219bcc250eae62fed8c22e1670907474dab3b135

commit 776fd4a557bb91337d790a3fe04ea914f1cd9acc
Author: Melissa Wang <melissa.wang@windriver.com>
Date:   Fri Dec 18 15:57:49 2020 -0500

build-img: Remove by-path symlinks for AWS images
    
    In the graphical installation of an AWS-compatible image, there are
    duplicate symlinks created by the udev persistent storage rules. This
    change comments out the deprecated persistent storage rules in
    /usr/lib/udev/rules.d/ during the setup work for the AWS image. This
    issue has not been found in regular qcow2 images.
    
    Tested on OpenStack in a distributed cloud system and on AWS.
    
    Story: 2007858
    Task: 41482
    
    Change-Id: Icbf9e753042e29536bd92b13e4d779d97de1c5f2
    Signed-off-by: Melissa Wang <melissa.wang@windriver.com>

commit 427193dd6c5bae371e6191916edbdc36f6332b66
Author: Don Penney <don.penney@windriver.com>
Date:   Mon Dec 21 15:25:27 2020 -0500

Exclude .tox folder when checking file ages
    
    The content of a .tox folder should not be considered when building
    srpms. Additionally, as this folder can be fairly large and include
    tens of thousands of files, a find listing can take some time. This
    commit updates these find commands to exclude the .tox folder from the
    search in order to save time if a user has run tox tests prior to the
    build.
    
    Change-Id: I035d730af8cdd443487f6ead3eec9e1e54f28709
    Related-Bug: 1908940
    Signed-off-by: Don Penney <don.penney@windriver.com>

commit db323440cdb28dc629d3d6e4645e8f1532b03713
Author: Poornima <poornima.y.n@intel.com>
Date:   Sat Dec 19 02:57:56 2020 +0530

Add SDO-rv-service to .gitignore
    
    - .gitignore: Add SDO-rv-service to .gitignore
    
    Story: 2008117
    Task: 41480
    
    Signed-off-by: Poornima <poornima.y.n@intel.com>
    Change-Id: Ifdf4d8950794f9d9711a860beb504c32629768bc

commit 357abccddc48e1a2382dab645ea30da2dce14431
Author: Scott Little <scott.little@windriver.com>
Date:   Thu Dec 17 13:55:52 2020 -0500

Correctly get the age of tarballs through their symlink
    
    find throws an error if the PATH_LIST references a tarball
    under stx/downloads, which is a symlink to the tarballs
    real location under the stx-tools.
    
    Modify the find command to follow the symlink.
    
    Closes-Bug: 1908570
    Signed-off-by: Scott Little <scott.little@windriver.com>
    Change-Id: Ia691c211715d561b3a4ffca6c4408927aa1e48e8

commit a64a3078aa1ee2cb411f786e9b9c3526b0c7f36d
Author: Scott Little <scott.little@windriver.com>
Date:   Thu Dec 17 14:02:40 2020 -0500

Ignore downloads-backup* directories
    
    Tell git to ignore the stx/download-backup* directories
    that are now produced by populate_downloads.sh.
    
    Partial-bug: 1908297
    Signed-off-by: Scott Little <scott.little@windriver.com>
    Change-Id: I7899250266d5145708312ed414a24f2a8815051b

commit 468097ba39cc7e34f9ea921d1250a6d565cd36f4
Author: Don Penney <don.penney@windriver.com>
Date:   Wed Dec 9 17:29:46 2020 -0500

Remove Django from wheels tarball
    
    Wheels added to the wheels.cfg file result in modification to the
    upper-constraints.txt included in the wheels tarball to point to the
    explicit wheel added by the wheels.cfg. The inclusion of Django
    resulted in an upper-constraints.txt entry locking the version to
    2.1.5.
    
    However, the openstack/horizon requirements.txt file includes the
    following: Django>=2.2,<3.0
    
    This results in a conflict between the constraints and requirements.
    Recent updates to the pip resolver exposed the conflict, and pip was
    unable to find a version of Django to fit both.
    
    Since the Django entry was just downloading a pre-built wheel, as
    opposed to building an appropriate wheel from tarball, this update
    removes the entry altogether to avoid creating this conflict.
    
    Change-Id: I870a72f312c781a9b000fc98c4ed104c22469ebd
    Closes-Bug: 1907290
    Signed-off-by: Don Penney <don.penney@windriver.com>

commit 77576b7207909f86ee2505ec90c71c1a4f216da4
Author: Scott Little <scott.little@windriver.com>
Date:   Wed Oct 28 14:12:13 2020 -0400

Add support CentOS-8/dnf/mock-2.6 based builds
    
    This update will retain support for CentOS-7/yum/mock-1.4 based builds.
    The build environment will be queried to discover which environment
    it is building in, and modify the commands we issue accordingly.
    
    In CentOS 8, DNF replaces both YUM and REPOQUERY.
    While DNF tries to be a transparent replacement of the old tools,
    there are also subtle changes to the supported arguments.
    
    I will provide independent mock.cfg.prototypes for centos7 vs centos8.
    Changes in generate-centos-repo.sh under stx-tools will be required to
    select the correct prototype.
    
    Add support for mock 2.6. Mock 2.6 is python 3, and it processes the
    'root' and 'rootdir' arguments slightly differently.
    
    Also change the order of arguments to tar within default_build_srpm.
    The latest tar only honors '--exclude' if it precedes other arguments.
    
    Story: 2006729
    Depends-On: https://review.opendev.org/762700
    Signed-off-by: Scott Little <scott.little@windriver.com>
    Change-Id: I826be2051e535e6a4c08ad17124f453b04210668

tags:

added: in-f-centos8

Ghada Khalil (gkhalil) on 2021-10-27

tags:

removed: stx.retestneeded

Revision history for this message

Thiago Paiva Brito (outbrito) wrote on 2022-01-10 (last edit on 2022-01-10):

#22

Hi folks,

I got into this LP after I failed to build the stx-platformclients and stx-pci-irq-affinity-agent following the docs and was checking the change at [1].

Shouldn't we have changed the docs to reflect the changes done here? I don't see any reference to use --python2 on build-wheels-tarball.sh or to set --wheels-py2 on build-stx-images.sh on the main docker image build doc referece at [2]:

[1] https://opendev.org/starlingx/root/commit/2bdac9ac3b09332604919c8e7d39c1676c106ae5
[2] https://docs.starlingx.io/developer_resources/build_docker_image.html#building-wheels

StarlingX

stx-platformclients docker image build doesn't include the correct content

Bug Description

Other bug subscribers

Bug attachments

Remote bug watches