intermittent build failure in containerd

Bug #1862038 reported by Scott Little
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
StarlingX
Fix Released
High
Don Penney

Bug Description

Brief Description
-----------------
Cengn build failed on package containerd.

Branch: master
Timestamp: 20200205T023000Z

There were no relevant change logs affecting that package vs the previous successful build at timestamp 20200204T162546Z.

The issue is seen only intermittently when doing a full build. It can be reliably reproduced with a targeted build of containerd.

The issue is likely a missing BuildRequires

Severity
--------
Major: intermittent build failures

Steps to Reproduce
------------------
in a clean workspace ...

build-pkgs --no-build-avoidance --no-descendants --no-required containerd

Expected Behavior
------------------
containerd builds successfully

Actual Behavior
----------------
containerd failes to build

Reproducibility
---------------
<Reproducible/Intermittent/Seen once>
reproducible

System Configuration
--------------------
N/A

Branch/Pull Time/Commit
-----------------------
20200205T023000Z

Last Pass
---------
20200204T162546Z

Timestamp/Logs
--------------
+ /usr/lib/rpm/find-debuginfo.sh --strict-build-id -m --run-dwz --dwz-low-mem-die-limit 10000000 --dwz-max-die-limit 110000000 /builddir/build/BUILD/src
extracting debug info from /builddir/build/BUILDROOT/containerd-1.3.0-1.tis.x86_64/usr/bin/runc
extracting debug info from /builddir/build/BUILDROOT/containerd-1.3.0-1.tis.x86_64/usr/bin/crictl
*** ERROR: No build ID note found in /builddir/build/BUILDROOT/containerd-1.3.0-1.tis.x86_64/usr/bin/crictl

Test Activity
-------------
Build

Workaround
----------
N/A

Don Penney (dpenney)
Changed in starlingx:
assignee: nobody → Don Penney (dpenney)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to integ (master)

Fix proposed to branch: master
Review: https://review.opendev.org/706009

Changed in starlingx:
status: New → In Progress
Revision history for this message
Ghada Khalil (gkhalil) wrote :

stx.4.0 / high priority - issue results in build failures

tags: added: stx.4.0 stx.build stx.integ
Changed in starlingx:
importance: Undecided → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to integ (master)

Reviewed: https://review.opendev.org/706009
Committed: https://git.openstack.org/cgit/starlingx/integ/commit/?id=77b632e28f27ab53a840f098fcfbba3db2714a1f
Submitter: Zuul
Branch: master

commit 77b632e28f27ab53a840f098fcfbba3db2714a1f
Author: Don Penney <email address hidden>
Date: Wed Feb 5 11:28:32 2020 -0500

    Fix containerd build failure

    The 20200205T023000Z CENGN build failed on containerd due to a build
    ordering issue. In the failed build, containerd was built ahead of
    rpm, and the mock build environment for the containerd build ran with
    the stock CentOS version of RPM. Unfortunately, it appears this
    version of RPM fails when trying to build the debuginfo for a golang
    package. There are currently two other golang packages in StarlingX,
    but these have debuginfo disabled in the spec.

    Adding a version-specific dependency in the containerd spec to ensure
    the newer RPM is installed resolves the issue.

    Change-Id: Ia7c85751012bbd0c3b83a2496bd7424e123eef93
    Closes-Bug: 1862038
    Co-Authored-By: Scott Little <email address hidden>
    Signed-off-by: Don Penney <email address hidden>

Changed in starlingx:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to integ (f/centos8)
Download full text (8.0 KiB)

Reviewed: https://review.opendev.org/705861
Committed: https://git.openstack.org/cgit/starlingx/integ/commit/?id=a3267c2016e1805f05e72e9063b2db8a227891c2
Submitter: Zuul
Branch: f/centos8

commit 77b632e28f27ab53a840f098fcfbba3db2714a1f
Author: Don Penney <email address hidden>
Date: Wed Feb 5 11:28:32 2020 -0500

    Fix containerd build failure

    The 20200205T023000Z CENGN build failed on containerd due to a build
    ordering issue. In the failed build, containerd was built ahead of
    rpm, and the mock build environment for the containerd build ran with
    the stock CentOS version of RPM. Unfortunately, it appears this
    version of RPM fails when trying to build the debuginfo for a golang
    package. There are currently two other golang packages in StarlingX,
    but these have debuginfo disabled in the spec.

    Adding a version-specific dependency in the containerd spec to ensure
    the newer RPM is installed resolves the issue.

    Change-Id: Ia7c85751012bbd0c3b83a2496bd7424e123eef93
    Closes-Bug: 1862038
    Co-Authored-By: Scott Little <email address hidden>
    Signed-off-by: Don Penney <email address hidden>

commit 7b7959e9b5cc9a68a6fcffba44bca2f84643b133
Author: Al Bailey <email address hidden>
Date: Tue Jan 28 07:49:23 2020 -0600

    Update pylint target for python3 and upper constraints

    This change imposes the upper constraint in tox to protect from
    future releases causing random breakage.

    Ex: A new version of python-libvirt was released Jan 23 2020
    which will not install on python2.

    This change also enables the python3 target for pylint which will
    allow the upper constraint to be changed to a more recent version
    when all the tox files are aligned.

    Change-Id: I9056778085d32b3401df60c20d67cff0a21dfe97
    Story: 2004515
    Task: 38496
    Signed-off-by: Al Bailey <email address hidden>

commit 424ba94a9aa9e64fe1c0d2099b63e7d979b492cd
Author: Jim Somerville <email address hidden>
Date: Fri Jan 24 12:36:06 2020 -0500

    Mellanox Driver: Disable use of kernel page pool functionality

    Problem: The out-of-tree Mellanox driver detects the presence
    of page pool support in the new kernel, and thus wants to use it.
    However, page pools are not configured to be on in the new kernel
    by default (CONFIG_PAGE_POOL), and not only that, the config
    option is hidden ie. it is not user selectable. The built-in
    Mellanox driver selects it, but we don't use the built-in driver.
    The out-of-tree driver does compile but not all pieces of it
    will load properly, specifically the mlx5 pieces which rely on
    page pool functionality being enabled in the kernel.

    Solution: Simply disable kernel page pool use in the
    out-of-tree Mellanox driver, making it work the same way as
    it did with the older kernel.

    Change-Id: If7e7155867d539352fcd0ea3acd5a17dd9d9579f
    Closes-Bug: 1860347
    Signed-off-by: Jim Somerville <email address hidden>

commit 7165b3539c75009311d3d4360a15b6ee4c7a4573
Author: Lin Shuicheng <email address hidden>
Date: Sun Jan 19 01:59:42 2020 +0000

   ...

Read more...

tags: added: in-f-centos8
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.