build-pkgs --edit does not work for kernel modules

Bug #1958411 reported by M. Vefa Bicakci
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
StarlingX
Fix Released
Medium
M. Vefa Bicakci

Bug Description

Brief Description
-----------------

I noticed last week that 'build-pkgs --edit' does not work for any kernel modules in the StarlingX kernel repository. Here is an example for a failure involving the i40e kernel module:

===
raw_create_tarballs_extract_script ...
EXTRACT_TO_DIR=.../std/srpm_work/i40e-kmod/gits/i40e-kmod.spec
++ umask 022
++ cd .../std/srpm_work/i40e-kmod/gits/i40e-kmod.spec
++ source scl_source enable devtoolset-8
+++ _scl_source_help='Usage: source scl_source <action> [<collection> ...]

Don'\''t use this script outside of SCL scriptlets!

Options:
    -h, --help display this help and exit'
+++ '[' 2 -eq 0 -o enable = -h -o enable = --help ']'
+++ '[' -z '' ']'
+++ _recursion=false
+++ '[' -z '' ']'
+++ _scl_scriptlet_name=enable
+++ shift 1
+++ '[' -z '' ']'
+++ _scl_dir=/etc/scl/conf
+++ '[' '!' -e /etc/scl/conf ']'
+++ for arg in '"$@"'
+++ _scl_prefix_file=/etc/scl/conf/devtoolset-8
++++ cat /etc/scl/conf/devtoolset-8
+++ _scl_prefix=
ERROR: Failed in script '.../std/srpm_work/i40e-kmod/extract_script'
ERROR: tar_and_spec_extract_to_git (2361): tarball_extract failed ...
===

Further debugging indicates that this functionality was broken with the following commit:
  https://review.opendev.org/plugins/gitiles/starlingx/kernel/+/f438b8b5f6fb3f9df660fbe9ee5575b0549ba837

I have a patch that resolves this issue almost ready for publication, and I am opening this bug report as a placeholder.

Severity
--------
Major: build-pkgs --edit does not work, preventing proper modification of kernel modules.

Steps to Reproduce
------------------
build-pkgs --edit --clean i40e-kmod
build-pkgs --edit i40e-kmod

Expected Behavior
------------------
build-pkgs --edit should work.

Actual Behavior
----------------
build-pkgs --edit reports the errors quoted in the description and exits.

Reproducibility
---------------
Reproducible.

System Configuration
--------------------
Not applicable. This is a build issue.

Branch/Pull Time/Commit
-----------------------
StarlingX master as of this writing is affected.

Last Pass
---------
Prior to the merge of the aforementioned commit, build-pkgs was working.

Timestamp/Logs
--------------
(Please see the description.)

Test Activity
-------------
Developer testing and normal use.

Workaround
----------
None, to the best of my knowledge.

Changed in starlingx:
status: New → Confirmed
assignee: nobody → M. Vefa Bicakci (vbicakci)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kernel (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/kernel/+/825333

Changed in starlingx:
status: Confirmed → In Progress
Ghada Khalil (gkhalil)
tags: added: stx.build stx.distro.other
Changed in starlingx:
importance: Undecided → Medium
tags: added: stx.7.0
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kernel (master)
Download full text (3.5 KiB)

Reviewed: https://review.opendev.org/c/starlingx/kernel/+/825333
Committed: https://opendev.org/starlingx/kernel/commit/98156ab9116d98d0e10ad618f6586476d3da5dc1
Submitter: "Zuul (22348)"
Branch: master

commit 98156ab9116d98d0e10ad618f6586476d3da5dc1
Author: M. Vefa Bicakci <email address hidden>
Date: Thu Jan 13 19:02:53 2022 -0500

    kernel-modules: Fix 'build-pkgs --edit'

    Since commit f438b8b5f6fb ("Support out of tree modules in kernel
    5.10.", 2021-06-30), the use of 'build-pkgs --edit' has resulted in the
    following errors:

      ===
      raw_create_tarballs_extract_script ...
      EXTRACT_TO_DIR=.../std/srpm_work/i40e-kmod/gits/i40e-kmod.spec
      ++ umask 022
      ++ cd .../std/srpm_work/i40e-kmod/gits/i40e-kmod.spec
      ++ source scl_source enable devtoolset-8
      +++ _scl_source_help='Usage: source scl_source <action> [<collection> ...]

      Don'\''t use this script outside of SCL scriptlets!

      Options:
          -h, --help display this help and exit'
      +++ '[' 2 -eq 0 -o enable = -h -o enable = --help ']'
      +++ '[' -z '' ']'
      +++ _recursion=false
      +++ '[' -z '' ']'
      +++ _scl_scriptlet_name=enable
      +++ shift 1
      +++ '[' -z '' ']'
      +++ _scl_dir=/etc/scl/conf
      +++ '[' '!' -e /etc/scl/conf ']'
      +++ for arg in '"$@"'
      +++ _scl_prefix_file=/etc/scl/conf/devtoolset-8
      ++++ cat /etc/scl/conf/devtoolset-8
      +++ _scl_prefix=
      ERROR: Failed in script '.../std/srpm_work/i40e-kmod/extract_script'
      ERROR: tar_and_spec_extract_to_git (2361): tarball_extract failed ...
      ===

    This is caused by the use of 'source scl_source enable ...' in the
    %prep sections of the out-of-tree kernel modules' spec files. In
    summary, the build procedure is executed with a shell that has 'xtrace'
    (i.e., 'set -x') enabled, and the 'source' statement to pull in the
    software collections for updated toolchains causes debugging lines to be
    printed out. These lines are in turn attempted to be parsed by the
    cgcs-root/build-tools/srpm-utils shell script to generate a source code
    extraction script for the '--edit' mode. However, the unexpected shell
    debugging output confuses srpm-utils, which causes 'build-pkgs --edit'
    to fail.

    All of this can be avoided by removing the 'source scl_source enable'
    lines from the %prep sections, which are only expected to unpack the
    source code, and hence are not involved in compilation, and hence do not
    need to be aware of updated toolchains. This commit takes this approach
    instead of attempting to modify the srpm-utils shell script, given the
    author's lack of familiarity with the script in question.

    This commit also removes unnecessary use of the 'llvm-toolset-7.0'
    software collection from the kernel modules for two reasons:
    - The modified kernel modules do not use the LLVM/clang toolchain.
    - The attempt to use the llvm-toolset-7.0 toolchain was likely broken
      from day 1, because the kernel modules do not actually depend on the
      llvm-toolset-7.0 toolchain via a 'Requires:' line in their spec files.
      No e...

Read more...

Changed in starlingx:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.