Debian: build-pkgs -c does not always regenerate *.dsc files
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
StarlingX |
Fix Released
|
Low
|
M. Vefa Bicakci |
Bug Description
Brief Description
-----------------
When a Debian-based StarlingX package makes use of a "dl_hook" and when the dl_files and src_files entries in the meta_data.yaml do /not/ take into account all of the files fed into the build system, then using "build-pkgs -c -p mypkg" does not regenerate the *.dsc and the *.orig.tar.gz files for the package, which may cause older versions of the "untracked" source files to be used.
In these cases, the only way to ensure a fresh build is to remove the *.dsc file from "/localdisk/
I have prepared a fix for this shortcoming, which is more easily triggered with some of my kernel packaging changes. This is why I am opening this bug.
Severity
--------
Minor: This is a build issue affecting packages whose meta_data.yaml files have a "dl_hook" entry and whose input files are only partially covered by meta_data.yaml.
I did not acquire the number of packages matching this criteria, but my thinking is that this is a minor issue, especially as the official nightly builds are always non-incremental builds (i.e., nightly builds always build from scratch and hence are not affected by this issue).
Note
----
I already have a patch for this, and I am opening this bug report as a placeholder to refer to. This is why this template is incomplete.
Changed in starlingx: | |
assignee: | nobody → M. Vefa Bicakci (vbicakci) |
description: | updated |
Changed in starlingx: | |
status: | New → In Progress |
Changed in starlingx: | |
importance: | Undecided → Low |
tags: | added: stx.8.0 stx.build stx.debian |
Reviewed: https:/ /review. opendev. org/c/starlingx /root/+ /855389 /opendev. org/starlingx/ root/commit/ d7c5e1c8ba6f787 1fe7e40ce901805 f159e0af8d
Committed: https:/
Submitter: "Zuul (22348)"
Branch: master
commit d7c5e1c8ba6f787 1fe7e40ce901805 f159e0af8d
Author: M. Vefa Bicakci <email address hidden>
Date: Fri Aug 26 18:37:53 2022 +0000
stx: build-pkgs: Ensure that -c rebuilds packages
Prior to this commit, the "build-pkgs -c -p <pkg>" command would not
rebuild the DSC file and the orig.tar.gz archive of packages (especially
those that use the "dl_hook" build method) in case the corresponding
input files tracked by meta_data.yaml had not changed.
This appears expected at first, but there are use cases which require
regenerating the DSC file and the orig.tar.gz archive. For example, the
dl_hook script may make use of directories and/or files that are not
tracked by the meta_data.yaml file, and it may not be viable to track
such directories and files using meta_data.yaml.
As a supporting example, calculating the MD5 hash of all of the files in
a StarlingX kernel git repository (after local changes to optimize
memory utilization of build-pkgs) takes about 4.5 minutes in the
author's build VM, which makes it unviable to track changes in the
kernel repository using meta_data.yaml due to the excessive time added
to most build-pkgs invocations.
Hence, this commit makes build-pkgs regenerate DSC files and orig.tar.gz
archives when the "-c" option is in use.
Note that the "-c" option sets the self.attrs['avoid'] member flag of
the BuildController class to False, which is checked by the change made
in this commit to skip the input file change detection code, which
allows forcibly regenerating the aforementioned build artifacts.
Verification
- 'build-pkgs -c -p linux' regenerates the *.dsc an *.orig.tar.gz files
as expected when used with (currently unpublished) packaging changes
that make use of kernel git repository clones instead of downloaded
tar archive snapshots.
- No adverse effects were observed with incremental builds (e.g.,
"build-pkgs -c -b std,rt -p linux,linux-rt") and builds from scratch
(e.g., "build-pkgs -c -a") as well as incremental builds carried out
during the author's day-to-day work in the past three business days.
Closes-Bug: 1988348 a8278ed54acb6e7 492122c02e7
Change-Id: Ie186aee5ba7391
Signed-off-by: M. Vefa Bicakci <email address hidden>