Debian/stx: build-pkgs: Quieten *.orig.tar.gz gen.
run_shell_cmd uses stdout=subprocess.PIPE and stderr=subprocess.PIPE to
capture the standard output and standard error of the command being
executed, and afterwards dumps all of it to the console and the log
file.
This is problematic for large software packages with many files (such as
the kernel) for two reasons:
- build-pkgs essentially caches the output of tar, increasing memory
utilization when tar generates large amount of output. The memory
utilization was admittedly not measured, but the builder.log file's
contents offer a hint: Excluding the timestamp prefix strings, tar
generates about 3,953,054 bytes (i.e., ~3.77 MiB) of output for the
standard kernel package. Afterwards, the run_shell_cmd function splits
this large string into lines and prints out each line in the string
one by one.
- build-pkgs floods the console with arguably unnecessary information.
For each kernel package, the output takes about 13 seconds to emit to
the console and consists of about 77640 lines:
Reviewed: https:/ /review. opendev. org/c/starlingx /root/+ /858433 /opendev. org/starlingx/ root/commit/ 230afa561bb1785 7f2aea53f164a57 efdc3b02e8
Committed: https:/
Submitter: "Zuul (22348)"
Branch: master
commit 230afa561bb1785 7f2aea53f164a57 efdc3b02e8
Author: M. Vefa Bicakci <email address hidden>
Date: Thu Sep 15 16:48:47 2022 +0000
Debian/stx: build-pkgs: Quieten *.orig.tar.gz gen.
run_shell_cmd uses stdout= subprocess. PIPE and stderr= subprocess. PIPE to
capture the standard output and standard error of the command being
executed, and afterwards dumps all of it to the console and the log
file.
This is problematic for large software packages with many files (such as
the kernel) for two reasons:
- build-pkgs essentially caches the output of tar, increasing memory
utilization when tar generates large amount of output. The memory
utilization was admittedly not measured, but the builder.log file's
contents offer a hint: Excluding the timestamp prefix strings, tar
generates about 3,953,054 bytes (i.e., ~3.77 MiB) of output for the
standard kernel package. Afterwards, the run_shell_cmd function splits
this large string into lines and prints out each line in the string
one by one.
- build-pkgs floods the console with arguably unnecessary information.
For each kernel package, the output takes about 13 seconds to emit to
the console and consists of about 77640 lines:
2022-09-15 16:20:31,396 - debrepack - INFO: [ Run - \ 10.112. orig.tar. gz \
linux- 5.10.112" ] 10.112/ CREDITS
0001-kernel- std-Remove- the-old- changelog- file.patch
"cd .../std/linux; tar czvf linux_5.
2022-09-15 16:21:09,950 - debrepack - DEBUG: linux-5.10.112/
...
2022-09-15 16:21:22,870 - debrepack - DEBUG: linux-5.
2022-09-15 16:21:22,878 - debrepack - INFO: Apply deb patch: \
Hence, this commit quietens the generation of orig.tar.gz archives.
Verification
- 'build-pkgs -b rt,std -c -a' is successful.
Closes-Bug: 1990175 bba542f1e47dc36 882d39a5996
Change-Id: I9ef2adbb85930f
Signed-off-by: M. Vefa Bicakci <email address hidden>