| 2022-09-19 16:54:31 |
M. Vefa Bicakci |
description |
The Debian build system's helper function "run_shell_cmd" in "build-tools/stx/utils.py" uses stdout=subprocess.PIPE and stderr=subprocess.PIPE to capture the standard output and standard error of the command being executed, and afterwards dumps all of it to the console and the log file. This makes it problematic to log the output of "tar" when generating orig.tar.gz archives for large software packages with many files (such as the kernel) for two reasons:
- build-pkgs essentially caches the output of tar, increasing memory utilization when tar generates large amount of output. The memory utilization was admittedly not measured, but the builder.log file's contents offer a hint: Excluding the timestamp prefix strings, tar generates about 3,953,054 bytes (i.e., ~3.77 MiB) of output for the standard kernel package. Afterwards, the run_shell_cmd function splits this large string into lines and prints out each line in the string one by one.
- build-pkgs floods the console with arguably unnecessary information. For each kernel package, the output takes about 13 seconds to emit to the console and consists of about 77640 lines:
2022-09-15 16:20:31,396 - debrepack - INFO: [ Run - "cd .../std/linux; tar czvf linux_5.10.112.orig.tar.gz linux-5.10.112" ]
2022-09-15 16:21:09,950 - debrepack - DEBUG: linux-5.10.112/
...
2022-09-15 16:21:22,870 - debrepack - DEBUG: linux-5.10.112/CREDITS
2022-09-15 16:21:22,878 - debrepack - INFO: Apply deb patch: 0001-kernel-std-Remove-the-old-changelog-file.patch
This issue can be resolved by quietening "tar" by removing its "v" (verbose) option from the command line used for generating orig.tar.gz archives in the following file and function:
File "build-tools/stx/debrepack.py":
def create_orig_tarball(self):
# ...
run_shell_cmd('cd %s; tar czvf %s %s' % (self.pkginfo["packdir"], origtargz, srcname), self.logger)
Severity
--------
Minor: This is a very minor issue; it only affects the amount of output generated by the build system.
Note
----
This bug report exists just so that I can publish/propose a commit fixing this issue. I did not fill out the full bug report template on purpose. |
Brief Description
-----------------
The Debian build system's helper function "run_shell_cmd" in "build-tools/stx/utils.py" uses stdout=subprocess.PIPE and stderr=subprocess.PIPE to capture the standard output and standard error of the command being executed, and afterwards dumps all of it to the console and the log file. This makes it problematic to log the output of "tar" when generating orig.tar.gz archives for large software packages with many files (such as the kernel) for two reasons:
- build-pkgs essentially caches the output of tar, increasing memory utilization when tar generates large amount of output. The memory utilization was admittedly not measured, but the builder.log file's contents offer a hint: Excluding the timestamp prefix strings, tar generates about 3,953,054 bytes (i.e., ~3.77 MiB) of output for the standard kernel package. Afterwards, the run_shell_cmd function splits this large string into lines and prints out each line in the string one by one.
- build-pkgs floods the console with arguably unnecessary information. For each kernel package, the output takes about 13 seconds to emit to the console and consists of about 77640 lines:
2022-09-15 16:20:31,396 - debrepack - INFO: [ Run - "cd .../std/linux; tar czvf linux_5.10.112.orig.tar.gz linux-5.10.112" ]
2022-09-15 16:21:09,950 - debrepack - DEBUG: linux-5.10.112/
...
2022-09-15 16:21:22,870 - debrepack - DEBUG: linux-5.10.112/CREDITS
2022-09-15 16:21:22,878 - debrepack - INFO: Apply deb patch: 0001-kernel-std-Remove-the-old-changelog-file.patch
This issue can be resolved by quietening "tar" by removing its "v" (verbose) option from the command line used for generating orig.tar.gz archives in the following file and function:
File "build-tools/stx/debrepack.py":
def create_orig_tarball(self):
# ...
run_shell_cmd('cd %s; tar czvf %s %s' % (self.pkginfo["packdir"], origtargz, srcname), self.logger)
Severity
--------
Minor: This is a very minor issue; it only affects the amount of output generated by the build system.
Note
----
This bug report exists just so that I can publish/propose a commit fixing this issue. I did not fill out the full bug report template on purpose. |
|
