build-tools: debian: build stuck in chroot creation

[Davlet Panech]

Brief Description
-----------------
Build sometimes gets stuck in chroot creation. This seems to happen after schroot calls apt, which fails to download one of the packages due to a network error.

Severity
--------
Major

Steps to Reproduce
------------------
Delete your old chroots, then try to build a package.

Expected Behavior
------------------
Build succeeds.

Actual Behavior
----------------
Sometimes build fails.

Reproducibility
---------------
Seen once.

System Configuration
--------------------
N/A

Branch/Pull Time/Commit
-----------------------
master/2022-08-17

Last Pass
---------
Unknown

Timestamp/Logs
--------------

From: http://mirror.starlingx.cengn.ca/mirror/starlingx/master/debian/monolithic/20220816T140236Z/logs/build-packages.log.txt

[2022-08-16T14:26:14.735Z] [...] { build-pkgs -a -b std,rt ; }
[...]
[2022-08-16T14:26:30.949Z] 2022-08-16 14:26:28,831 - debcontroller - DEBUG: Waiting for log file /localdisk/pkgbuilder/jenkins/debian-master/chroot.log
[...]
[2022-08-16T14:36:52.204Z] I: Retrieving InRelease
[2022-08-16T14:36:52.204Z] I: Checking Release signature
[2022-08-16T14:36:52.204Z] I: Valid Release signature (key id A4285295FC7B1A81600062A9605C66F00D6C9793)
[2022-08-16T14:36:52.204Z] I: Retrieving Packages
[2022-08-16T14:36:52.204Z] I: Validating Packages
[2022-08-16T14:36:52.204Z] I: Resolving dependencies of required packages...
[2022-08-16T14:36:52.204Z] I: Resolving dependencies of base packages...
[2022-08-16T14:36:52.204Z] I: Checking component main on http://snapshot.debian.org/archive/debian/20220331T000000Z...
[2022-08-16T14:36:52.204Z] I: Retrieving libacl1 2.2.53-10
[2022-08-16T14:36:52.204Z] I: Validating libacl1 2.2.53-10
[2022-08-16T14:36:52.204Z] I: Retrieving adduser 3.118
[2022-08-16T14:36:52.204Z] I: Validating adduser 3.118
[...]
[2022-08-16T14:36:52.204Z] W: Couldn't download package libc-dev-bin (ver 2.31-13+deb11u3 arch amd64) at http://snapshot.debian.org/archive/debian/20220331T000000Z/pool/main/g/glibc/libc-dev-bin_2.31-13+deb11u3_amd64.deb
[...]
(stuck forever, had to be terminated)

Test Activity
-------------
N/A

Workaround
----------
Re-try the build

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/root/+/854342

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/root/+/854342
Committed: https://opendev.org/starlingx/root/commit/2476b0e5038fb02ba8aa338258c0405c7bbbd0a5
Submitter: "Zuul (22348)"
Branch: master

commit 2476b0e5038fb02ba8aa338258c0405c7bbbd0a5
Author: Wentao Zhang <email address hidden>
Date: Wed Aug 24 11:24:18 2022 +0800

    debian: build-tools: build stuck in chroot creation

    Build-pkgs only catches the successful messages of chroot
    creating in the chroot.log and returns, it does not catch
    the failed or error messages in this log file. This will
    cause the build-pkgs to get stuck and not to return from
    waiting to show this log file if fails to creat chroot.
    To fix this we make build-pkgs to quit after catching any
    error message with 'E: ' prefix in chroot.log.

    Test Plan:
    Pass: Created a local repository and removed 'libc-dev-bin'
          from that without repository updating. Set this
          repository to chroot creating. Remove all the old
          chroots, then run 'build-pkgs -c -p <package name>'

    Closes-bug: 1986864
    Signed-off-by: Wentao Zhang <email address hidden>
    Change-Id: Ia0ed99a9ccfbe96b169c5ebf4887745d6321bb2d