Unable to install ubuntu1804 build with Debootstrap warning on witherspoon system

[bugproxy]

Problem Description:
===================
Ubuntu1804 installer shows Debootstrap warning messages and not able to proceed with installation

Steps to re-create:
==================
> on Witherspoon test system, tried to install ubuntu1804 using netboot install.

> Selected mirror path:http://10.xx.11.31:3128

  ???????????????? [!] Choose a mirror of the Ubuntu archive ????????????????
  ? ?
  ? Please select an Ubuntu archive mirror. You should use a mirror in ?
  ? your country or region if you do not know which mirror has the best ?
  ? Internet connection to you. ?
  ? ?
  ? Usually, <your country code>.archive.ubuntu.com is a good choice. ?
  ? ?
  ? Ubuntu archive mirror: ?
  ? ?
  ? us.ports.ubuntu.com ?
  ? ?
  ? <Go Back> ?
  ? ?
  ???????????????????????????????????????????????????????????????????????????

> Able to select the disks

  ?????????????????????????? [!!] Partition disks ???????????????????????????
  ? ?
  ? Note that all data on the disk you select will be erased, but not ?
  ? before you have confirmed that you really want to make the changes. ?
  ? ?
  ? Select disk to partition: ?
  ? ?
  ? /dev/nvme0n1 - 409.6 GB Unknown ?
  ? SCSI1 (0,0,0) (sda) - 3.8 TB ATA MTFDDAK3T8TCB ?
  ? SCSI2 (0,0,0) (sdb) - 3.8 TB ATA MTFDDAK3T8TCB ?
  ? ?
  ? <Go Back> ?
  ? ?
  ???????????????????????????????????????????????????????????????????????????

> Started installing base system

  ??????????????????????? Installing the base system ????????????????????????
  ? ?
  ? 6% ?
  ? ?
  ? Retrieving libc6... ?
  ? ?
  ???????????????????????????????????????????????????????????????????????????

> Then seen Debootstrap warning messages as below

      ??????????????????? [!!] Install the base system ???????????????????
  ????? ? ??
  ? ? Debootstrap warning ? ?
  ? ? Warning: Failure trying to run: chroot /target dpkg-deb -f ? ?
  ? ? /var/cache/apt/archives/dpkg_1.19.0.4ubuntu1_ppc64el.deb Version ? ?
  ? Ex? ? ?
  ? ? <Go Back> <Continue> ? ?
  ????? ? ??
      ????????????????????????????????????????????????????????????????????

  ??????????????????????? [!!] Install the base system ??????????????????????
  ? ? ? ?
  ? ? Debootstrap warning ? ?
  ? ? Warning: See the log for details ? ?
  ? Extracting zlib1g.? ? ?
  ? ? <Go Back> <Continue> ? ?
  ????????????????????? ? ??????????????????
                      ????????????????????????????????????

     ???????????????????? [!!] Install the base system ????????????????????
  ???? ? ?
  ? ? Debootstrap warning ? ?
  ? ? Warning: Failure trying to run: chroot /target mount -t proc proc ? ?
  ? ? /proc ? ?
  ? E? ? ?
  ? ? <Go Back> <Continue> ? ?
  ???? ? ?
     ??????????????????????????????????????????????????????????????????????

      ?????????????????? [!!] Install the base system ???????????????????
  ????? ? ???
  ? ? Base system installation error ? ?
  ? ? The debootstrap program exited with an error (return value 1). ? ?
  ? ? ? ?
  ? Ex? Check /var/log/syslog or see virtual console 4 for the details. ? ?
  ? ? ? ?
  ????? <Go Back> <Continue> ? ???
      ? ?
      ???????????????????????????????????????????????????????????????????

      ?????????????????? [!!] Install the base system ???????????????????
      ? ?
      ? Failed to install the base system ?
      ? The base system installation into /target/ failed. ?
      ? ?
      ? Check /var/log/syslog or see virtual console 4 for the details. ?
      ? ?
      ? <Go Back> <Continue> ?
      ? ?
      ???????????????????????????????????????????????????????????????????

     ??????????????????? [!!] Install the base system ????????????????????
     ? ?
     ? Installation step failed ?
     ? An installation step failed. You can try to run the failing item ?
     ? again from the menu, or skip it and choose something else. The ?
     ? failing step is: Install the base system ?
     ? ?
     ? <Continue> ?
     ? ?
     ???????????????

> Unable to proceed with ubutnu1804 installation with debootstrap warning messages from installer.

> Attached /var/log/syslog messages from installer

== Comment: #3 - Vaishnavi Bhat <email address hidden> - 2017-12-11 05:19:04 ==
Seeing the following debootstrap messages from the /var/log/syslog:

Dec 8 18:12:33 debootstrap: tar: skipping unsafe symlink to '../libuuid1/changelog.Debian.gz' in archive, set EXTRACT_UNSAFE_SYMLINKS=1 to extract
Dec 8 18:12:33 debootstrap: tar: skipping unsafe symlink to '../libc6/changelog.Debian.gz' in archive, set EXTRACT_UNSAFE_SYMLINKS=1 to extract
Dec 8 18:12:33 debootstrap: tar: skipping unsafe symlink to '/lib/powerpc64le-linux-gnu/ld-2.26.so' in archive, set EXTRACT_UNSAFE_SYMLINKS=1 to extract
Dec 8 18:12:33 debootstrap: tar: skipping unsafe symlink to '../libuuid1/changelog.Debian.gz' in archive, set EXTRACT_UNSAFE_SYMLINKS=1 to extract
Dec 8 18:12:33 debootstrap: tar: skipping unsafe symlink to '../libuuid1/changelog.Debian.gz' in archive, set EXTRACT_UNSAFE_SYMLINKS=1 to extract
Dec 8 18:12:33 debootstrap: tar: skipping unsafe symlink to '../libpam0g/NEWS.Debian.gz' in archive, set EXTRACT_UNSAFE_SYMLINKS=1 to extract
Dec 8 18:12:33 debootstrap: tar: skipping unsafe symlink to '../libpam0g/NEWS.Debian.gz' in archive, set EXTRACT_UNSAFE_SYMLINKS=1 to extract
Dec 8 18:12:33 debootstrap: tar: skipping unsafe symlink to '../libpam0g/NEWS.Debian.gz' in archive, set EXTRACT_UNSAFE_SYMLINKS=1 to extract
Dec 8 18:12:34 debootstrap: tar: skipping unsafe symlink to '../libcomerr2/changelog.Debian.gz' in archive, set EXTRACT_UNSAFE_SYMLINKS=1 to extract
Dec 8 18:12:34 debootstrap: tar: skipping unsafe symlink to '../libmount1/changelog.Debian.gz' in archive, set EXTRACT_UNSAFE_SYMLINKS=1 to extract
Dec 8 18:12:34 debootstrap: tar: skipping unsafe symlink to '../libtinfo5/changelog.Debian.gz' in archive, set EXTRACT_UNSAFE_SYMLINKS=1 to extract
Dec 8 18:12:34 debootstrap: tar: skipping unsafe symlink to '../libprocps6/NEWS.Debian.gz' in archive, set EXTRACT_UNSAFE_SYMLINKS=1 to extract
Dec 8 18:12:34 debootstrap: tar: skipping unsafe symlink to '/sbin/killall5' in archive, set EXTRACT_UNSAFE_SYMLINKS=1 to extract
Dec 8 18:12:34 debootstrap: tar: skipping unsafe symlink to '../libblkid1/changelog.Debian.gz' in archive, set EXTRACT_UNSAFE_SYMLINKS=1 to extract
Dec 8 18:12:34 debootstrap: chroot: can't execute 'dpkg-deb': No such file or directory
Dec 8 18:12:34 debootstrap: chroot: can't execute 'mount': No such file or directory
Dec 8 18:13:12 base-installer: error: exiting on error base-installer/debootstrap-failed
Dec 8 18:13:18 main-menu[1400]: WARNING **: Configuring 'bootstrap-base' failed with error code 1
Dec 8 18:13:18 main-menu[1400]: WARNING **: Menu item 'bootstrap-base' failed.
Dec 8 18:13:30 main-menu[1400]: INFO: Modifying debconf priority limit from 'high' to 'medium'
Dec 8 18:13:30 debconf: Setting debconf/priority to medium

[bugproxy (bugproxy)]

Default Comment by Bridge

[Steve Langasek (vorlon)]

This appears to be a regression introduced by busybox 1:1.27.2-1ubuntu4:
  * SECURITY UPDATE: directory traversal via tar symlink extraction
    - debian/patches/CVE-2011-5325-2.patch: do not extract unsafe symlinks
      unless env variable is set in archival/libarchive/Kbuild.src,
      archival/libarchive/data_extract_all.c,
      archival/libarchive/unsafe_symlink_target.c, archival/tar.c,
      coreutils/link.c, include/bb_archive.h, libbb/copy_file.c,
      testsuite/tar.tests.
    - CVE-2011-5325
[...]

 -- Marc Deslauriers <email address hidden> Fri, 24 Nov 2017 12:55:21 -0500

Marc, this patch seems to have not been included upstream in Debian, and it's definitely a behavior difference vs. the other tar implementations as used by debootstrap (i.e. GNU tar). Is this actually fixing a security vulnerability, or is it an attempt to mitigate future unknown vulnerabilities?

[Steve Langasek (vorlon)]

I'll also mention that while this is being reported on ppc64el, it's an architecture-independent problem and is keeping daily images from being promoted currently.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-12-12 11:57 EDT-------
Can i take whip system back for other testing.

Regards,
Indira

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-12-12 12:29 EDT-------
We hit the same issue on BostonLC as well. We are blocked on this in order to go forward.
Breno, do you have any workaround to bypass this issue ?

[Jb (jebsolutions)]

I have a work around for Lubuntu Alternate installer 64 Bit fix (same console installer as Ubuntu Server).

Might work on server.

https://bugs.launchpad.net/ubuntu/+source/debootstrap/+bug/1736309

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-12-14 12:05 EDT-------
Can we consider this fixed in the daily images already?

[Launchpad Janitor (janitor)]

This bug was fixed in the package busybox - 1:1.27.2-2ubuntu3

---------------
busybox (1:1.27.2-2ubuntu3) bionic; urgency=medium

  * debian/patches/CVE-2011-5325-2.patch: disable patch for now as the
    behaviour is relied upon by debootstrap. (LP: #1737662)

 -- Marc Deslauriers <email address hidden> Tue, 12 Dec 2017 12:58:01 -0500

[Steve Langasek (vorlon)]

On Thu, Dec 14, 2017 at 05:09:56PM -0000, bugproxy wrote:
> ------- Comment From <email address hidden> 2017-12-14 12:05 EDT-------
> Can we consider this fixed in the daily images already?

When you this is fixed, the daily images should again pass their smoketests
and you should see updated images at
http://cdimage.ubuntu.com/ubuntu-server/daily/current/

The fix has only just landed in the bionic release, so will take time to
publish and make it into a daily image.

[Ubuntu QA Website (ubuntuqa)]

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
http://iso.qa.ubuntu.com/qatracker/reports/bugs/1737662

[Steve Langasek (vorlon)]

This is not yet resolved because debian-installer needs rebuilt to pick the fixed busybox up into the d-i initramfs. I have uploaded this now.

[Launchpad Janitor (janitor)]

This bug was fixed in the package debian-installer - 20101020ubuntu526

---------------
debian-installer (20101020ubuntu526) bionic; urgency=medium

  * Rebuild with fixed busybox 1:1.27.2-2ubuntu3. LP: #1737662.

 -- Steve Langasek <email address hidden> Mon, 18 Dec 2017 17:06:18 -0800

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-12-19 01:56 EDT-------
Please update the bug once daily build picks up the fix for installer issue, we are waiting for the fix as both NV and KVM test environments are blocked with test execution.

Regards,
Indira

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-12-19 04:17 EDT-------
> Have tried todays build (19/12) and i see issue is not resolved. Means fix is not yet picked up by the ubuntu1804 daily build.

????????????????????? [!!] Install the base system ??????????????????????
?? ?
?? Debootstrap warning ?
?? Warning: ?
?? http://us.ports.ubuntu.com/ubuntu-ports/dists/bionic/main/binary-ppc6 ?
?? 4el/Packages.xz was corrupt ?
?? ?
?? <Go Back> <Continue> ?
?? ?
?????????????????????????????????????????????????????????????????????????
???????????????????? [!!] Install the base system ??????????????????????
?? ?
?? Debootstrap warning ?
?? Warning: ?
?? http://us.ports.ubuntu.com/ubuntu-ports/dists/bionic/main/binary-ppc6 ?
?? 4el/Packages.gz was corrupt ?
?? ?
?? <Go Back> <Continue> ?
?? ?
?????????????????????????????????????????????????????????????????????????

Regards,
Indira

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-12-19 06:22 EDT-------
> Retried again with today's build (19/12)and faced same debootstrap failure messages as below

?????????????????? [!!] Install the base system ???????????????????
? ?
? Failed to install the base system ?
? The base system installation into /target/ failed. ?
? ?
? Check /var/log/syslog or see virtual console 4 for the details. ?
? ?
? <Go Back> <Continue> ?
? ?
???????????????????????????????????????????????????????????????????

??????????????????? [!!] Install the base system ????????????????????
? ?
? Installation step failed ?
? An installation step failed. You can try to run the failing item ?
? again from the menu, or skip it and choose something else. The ?
? failing step is: Install the base system ?
? ?
? <Continue> ?
? ?
?????????????????????????????????????????????????????????????????????

Regards,
Indira

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-12-19 13:42 EDT-------
Hi Breno,

As discussed in ST, i still see debootstrap messages while installation and
attaching installer logs from /var/log/syslog file for below debootstrap messages
during installation.

??????????????????? [!!] Install the base system ???????????????????
????? ? ??
? ? Debootstrap warning ? ?
? ? Warning: Failure trying to run: chroot /target dpkg-deb -f ? ?
? ? /var/cache/apt/archives/dpkg_1.19.0.4ubuntu1_ppc64el.deb Version ? ?
? Ex? ? ?
? ? <Go Back> <Continue> ? ?
????? ? ??
????????????????????????????????????????????????????????????????????

11:56:32 PM:
??????????????????????? [!!] Install the base system ??????????????????????
? ? ? ?
? ? Debootstrap warning ? ?
? ? Warning: See the log for details ? ?
? Extracting zlib1g.? ? ?
? ? <Go Back> <Continue> ? ?
????????????????????? ? ??????????????????
????????????????????????????????????

11:56:38 PM:

???????????????????? [!!] Install the base system ????????????????????
???? ? ?
? ? Debootstrap warning ? ?
? ? Warning: Failure trying to run: chroot /target mount -t proc proc ? ?
? ? /proc ? ?
? E? ? ?
? ? <Go Back> <Continue> ? ?
???? ? ?
??????????????????????????????????????????????????????????????????????

11:56:50 PM:

?????????????????? [!!] Install the base system ???????????????????
????? ? ???
? ? Base system installation error ? ?
? ? The debootstrap program exited with an error (return value 1). ? ?
? ? ? ?
? Ex? Check /var/log/syslog or see virtual console 4 for the details. ? ?
? ? ? ?
????? <Go Back> <Continue> ? ???
? ?
???????????????????????????????????????????????????????????????????

Regards,
Indira

[bugproxy (bugproxy)]

------- Comment (attachment only) From <email address hidden> 2017-12-19 13:43 EDT-------

[Steve Langasek (vorlon)]

The fixed debian-installer missed inclusion in the 20171219 image build. There is a just-finished 20171219.2 build which uses the correct debian-installer version, and which will go through our testing shortly.

[Steve Langasek (vorlon)]

The 20171219.3 ppc64el image has passed smoketesting and will be promoted imminently to http://cdimage.ubuntu.com/ubuntu-server/daily/current/

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-12-21 01:06 EDT-------
Able to install ubunu1804 build which is out on 20/12 with ISO file successfully without any debootstrap messages. Issue fixed on 20/12 dated daily build (using ISO )

root@whip:/home/dev# uname -a
Linux whip 4.13.0-17-generic #20-Ubuntu SMP Mon Nov 6 10:03:08 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux
root@whip:/home/dev# uname -r
4.13.0-17-generic

Regards,
Indira

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-01-17 01:24 EDT-------
*** Bug 163644 has been marked as a duplicate of this bug. ***

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-01-18 01:58 EDT-------
*** Bug 163644 has been marked as a duplicate of this bug. ***

------- Comment From <email address hidden> 2018-01-18 01:59 EDT-------
Jan 17 15:22:47 main-menu[384]: INFO: Menu item 'bootstrap-base' selected
Jan 17 15:22:48 debootstrap: gpgv: Signature made Wed Jan 17 21:09:59 2018 UTC
Jan 17 15:22:48 debootstrap: gpgv: using RSA key
Jan 17 15:22:48 debootstrap: gpgv: Good signature from "Ubuntu Archive Automatic Signing Key (2012) <email address hidden>"
Jan 17 15:23:01 kernel: [ 1733.162901] sd 0:0:5:0: [sdf] tag#1 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
Jan 17 15:23:01 kernel: [ 1733.162924] sd 0:0:5:0: [sdf] tag#1 Sense Key : Illegal Request [current]
Jan 17 15:23:01 kernel: [ 1733.162929] sd 0:0:5:0: [sdf] tag#1 Add. Sense: Invalid field in cdb
Jan 17 15:23:01 kernel: [ 1733.162934] sd 0:0:5:0: [sdf] tag#1 CDB: Write(10) 2a 00 06 04 4a 48 00 03 f0 00
Jan 17 15:23:01 kernel: [ 1733.162947] print_req_error: critical target error, dev sdf, sector 100944456
Jan 17 15:23:01 kernel: [ 1733.172558] Aborting journal on device sdf2-8.
Jan 17 15:23:01 kernel: [ 1733.173420] EXT4-fs error (device sdf2): ext4_journal_check_start:60: Detected aborted journal
Jan 17 15:23:01 kernel: [ 1733.173424] EXT4-fs (sdf2): Remounting filesystem read-only
Jan 17 15:23:01 debootstrap: tar: write error: Read-only file system
Jan 17 15:23:05 main-menu[384]: (process:13638): mv: can't rename '/target/etc/fstab.orig': Read-only file system
Jan 17 15:23:05 main-menu[384]: WARNING **: Configuring 'bootstrap-base' failed with error code 1
Jan 17 15:23:05 main-menu[384]: WARNING **: Menu item 'bootstrap-base' failed.
Jan 17 15:23:06 main-menu[384]: INFO: Modifying debconf priority limit from 'high' to 'medium'
Jan 17 15:23:06 debconf: Setting debconf/priority to medium
Jan 17 15:23:11 main-menu[384]: INFO: Menu item 'di-utils-shell' selected

Continue to see the issue with latest iso available.
kernel 4.13.0-25-generic

[bugproxy (bugproxy)]

------- Comment (attachment only) From <email address hidden> 2018-01-18 02:00 EDT-------

[Steve Langasek (vorlon)]

The log in the last comment shows an error from the kernel driver for the disk, which is either a hardware or kernel bug. It is unrelated to the issue reported in this launchpad bug.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-02-28 06:05 EDT-------
*** Bug 165223 has been marked as a duplicate of this bug. ***