Ubuntu 16.10 installation fails after the 'Select and install software' step

[bugproxy]

---uname output---
4.8.0-32-generic

Machine Type = 8286-42A

---boot type---
Network boot

---bootloader---
grub

---Kernel cmdline used to launch install---
NIM server: /nim/build_net/tools/pushit beesnest.aus.stglabs.ibm.com linuxUbuntu16.04_GA noprompt default Auto Auto no chrp_mp mksysb

---Bootloader protocol---
unsure

---Install repository type---
Local repository

---Install repository Location---
Linux install images are in /nim/install/linuxUbuntu16.04_GA on iofnim.aus.stglabs.ibm.com

---Point of failure---
Other failure during installation (stage 1)

Several attempts to install 16.10 fail after the 'Select and install software' step (details below).

latest 16.10 (4.8.0-32-generic) fails to install,
older 16.04 (4.4.0-31-generic) installs successfully **on the same Hardware**

==========================================================================
16.10 FAILED INSTALLATION

The following partitions are going to be formatted:
partition #2 of SCSI2 (2,9,0) (sdaa) as ext4
partition #3 of SCSI2 (2,9,0) (sdaa) as swap
Write the changes to disks?
1: Yes 2: No [*]
Prompt: '?' for help, default=2> 1

Partitions formatting ... 33%
Partitions formatting

OF stdout device is: /vdevice/[e-mail] 35.19MiB 100% 2.09MiB/s ]
Preparing to boot Linux version 4.8.0-22-generic (buildd@bos01-ppc64el-027) (gcc vers
ion 6.2.0 20161005 (Ubuntu 6.2.0-5ubuntu12) ) #24-Ubuntu SMP Sat Oct 8 09:14:41 UTC 2
016 (Ubuntu 4.8.0-22.24-generic 4.8.0)
Detected machine type: 0000000000000101
Max number of cores passed to firmware: 256 (NR_CPUS = 2048)
Calling ibm,client-architecture-support...

TFTP BOOT ---------------------------------------------------
Server IP.....................9.3.80.16
Client IP.....................9.3.139.141
Gateway IP....................9.3.139.129
Subnet Mask...................255.255.255.128
( 1 ) Filename................./ubuntu-installer/ppc64el/powerpc-ieee1275/core.elf
TFTP Retries..................5
Block Size....................512
FINAL PACKET COUNT = 301
FINAL FILE SIZE = 153628 BYTES

                     GNU GRUB version 2.02~beta2-36ubuntu8

 +----------------------------------------------------------------------------+
 |*Install |
 | Rescue |
 | |
 | |
 | |
 | |
 | |
 | |
 | |
 | |
 | |
 | |
 +----------------------------------------------------------------------------+

      Use the ^ and v keys to select which entry is highlighted.
      Press enter to boot the selected OS, `e' to edit the commands
      before booting or `c' for a command-line.
   The highlighted entry will be executed automatically in 1s.

If you need to use a HTTP proxy to access the outside world, enter the proxy
information here. Otherwise, leave this blank.

The proxy information should be given in the standard form of
"http://[[user][:pass]@]host[:port]/".
HTTP proxy information (blank for none):
Prompt: '?' for help, default=software.linux.c> software.linux.ibm.com

Checking the Ubuntu archive mirror
Loading additional components ... 10%... 20%... 30%... 40%... 50%... 60%... 70%... 8
0%... 90%... 100%
Generating SSH host key
Detecting disks and all other hardware ... 95%... 100%
Loading additional components ... 12%... 25%... 37%... 50%... 62%... 75%... 87%... 1
00%
Starting up the partitioner ... 13%... 21%... 30%... 43%

... 91%... 100%
Partition disks
---------------

The installer can guide you through partitioning a disk (using different
standard schemes) or, if you prefer, you can do it manually. With guided
partitioning you will still have a chance later to review and customise the
results.

If you choose guided partitioning for an entire disk, you will next be asked
which disk should be used.
Partitioning method:
  1: Guided - resize SCSI1 (2,4,0), partition #2 (sde) and use freed space,
  2: Guided - resize SCSI2 (2,4,0), partition #2 (sdv) and use freed space,
  3: Guided - reuse partition, SCSI1 (2,4,0), partition #2 (sde),
  4: Guided - reuse partition, SCSI2 (2,4,0), partition #2 (sdv),
  5: Guided - use entire disk,
  6: Guided - use entire partition, SCSI1 (2,4,0), partition #2 (sde),
  7: Guided - use entire partition, SCSI2 (2,4,0), partition #2 (sdv),
  8: Guided - use the largest continuous free space,
  9: Guided - use entire disk and set up LVM,
 10: Guided - use entire disk and set up encrypted LVM,
 11: Manual,
Prompt: '?' for help> 5

---------------

The installer can guide you through partitioning a disk (using different
standard schemes) or, if you prefer, you can do it manually. With guided
partitioning you will still have a chance later to review and customise the
results.

If you choose guided partitioning for an entire disk, you will next be asked
which disk should be used.
Partitioning method:
  1: Guided - resize SCSI1 (2,4,0), partition #2 (sde) and use freed space,
  2: Guided - resize SCSI2 (2,4,0), partition #2 (sdv) and use freed space,
  3: Guided - reuse partition, SCSI1 (2,4,0), partition #2 (sde),
  4: Guided - reuse partition, SCSI2 (2,4,0), partition #2 (sdv),
  5: Guided - use entire disk,
  6: Guided - use entire partition, SCSI1 (2,4,0), partition #2 (sde),
  7: Guided - use entire partition, SCSI2 (2,4,0), partition #2 (sdv),
  8: Guided - use the largest continuous free space,
  9: Guided - use entire disk and set up LVM,
 10: Guided - use entire disk and set up encrypted LVM,
 11: Manual,
Prompt: '?' for help> 5

Note that all data on the disk you select will be erased, but not before you

have confirmed that you really want to make the changes.
Select disk to partition:
  1: SCSI1 (2,0,0) (sda) - 2.3 TB IBM IPR-5 69002400,
  2: SCSI2 (2,9,0) (sdaa) - 177.8 GB IBM IPR-0 69000800,
  3: SCSI2 (2,10,0) (sdab) - 177.8 GB IBM IPR-0 69000800,
  4: SCSI2 (2,11,0) (sdac) - 1.1 TB IBM IPR-0 69002400,
  5: SCSI2 (2,12,0) (sdad) - 177.8 GB IBM IPR-0 69000800,
  6: SCSI2 (2,13,0) (sdae) - 177.8 GB IBM IPR-0 69000800,
  7: SCSI2 (2,14,0) (sdaf) - 177.8 GB IBM IPR-0 69000800,
  8: SCSI2 (2,15,0) (sdag) - 177.8 GB IBM IPR-0 69000800,
  9: SCSI2 (2,16,0) (sdah) - 177.8 GB IBM IPR-0 69000800,
 10: SCSI1 (2,1,0) (sdb) - 1.1 TB IBM IPR-0 69000500,
 11: SCSI1 (2,2,0) (sdc) - 1.1 TB IBM IPR-0 69000500,
 12: SCSI1 (2,3,0) (sdd) - 2.6 TB IBM IPR-5T2 5EC45000,
 13: SCSI1 (2,4,0) (sde) - 177.8 GB IBM IPR-0 69000800,
 14: SCSI1 (2,5,0) (sdf) - 177.8 GB IBM IPR-0 69000800,
 15: SCSI1 (2,6,0) (sdg) - 1.3 TB IBM IPR-10T25EC45000,
 16: SCSI1 (2,7,0) (sdh) - 177.8 GB IBM IPR-0 5EC45000,
 17: SCSI1 (2,8,0) (sdi) - 1.1 TB IBM IPR-0 69002400,
 18: SCSI1 (2,9,0) (sdj) - 177.8 GB IBM IPR-0 69000800,
 19: SCSI1 (2,10,0) (sdk) - 177.8 GB IBM IPR-0 69000800,
 20: SCSI1 (2,11,0) (sdl) - 1.1 TB IBM IPR-0 69002400,
Next choices are available with '+'
Prompt: '?' for help> 2

Guided partitioning ... 20%... 40%... 60%... 80%
If you continue, the changes listed below will be written to the disks.
Otherwise, you will be able to make further changes manually.

The partition tables of the following devices are changed:
   SCSI2 (2,9,0) (sdaa)

The following partitions are going to be formatted:
   partition #2 of SCSI2 (2,9,0) (sdaa) as ext4
   partition #3 of SCSI2 (2,9,0) (sdaa) as swap
Write the changes to disks?
  1: Yes 2: No [*]
Prompt: '?' for help, default=2> 1

Partitions formatting ... 33%
Partitions formatting
Installing the base system ... 17%... 20%... 30%... 40%... 50%... 60%... 70%... 83%.
.. 90%
Install the base system
-----------------------

!! ERROR: Unable to install the selected kernel

An error was returned while trying to install the kernel into the target
system.

Kernel package: 'linux-generic'.

Check /var/log/syslog or see virtual console 4 for the details.
[Press enter to continue]

!! ERROR: Installation step failed

An installation step failed. You can try to run the failing item again from the
menu, or skip it and choose something else. The failing step is: Install the
base system
[Press enter to continue]

hardware summary shows -

lsmod: usb_storage 74715 0
df: Filesystem 1K-blocks Used Available Use% Mounted on
df: none 2089472 3392 2086080 0% /run
df: devtmpfs 10393344 0 10393344 0% /dev
df: /dev/sdaa2 162895708 619660 153931728 0% /target
free: total used free shared buffers
free: Mem: 20894272 2906944 17987328 256704 243136
free: -/+ buffers: 2663808 18230464
free: Swap: 7069632 0 7069632
dmraid -s: ERROR: unsupported sector size 4096 on /dev/sdb.
dmraid -s: ERROR: unsupported sector size 4096 on /dev/sds.
dmraid -s: ERROR: unsupported sector size 4096 on /dev/sdc.
dmraid -s: ERROR: unsupported sector size 4096 on /dev/sdt.
dmraid -s: ERROR: unsupported sector size 4096 on /dev/sda.
dmraid -s: ERROR: unsupported sector size 4096 on /dev/sdr.
dmraid -s: no raid disks
dmraid -r: ERROR: unsupported sector size 4096 on /dev/sdb.
dmraid -r: ERROR: unsupported sector size 4096 on /dev/sds.
dmraid -r: ERROR: unsupported sector size 4096 on /dev/sdc.
dmraid -r: ERROR: unsupported sector size 4096 on /dev/sdt.
dmraid -r: ERROR: unsupported sector size 4096 on /dev/sda.
dmraid -r: ERROR: unsupported sector size 4096 on /dev/sdr.
dmraid -r: no raid disks
dmsetup table: No devices found

Syslog shows

Jan 9 07:43:58 debootstrap: gpgv: Signature made Thu Oct 13 13:29:09 2016 UTC
Jan 9 07:43:58 debootstrap: gpgv: using DSA key
Jan 9 07:43:58 debootstrap: gpgv: Good signature from "Ubuntu Archive Automatic Signing Key <email address hidden>"
Jan 9 07:43:58 debootstrap: gpgv: Signature made Thu Oct 13 13:29:09 2016 UTC
Jan 9 07:43:58 debootstrap: gpgv: using RSA key
Jan 9 07:43:58 debootstrap: gpgv: Good signature from "Ubuntu Archive Automatic Signing Key (2012) <email address hidden>"
Jan 9 07:44:09 debootstrap: mknod: /target/dev/null: File exists
Jan 9 07:44:09 base-installer: error: exiting on error base-installer/debootstrap-failed
Jan 9 07:44:27 main-menu[373]: WARNING **: Configuring 'bootstrap-base' failed with error code 1
Jan 9 07:44:27 main-menu[373]: WARNING **: Menu item 'bootstrap-base' failed.

------------------------

[bugproxy (bugproxy)]

------- Comment (attachment only) From <email address hidden> 2017-01-09 04:46 EDT-------

[bugproxy (bugproxy)]

------- Comment (attachment only) From <email address hidden> 2017-01-09 04:46 EDT-------

[bugproxy (bugproxy)]

------- Comment (attachment only) From <email address hidden> 2017-01-09 04:47 EDT-------

[bugproxy (bugproxy)]

------- Comment (attachment only) From <email address hidden> 2017-01-09 04:46 EDT-------

[bugproxy (bugproxy)]

------- Comment (attachment only) From <email address hidden> 2017-01-09 04:47 EDT-------

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-01-19 00:54 EDT-------
Please note this issue is seen only with 16.10 version and they use following path for the install.

ports.ubuntu.com/ubuntu-ports/dists/yakkety-updates/main/installer-ppc64el/current/images/netboot/ netboot.tar.gz

Jan 17 11:16:24 in-target: 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Jan 17 11:16:24 pkgsel: installing additional packages
Jan 17 11:16:24 in-target: Reading package lists...
Jan 17 11:16:25 in-target:
Jan 17 11:16:25 in-target: Building dependency tree...
Jan 17 11:16:25 in-target:
Jan 17 11:16:25 in-target: Reading state information...
Jan 17 11:16:25 in-target:
Jan 17 11:16:25 in-target: Package libnss3-1d is not available, but is referred to by another package.
Jan 17 11:16:25 in-target: This may mean that the package is missing, has been obsoleted, or
Jan 17 11:16:25 in-target: is only available from another source
Jan 17 11:16:25 in-target:
Jan 17 11:16:25 in-target: E: Package 'libnss3-1d' has no installation candidate
Jan 17 11:16:25 main-menu[568]: WARNING **: Configuring 'pkgsel' failed with error code 100
Jan 17 11:16:25 main-menu[568]: WARNING **: Menu item 'pkgsel' failed.
Jan 17 11:16:25 main-menu[568]: INFO: Modifying debconf priority limit from 'medium' to 'low'
Jan 17 11:16:25 debconf: Setting debconf/priority to low
Jan 17 11:21:56 main-menu[568]: INFO: Menu item 'save-logs' selected

Issue is not seen with 17.04 Ubuntu version.

ports.ubuntu.com/ubuntu-ports/dists/zesty/main/installer-ppc64el/current/images/netboot/netboot.tar.gz

[Steve Langasek (vorlon)]

There are a number of errors in the syslog output like

Jan 9 08:59:04 base-installer: Could not open file /dev/null - open (13: Permission denied) Connection timed out Error writing to the file - write (9: Bad file descriptor) [IP: 91.189.91.11 80]

Perhaps there is a bit flip error in this particular image.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-02-07 03:38 EDT-------
Can you upload or point right image so that we can test. Currently we are using image in the path for testing.

ports.ubuntu.com/ubuntu-ports/dists/yakkety-updates/main/installer-ppc64el/current/images/netboot/ netboot.tar.gz

[Adam Conrad (adconrad)]

Jan 9 00:51:20 in-target: Err:1 http://us.ports.ubuntu.com/pub/ubuntu-ppc64el-repository yakkety/main ppc64el libnl-3-200 ppc64el 3.2.27-1
Jan 9 00:51:20 in-target: 404 Not Found [IP: 91.189.91.11 80]
Jan 9 00:51:27 in-target: E: Failed to fetch http://us.ports.ubuntu.com/pub/ubuntu-ppc64el-repository/pool/main/libn/libnl3/libnl-3-200_3.2.27-1_ppc64el.deb 404 Not Found [IP: 91.189.91.11 80]
Jan 9 00:51:27 in-target: E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
Jan 9 00:51:32 base-installer: error: exiting on error base-installer/kernel/failed-install

I have no idea what you've done to ask the installer to look for those paths on our ports mirrors, but those 404s are what's killing it, and it's certainly coming from your end, not ours.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-02-10 00:18 EDT-------
Can you point me the default path of libnl-3-200_3.2.27-1_ppc64el.deb in http://us.ports.ubuntu.com for ppc64el-repository yakkety release ?

[Steve Langasek (vorlon)]

The correct path to the Ubuntu repository is 'http://us.ports.ubuntu.com/'. There is no 'pub/ubuntu-ppc64el-repository' as part of the path.