compiled binaries don't work on arm64 64k pages kernel due to alignment

[Lorenzo Pieralisi]

Some binaries in 15.10 for aarch64 seem to be compiled with maxpagesize=4K which triggers issues when run on 64K pages
arm64 kernels (tested on all kernels back to 4.0). I spotted this while trying to boot an arm64 kernel with 64K pages enabled on 15.10 Ubuntu filesystem.

[Ubuntu Foundations Team Bug Bot (crichton)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. It seems that your bug report is not filed about a specific source package though, rather it is just filed against Ubuntu in general. It is important that bug reports be filed about source packages so that people interested in the package can find the bugs about it. You can find some hints about determining what package your bug might be about at https://wiki.ubuntu.com/Bugs/FindRightPackage. You might also ask for help in the #ubuntu-bugs irc channel on Freenode.

To change the source package that this bug is filed about visit https://bugs.launchpad.net/ubuntu/+bug/1520162/+editstatus and add the package name in the text box next to the word Package.

[This is an automated message. I apologize if it reached you inappropriately; please just reply to this message indicating so.]

[Martin Pitt (pitti)]

That's very vague -- can you please post a journal or at least a screenshot with the error? What does "trigger issues" mean? systemd does not have any special compiler option for arm64, in particular not a "maxpagesize" option -- this sounds much more like a toolchain issue?

[Lorenzo Pieralisi (lorenzo-pieralisi)]

The following splat is from an aarch64 4.4-rc2 kernel, compiled with 64K pages support, system can't reach
user space since systemd fails to execute (the same set-up works for 4K pages kernel).

Freeing unused kernel memory: 576K (fffffe0000830000 - fffffe00008c0000)
Starting init: /sbin/init exists but couldn't execute it (error -22)
Starting init: /etc/init exists but couldn't execute it (error -13)
Starting init: /bin/sh exists but couldn't execute it (error -14)
Kernel panic - not syncing: No working init found. Try passing init= option to kernel. See Linux Documentation/init.txt for guidance.
CPU: 2 PID: 1 Comm: init Not tainted 4.4.0-rc2+ #11
Hardware name: ARM Juno development board (r0) (DT)
Call trace:
[<fffffe0000096a90>] dump_backtrace+0x0/0x118
[<fffffe0000096bbc>] show_stack+0x14/0x20
[<fffffe000031660c>] dump_stack+0x94/0xd0
[<fffffe000014b65c>] panic+0xd4/0x21c
[<fffffe00005c0dec>] kernel_init+0xdc/0xe0
[<fffffe0000093980>] ret_from_fork+0x10/0x50
CPU1: stopping
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.4.0-rc2+ #11
Hardware name: ARM Juno development board (r0) (DT)
Call trace:
[<fffffe0000096a90>] dump_backtrace+0x0/0x118
[<fffffe0000096bbc>] show_stack+0x14/0x20
[<fffffe000031660c>] dump_stack+0x94/0xd0
[<fffffe000009bde0>] handle_IPI+0x190/0x1a0
[<fffffe000009047c>] gic_handle_irq+0x8c/0xa8
Exception stack(0xfffffe09401efdb0 to 0xfffffe09401efed0)
fda0: 00000000e6c29688 fffffe007ec54c00
fdc0: fffffe09401eff00 fffffe000049edf4 0000000080000045 0000000000000000
fde0: 0000000000000000 fffffe09401ec000 0000000000000038 0000000000000001
fe00: 0000000000000002 0000000000000000 001a76e700000000 00000000000005dc
fe20: fffffe09401a8090 fffffe09401efed0 0000000000000730 0000000000000000
fe40: 0000000000000000 0000000000000000 0000000000000003 00000000ffffff80
fe60: 0000000000000000 0000000000000000 fffffdfee27fa440 00000000e6c29688
fe80: fffffe007ec54c00 0000000000000000 0000000000000000 fffffe0000947938
fea0: 00000000e68d40a0 fffffe0000900000 fffffe09401ec000 fffffe007ec54c00
fec0: fffffe0000947878 fffffe09401eff00
[<fffffe00000934e8>] el1_irq+0x68/0xd8
[<fffffe000049eef0>] cpuidle_enter+0x18/0x20
[<fffffe00000f56f4>] call_cpuidle+0x24/0x50
[<fffffe00000f5864>] cpu_startup_entry+0x144/0x218
[<fffffe000009b8ac>] secondary_start_kernel+0x11c/0x140
[<000000008009073c>] 0x8009073c
CPU0: stopping
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.4.0-rc2+ #11
Hardware name: ARM Juno development board (r0) (DT)

[Lorenzo Pieralisi (lorenzo-pieralisi)]

And this is where I think the problem is, in particular the align field for the LOAD headers.

aarch64-linux-gnu-readelf -Wl /lib/systemd/systemd

Elf file type is DYN (Shared object file)
Entry point 0x2c8d0
There are 10 program headers, starting at offset 64

Program Headers:
  Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
  PHDR 0x000040 0x0000000000000040 0x0000000000000040 0x000230 0x000230 R 0x8
  INTERP 0x000270 0x0000000000000270 0x0000000000000270 0x00001b 0x00001b R 0x1
      [Requesting program interpreter: /lib/ld-linux-aarch64.so.1]
  LOAD 0x000000 0x0000000000000000 0x0000000000000000 0x120d5c 0x120d5c R E 0x1000
  LOAD 0x120db0 0x0000000000121db0 0x0000000000121db0 0x01faf0 0x020438 RW 0x1000
  DYNAMIC 0x13f150 0x0000000000140150 0x0000000000140150 0x0002b0 0x0002b0 RW 0x8
  NOTE 0x00028c 0x000000000000028c 0x000000000000028c 0x000044 0x000044 R 0x4
  GNU_EH_FRAME 0x120d54 0x0000000000120d54 0x0000000000120d54 0x000008 0x000008 R 0x4
  GNU_STACK 0x000000 0x0000000000000000 0x0000000000000000 0x000000 0x000000 RW 0
  TLS 0x120db0 0x0000000000121db0 0x0000000000121db0 0x000008 0x000084 R 0x8
  GNU_RELRO 0x120db0 0x0000000000121db0 0x0000000000121db0 0x01f250 0x01f250 RW 0x8

[Martin Pitt (pitti)]

I reassign that to binutils for now, which provides the linker. Looks like the alignment of the ELF sections need to be adjusted for 64k pages?

[Lorenzo Pieralisi (lorenzo-pieralisi)]

Yes, even though some of the executables are 64K aligned (eg /bin/bash and I can use that as init= without an issue), I do not know why that's the case or if I did something wrong while updating packages.

[Sudeep Holla (sudeep.holla)]

I too hit the same issue recently. It works fine in 15.04, so seems like a regression to me in 15.10.
Any suggestions or workaround ?

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Ali (asaidi)]

I just ran into this as well. Interestingly the only package I can find that has the issue is systemd. It looks like systemd was linked with gold and nothing else I have installed is.

This issues was fixed in upstream bin-utils by the following change:
https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commit;h=3b0357dadaf2366cc418ec725dec55b1cea1a2e7

[dann frazier (dannf)]

I cannot reproduce this issue in xenial:

dannf@mustang:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu Xenial Xerus (development branch)
Release: 16.04
Codename: xenial
dannf@mustang:~$ getconf PAGE_SIZE
65536
dannf@mustang:~$

[Matthias Klose (doko)]

apparently fixed