DEP8 failure in lunar

[Andreas Hasenack]

I don't know what's wrong with this package, or with the kernel package:

ubuntu@l-crash:~$ sudo crash -st /usr/lib/debug/boot/vmlinux-6.1.0-16-generic
WARNING: /usr/lib/debug/boot/vmlinux-6.1.0-16-generic
         and /proc/version do not match!

WARNING: /proc/version indicates kernel version: 6.1.0-16-generic

crash: please use the vmlinux file for that kernel version, or try using
       the System.map for that kernel version as an additional argument.

It is the same version:
ubuntu@l-crash:~$ uname -a
Linux l-crash 6.1.0-16-generic #16-Ubuntu SMP PREEMPT_DYNAMIC Fri Feb 24 14:37:30 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

$ dpkg -l|grep linux-
ii binutils-x86-64-linux-gnu 2.40-2ubuntu1
ii linux-base 4.5ubuntu9
ii linux-firmware 20230302.git60971a64-0ubuntu1
ii linux-headers-6.1.0-16 6.1.0-16.16
ii linux-headers-6.1.0-16-generic 6.1.0-16.16
ii linux-headers-generic 6.1.0.16.16
ii linux-image-6.1.0-16-generic 6.1.0-16.16
ii linux-image-6.1.0-16-generic-dbgsym 6.1.0-16.16
ii linux-image-generic 6.1.0.16.16
ii linux-image-unsigned-6.1.0-16-generic-dbgsym 6.1.0-16.16
ii linux-libc-dev:amd64 6.1.0-16.16
ii linux-modules-6.1.0-16-generic 6.1.0-16.16
ii linux-modules-extra-6.1.0-16-generic 6.1.0-16.16
ii util-linux-extra 2.38.1-4ubuntu1

$ apt-cache policy linux-image-unsigned-6.1.0-16-generic-dbgsym
linux-image-unsigned-6.1.0-16-generic-dbgsym:
  Installed: 6.1.0-16.16
  Candidate: 6.1.0-16.16
  Version table:
 *** 6.1.0-16.16 500
        500 http://ddebs.ubuntu.com lunar/main amd64 Packages
        100 /var/lib/dpkg/status

[Mauricio Faria de Oliveira (mfo)]

This is apparently a regression due to (ubuntu-only) kernel commit [1]
commit e9ec6b526c6a ("UBUNTU: SAUCE: (no-up) version: Implement version_signature proc file.")

It modifies the contents of `/proc/version` by appending a version signature
(available individually in `/proc/version_signature`, a new file in `/proc`),
which comes from the `linux_proc_banner` variable/format string.

However, it doesn't modify a related string in the `vmlinux` object file,
which comes from `linux_banner` (no `proc`); i.e., it's without signature.
(This might be on purpose.)

The crash utility, in live system form (without a core dump/memory image),
searches for the contents of `/proc/version` _within_ the output of the
`strings vmlinux` command, to check that `vmlinux` and running system match.
(See source in `crash/filesys.c:match_proc_version() -> match_file_string()`).

The issue then happens because the appended version signature cannot be
found within `vmlinux` (`linux_proc_banner` no longer matches `linux_banner`).

[1] https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/lunar/commit/?id=e9ec6b526c6ac1f4392430e008d9ff6c3c9eeee1

[Mauricio Faria de Oliveira (mfo)]

Note: this is not an issue in the more common, dumpfile form (w/ core dump),
since it does not check `/proc/version`.

1) live system form:

$ sudo crash -st ./usr/lib/debug/boot/vmlinux-6.1.0-16-generic
WARNING: ./usr/lib/debug/boot/vmlinux-6.1.0-16-generic
         and /proc/version do not match!

WARNING: /proc/version indicates kernel version: 6.1.0-16-generic

crash: please use the vmlinux file for that kernel version, or try using
       the System.map for that kernel version as an additional argument.

2) dumpfile form:

$ sudo crash -st ./usr/lib/debug/boot/vmlinux-6.1.0-16-generic /var/crash/202303131312/dump.202303131312
Mon Mar 13 13:12:36 UTC 2023

[Mauricio Faria de Oliveira (mfo)]

Some options/considerations to address this (need input from the kernel team),
given there might be impact to (other) userspace tools:

1) Make `linux_banner` like `linux_proc_banner`, i.e., append version signature.

Fixes this issue; expands regression potential to other userspace tools.)

Perhaps this impacts the goal of the patch?

And there's warning quotes on the source, for not doing this. [2]

2) Revert `linux_proc_banner` not to have version signature.

Fixes this issue; reduces regression potential on other userspace tools.

3) Patch `crash` to deal with this

FFixes this isse; keeps regression potential on other userspace tools.

...

[2]

diff --git a/init/version-timestamp.c b/init/version-timestamp.c
...
 /* FIXED STRINGS! Don't touch! */
 const char linux_banner[] =
  "Linux version " UTS_RELEASE " (" LINUX_COMPILE_BY "@"
- LINUX_COMPILE_HOST ") (" LINUX_COMPILER ") " UTS_VERSION "\n";
+ LINUX_COMPILE_HOST ") (" LINUX_COMPILER ") " UTS_VERSION
+#ifdef CONFIG_VERSION_SIGNATURE
+ " (" CONFIG_VERSION_SIGNATURE ")"
+#endif
+ "\n";

[Mauricio Faria de Oliveira (mfo)]

GDB session with crash from git.

(gdb) p pc.namelist
$11 = 0x7fffffffed7f "../usr/lib/debug/boot/vmlinux-6.1.0-16-generic"

(gdb) p kt.proc_version
$12 = "Linux version 6.1.0-16-generic (buildd@lcy02-amd64-014) (x86_64-linux-gnu-gcc-12 (Ubuntu 12.2.0-14ubuntu2) 12.2.0, GNU ld (GNU Binutils for Ubuntu) 2.40) #16-Ubuntu SMP PREEMPT_DYNAMIC Fri Feb 24 14:3"...

Breakpoint 6, match_file_string (filename=0x7fffffffed7f "../usr/lib/debug/boot/vmlinux-6.1.0-16-generic",
    string=0x5555560f0e20 <kernel_table+128> "Linux version 6.1.0-16-generic (buildd@lcy02-amd64-014) (x86_64-linux-gnu-gcc-12 (Ubuntu 12.2.0-14ubuntu2) 12.2.0, GNU ld (GNU Binutils for Ubuntu) 2.40) #16-Ubuntu SMP PREEMPT_DYNAMIC Fri Feb 24 14:3"...,
    buffer=buffer@entry=0x7fffffffdce0 "crash.o") at ./filesys.c:4314
4314 {
(gdb) l
...
4312 static int
4313 match_file_string(char *filename, char *string, char *buffer)
4314 {
4315 int found;
4316 char command[BUFSIZE];
4317 FILE *pipe;
4318
(gdb)
4319
4320 sprintf(command, "/usr/bin/strings %s", filename);
4321 if ((pipe = popen(command, "r")) == NULL) {
4322 error(INFO, "%s: %s\n", filename, strerror(errno));
4323 return FALSE;
4324 }
4325
4326 found = FALSE;
4327 while (fgets(buffer, BUFSIZE-1, pipe)) {
4328 if (strstr(buffer, string)) {
(gdb)
4329 found = TRUE;
4330 break;
4331 }
4332 }
4333 pclose(pipe);
4334
4335 return found;
4336 }
...

(gdb) printf "%s\n", string
Linux version 6.1.0-16-generic (buildd@lcy02-amd64-014) (x86_64-linux-gnu-gcc-12 (Ubuntu 12.2.0-14ubuntu2) 12.2.0, GNU ld (GNU Binutils for Ubuntu) 2.40) #16-Ubuntu SMP PREEMPT_DYNAMIC Fri Feb 24 14:37:30 UTC 2023 (Ubuntu 6.1.0-16.16-generic 6.1.6)

(gdb) shell

# cat /proc/version
Linux version 6.1.0-16-generic (buildd@lcy02-amd64-014) (x86_64-linux-gnu-gcc-12 (Ubuntu 12.2.0-14ubuntu2) 12.2.0, GNU ld (GNU Binutils for Ubuntu) 2.40) #16-Ubuntu SMP PREEMPT_DYNAMIC Fri Feb 24 14:37:30 UTC 2023 (Ubuntu 6.1.0-16.16-generic 6.1.6)

# cat /proc/version_signature
Ubuntu 6.1.0-16.16-generic 6.1.6

# strings ../usr/lib/debug/boot/vmlinux-6.1.0-16-generic | grep 'Linux version'
Linux version 6.1.0-16-generic (buildd@lcy02-amd64-014) (x86_64-linux-gnu-gcc-12 (Ubuntu 12.2.0-14ubuntu2) 12.2.0, GNU ld (GNU Binutils for Ubuntu) 2.40) #16-Ubuntu SMP PREEMPT_DYNAMIC
Linux version 6.1.0-16-generic (buildd@lcy02-amd64-014) (x86_64-linux-gnu-gcc-12 (Ubuntu 12.2.0-14ubuntu2) 12.2.0, GNU ld (GNU Binutils for Ubuntu) 2.40) #16-Ubuntu SMP PREEMPT_DYNAMIC Fri Feb 24 14:37:30 UTC 2023

[Andy Whitcroft (apw)]

This is caused by a patch miss-applying to the wrong variable, when the intended target moved to another file. We have applied a fix for the 6.2 kernel for this.

[Andy Whitcroft (apw)]

commit 73f1fdeed7d9f67d3eaac4719b9c48e130284d41 (HEAD -> cranky/master-next)
Author: Andy Whitcroft <email address hidden>
Date: Wed Mar 8 16:09:07 2023 +0000

    UBUNTU: SAUCE: (no-up) version: fix concatenation of CONFIG_VERSION_SIGNATURE to banner

    Signed-off-by: Andy Whitcroft <email address hidden>

diff --git a/init/version-timestamp.c b/init/version-timestamp.c
index 043cbf80a766..649275e25ff4 100644
--- a/init/version-timestamp.c
+++ b/init/version-timestamp.c
@@ -27,4 +27,8 @@ struct uts_namespace init_uts_ns = {
 /* FIXED STRINGS! Don't touch! */
 const char linux_banner[] =
        "Linux version " UTS_RELEASE " (" LINUX_COMPILE_BY "@"
- LINUX_COMPILE_HOST ") (" LINUX_COMPILER ") " UTS_VERSION "\n";
+ LINUX_COMPILE_HOST ") (" LINUX_COMPILER ") " UTS_VERSION
+#ifdef CONFIG_VERSION_SIGNATURE
+ " (" CONFIG_VERSION_SIGNATURE ")"
+#endif
+ "\n";
diff --git a/init/version.c b/init/version.c
index 702100a4f6ab..f117921811b4 100644
--- a/init/version.c
+++ b/init/version.c
@@ -35,11 +35,7 @@ early_param("hostname", early_hostname);
 const char linux_proc_banner[] =
        "%s version %s"
        " (" LINUX_COMPILE_BY "@" LINUX_COMPILE_HOST ")"
- " (" LINUX_COMPILER ") %s"
-#ifdef CONFIG_VERSION_SIGNATURE
- " (" CONFIG_VERSION_SIGNATURE ")"
-#endif
- "\n";
+ " (" LINUX_COMPILER ") %s\n";

 BUILD_SALT;
 BUILD_LTO_INFO;

[Mauricio Faria de Oliveira (mfo)]

Andy mentioned this is known and there's a fix in the works (LP bug unknown at the moment).

[Mauricio Faria de Oliveira (mfo)]

Oops; I missed a refresh before posting my previous comment. Sorry.

The change should address the issue in crash, as /proc/version would be a substring of a 'Linux version' string in vmlinux.

Marking as Fix Committed.

[Andy Whitcroft (apw)]

This was reported without a bug from ADT testing, and fixed without a Launchpad Bug attached. This was uploaded with Ubuntu-6.2.0-16.16. Once that hits -release this bug can be closed off.

[Oibaf (oibaf)]

Fixed in 6.2.0-16.16 as per previous comment.