Apt hook of needrestart hang with DEBIAN_FRONTEND=noninteractive

[Christian Ehrhardt ]

I had some of my test runs blocked and eventually found this was due to a kernel upgrade without reboot making needrestart to block ignoring DEBIAN_FRONTEND=noninteractive in the apt post invoke for needrestart.

Repro:
1. Start in a Focal VM
2. sudo vim /etc/apt/sources.list
   # bump it to hirsute

Situation Now:
$ uname -a
Linux f-testupgradeissue-needrestart 5.4.0-59-generic #65-Ubuntu SMP Thu Dec 10 12:01:51 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
$ ll /boot/vmlinuz*
lrwxrwxrwx 1 root root 25 Aug 25 14:02 /boot/vmlinuz -> vmlinuz-5.11.0-31-generic
-rw------- 1 root root 14742976 Aug 11 11:32 /boot/vmlinuz-5.11.0-31-generic
-rw------- 1 root root 11686656 Dec 10 2020 /boot/vmlinuz-5.4.0-59-generic
lrwxrwxrwx 1 root root 24 Jan 5 2021 /boot/vmlinuz.old -> vmlinuz-5.4.0-59-generic

(any other means of a mismatch in running / on-disk kernel will do as well)

3. Without reboot needrestart will rightfully complain like

```
$ needrestart -k
Scanning linux images...

Pending kernel upgrade!

Running kernel version:
  5.4.0-59-generic

Diagnostics:
  The currently running kernel version is not the expected kernel version 5.11.0-31-generic.

Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. [Return]
```

That is fine if called interactively.

The same check is also done in the hook that is in apt so apt actions will
trigger this as well:

In a shell with apt, needrestart will eventually detect that and use ncurses
style popup (fine).

The problem comes with non-intractive modes.
This is meant to be used by scripts and any such to NOT be locked in
an interactive prompt while running automation.

But the following combinations still run into interactive prompts:
a) export DEBIAN_FRONTEND=noninteractive; sudo apt install --reinstall hello
   => blocks in the ncurses popup
b) removing the terminal if running the same via a script blocks in the
   console output

```
ubuntu@f-testupgradeissue-needrestart:~$ cat ./test.sh
#!/bin/bash
export DEBIAN_FRONTEND=noninteractive
apt-get -q --assume-yes --allow-unauthenticated -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install --reinstall hello
echo done
ubuntu@f-testupgradeissue-needrestart:~$ sudo ./test.sh
Reading package lists...
Building dependency tree...
Reading state information...
The following packages were automatically installed and are no longer required:
  at bsdmainutils dconf-gsettings-backend dconf-service gcc-10-base glib-networking glib-networking-common glib-networking-services gsettings-desktop-schemas libdconf1 libdns-export1109
  libffi7 libfl2 libhogweed5 libicu66 libjson-c4 libmpdec2 libnettle7 libperl5.30 libproxy1v5 libpython3.8 libpython3.8-minimal libpython3.8-stdlib libreadline5 libsoup2.4-1 ncal
  perl-modules-5.30 popularity-contest python3-entrypoints python3-requests-unixsocket python3.8 python3.8-minimal
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 1 reinstalled, 0 to remove and 0 not upgraded.
Need to get 0 B/28.2 kB of archives.
After this operation, 0 B of additional disk space will be used.
(Reading database ... 100504 files and directories currently installed.)
Preparing to unpack .../hello_2.10-2ubuntu2_amd64.deb ...
Unpacking hello (2.10-2ubuntu2) over (2.10-2ubuntu2) ...
Setting up hello (2.10-2ubuntu2) ...
Processing triggers for man-db (2.9.4-2) ...
Processing triggers for install-info (6.7.0.dfsg.2-6) ...
Scanning processes...
Scanning candidates...
Scanning linux images...

Pending kernel upgrade!

Running kernel version:
  5.4.0-59-generic

Diagnostics:
  The currently running kernel version is not the expected kernel version 5.11.0-31-generic.

Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. [Return]
```

IMHO the noninteractive should really stay non-interactive or at least have
a timeout. I had a few automations of mine blocked by this infinitely.

We have
$ cat /etc/apt/apt.conf.d/99needrestart
...
DPkg::Post-Invoke {"test -x /usr/lib/needrestart/apt-pinvoke && /usr/lib/needrestart/apt-pinvoke || true"; };

$ cat /usr/lib/needrestart/apt-pinvoke
...
exec /usr/sbin/needrestart "$@"

This in our case comes down to match:
$ sudo DEBIAN_FRONTEND=noninteractive /usr/sbin/needrestart

And indeed that behaves the same as needrestart does:

$is_tty = 0 if($opt_r eq 'i' && exists($ENV{DEBIAN_FRONTEND}) && $ENV{DEBIAN_FRONTEND} eq 'noninteractive');
$opt_r = 'l' if(!$is_tty && $opt_r eq 'i');

If we explicitly set "listing" mode it is really non-interactive.
$ sudo DEBIAN_FRONTEND=noninteractive /usr/sbin/needrestart -r l
^^ works fine

So what do these lines in the code above actually mean.
The second one is "downgrade to l if there is no tty)
The first one means
  If "i" is requested
   AND noninteractive
  Then set is_tty=0

You'd think that makes it run like "-r l" in DEBIAN_FRONTEND=noninteractive
but it does not.
Debugging-wise is seems to work, pre/mod/post these lines I got:

I: is_tty = 1 opt_r = i DEBIAN_FRONTEND = noninteractive
II: is_tty = 0 opt_r = i DEBIAN_FRONTEND = noninteractive
III: is_tty = 0 opt_r = l DEBIAN_FRONTEND = noninteractive

So it does reset is_tty and it does set opt_r to l, but eventually it
still blocks :-/

This happens in
```
sub _announce {
    my $self = shift;
    my $message = shift;
    my %vars = @_;

    print "\n";
    $self->wprint(\*STDOUT, '', '', __x("Pending kernel upgrade!\n\nRunning kernel version:\n {kversion}\n\nDiagnostics:\n {message}\n\nRestarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. [Return]\n",
                                         kversion => $vars{KVERSION},
                                         message => $message,
                   ));
    <STDIN> if (-t *STDIN && -t *STDOUT);
}
```

Of file:
  /usr/share/perl5/NeedRestart/UI/stdio.pm

We see that this is a problem because it only checks if it has an STDIN, but not
if it is meant to run non-interactively.

I'd suggest to prepend with an argument that can force-disable the
interactive wait.

From looking at the code the same seems to apply to announce_ehint and
announce_ucode.

And indeed running it in easy mode stops it twice,
once for the kernel report that I already reported and once
for the same code in annunce_ehint for the easy mode.

So I guess all those announces should be stopped from being interactive.
Adding arguments and logic might make this change rather messy.
I wondered if the following could be enough:

The following is a way smaller fix, but it pushes debian awareness deeper into
the code. At least it seems to work fine for my testing,
but I'm interested in a second pair of eyes on this:

--- /usr/share/perl5/NeedRestart/UI/stdio.pm.orig 2021-08-26 08:39:15.601085356 +0000
+++ /usr/share/perl5/NeedRestart/UI/stdio.pm 2021-08-26 08:43:05.257583706 +0000
@@ -43,7 +43,7 @@
       kversion => $vars{KVERSION},
       message => $message,
      ));
- <STDIN> if (-t *STDIN && -t *STDOUT);
+ <STDIN> if (-t *STDIN && -t *STDOUT && !(exists($ENV{DEBIAN_FRONTEND}) && $ENV{DEBIAN_FRONTEND} eq 'noninteractive'));
 }

@@ -77,7 +77,7 @@

 EHINT

- <STDIN> if (-t *STDIN && -t *STDOUT);
+ <STDIN> if (-t *STDIN && -t *STDOUT && !(exists($ENV{DEBIAN_FRONTEND}) && $ENV{DEBIAN_FRONTEND} eq 'noninteractive'));
 }

@@ -90,7 +90,7 @@
     current => $vars{CURRENT},
     avail => $vars{AVAIL},
      ));
- <STDIN> if (-t *STDIN && -t *STDOUT);
+ <STDIN> if (-t *STDIN && -t *STDOUT && !(exists($ENV{DEBIAN_FRONTEND}) && $ENV{DEBIAN_FRONTEND} eq 'noninteractive'));
 }

@@ -141,7 +141,7 @@
      return $s;
  }

- $i = <STDIN> if(-t *STDIN && -t *STDOUT);
+ $i = <STDIN> if (-t *STDIN && -t *STDOUT && !(exists($ENV{DEBIAN_FRONTEND}) && $ENV{DEBIAN_FRONTEND} eq 'noninteractive'));
  unless(defined($i)) {
      $i = 'n';
      last;

[Robie Basak (racb)]

Your suggestion will work for the noninteractive case but not for any other case where debconf is being adjusted - for example when requesting critical prompts only and that kind of thing.

I wonder if it's possible to use debconf itself for the prompt so that it will automatically comply with whatever users are already doing. I realise that it may be odd to put it in an apt hook and not have it running from dpkg, but it might be the least worst solution.

Another thought: I wonder if redirecting stdin to /dev/null works. However, I'm reluctant to suggest it as a workaround as the main issue here I think is that the existing known steps to ensure non-interactiveness no longer work, so asking users to do yet another thing is suboptimal.

Finally, until a good solution is found, can we revert the regression immediately by just dropping the blocking prompt, and printing the message to the terminal instead?

[Robie Basak (racb)]

Upstream bug: https://github.com/liske/needrestart/issues/129
Upstream PR (wontfix): https://github.com/liske/needrestart/pull/168
Cherry-picked by xnox in 3.5-4ubuntu2: https://git.launchpad.net/~usd-import-team/ubuntu/+source/needrestart/commit/?id=66c0350190f01ee471cf0f74ed07686fd3df196d

[Christian Ehrhardt  (paelzer)]

Thanks for the checks Robie, thereby only to consider improving Hirsute then I guess?

[Brian Murray (brian-murray)]

The Hirsute Hippo has reached End of Life, so this bug will not be fixed for that release.