apt-get dist-upgrade -y hangs during e2fsprogs update

[Jonathan Kamens]

My recent apt-get dist-upgrade -y hung here:

...
update-initramfs: deferring update (trigger activated)
e2scrub_all.service is a disabled or a static unit not running, not starting it.

I did a pstree on the apt-get process and saw this:

apt-get(31290)───dpkg(562)───e2fsprogs.posti(587)───systemctl(722)───systemd-tty-ask(863)

Relevant processes:

root 863 0.0 0.0 15976 5996 pts/12 S+ 09:13 0:00 /bin/systemd-tty-ask-password-agent --watch
root 722 0.0 0.0 19668 3672 pts/12 S+ 09:13 0:00 /bin/systemctl restart e2scrub_all.timer e2scrub_reap.service

I killed the systemd-tty-ask-password-agent process, and it became defunct but didn't die. So I killed the systemctl restart process, and the upgrade continued, apparently without noticing that the systemctl restart failed, which is a separate issue.

When I ran "sudo systemctl restart e2scrub_all.timer e2scrub_reap.service" from the command line after the upgrade, it also hung with systemd-tty-ask-password-agent apparently waiting for something, so this problem isn't unique to upgrades.

ProblemType: Bug
DistroRelease: Ubuntu 19.10
Package: e2fsprogs 1.45.3-4ubuntu2.1
ProcVersionSignature: Ubuntu 5.3.0-26.28-generic 5.3.13
Uname: Linux 5.3.0-26-generic x86_64
NonfreeKernelModules: zfs zunicode zavl icp zcommon znvpair
ApportVersion: 2.20.11-0ubuntu8.2
Architecture: amd64
CurrentDesktop: ubuntu:GNOME
Date: Thu Jan 23 09:37:04 2020
InstallationDate: Installed on 2019-09-12 (132 days ago)
InstallationMedia: Ubuntu 19.04 "Disco Dingo" - Release amd64 (20190416)
SourcePackage: e2fsprogs
UpgradeStatus: Upgraded to eoan on 2019-09-20 (125 days ago)
modified.conffile..etc.apport.crashdb.conf: [modified]
mtime.conffile..etc.apport.crashdb.conf: 2019-10-23T16:51:18.143596

[Jonathan Kamens (jik)]

e2scrub_all.timer restarts just fine; it's the e2scrub_reap.service restart that's hanging.

[Theodore Ts'o (tytso)]

Hmm... I can't duplicate this on Debian testing, which is using a newer version of e2fsprogs (1.45.5-2). The e2scrub_reap.service file is identical with Ubuntu's 1.45.3-4ubuntu2.1, though. And looking at the service file, I have no idea why systemctl would be trying to paskk for a password, given that it is running as root.

<tytso@lambda> {/home/tytso}
1046% sudo systemctl restart e2scrub_reap.service
[sudo] password for tytso:
<tytso@lambda> {/home/tytso}
1047% sudo systemctl status e2scrub_reap.service
● e2scrub_reap.service - Remove Stale Online ext4 Metadata Check Snapshots
     Loaded: loaded (/lib/systemd/system/e2scrub_reap.service; enabled; vendor preset: enabled)
     Active: inactive (dead) since Fri 2020-01-24 22:20:55 EST; 3s ago
       Docs: man:e2scrub_all(8)
    Process: 1106527 ExecStart=/sbin/e2scrub_all -A -r (code=exited, status=0/SUCCESS)
   Main PID: 1106527 (code=exited, status=0/SUCCESS)

Jan 24 22:20:55 lambda systemd[1]: Starting Remove Stale Online ext4 Metadata Check Snapshots...
Jan 24 22:20:55 lambda systemd[1]: e2scrub_reap.service: Succeeded.
Jan 24 22:20:55 lambda systemd[1]: Started Remove Stale Online ext4 Metadata Check Snapshots.

It is true that /sbin/e2scrub_all has changed significantly between 1.45.3 and 1.45.5; but it looked like you never got as far as actually executing e2scrub_reap.service, and if it did, it shouldn't be calling systemctl in reap mode.

Does "sudo /sbin/e2scrub_all -A -r" hang for you?

The only other thing which you might try i, commenting out this line from /lib/systemd/system/e2scrpb_reap.service, and see if it fixes the hang:

AmbientCapabilities=CAP_SYS_ADMIN CAP_SYS_RAWIO

If it does, I have no idea why it might be working for me using Debian testing (which is using systemd 244-3) and why it ight be failing for you with Ubuntu. Remember, systemd is your *friend*. It is the best init replacement compared to all the others. All Hail Systemd! :-/

[Jonathan Kamens (jik)]

I can no longer reproduce it either. *shrug*

P.S. Hi Ted long time no see.