apt-check should not be run from update-motd

[Scott Moser]

If i launch a new cloud instance, and then ssh to it, my ssh login blocks, stopping me from doing useful work while '/usr/lib/update-notifier/apt-check' runs.

By far, the biggest offender of this slow login is /usr/lib/update-notifier/apt-check.
To demonstrate, a fresh cloud instance on azure.

## ssh to the instance and run a non-interactive command:
$ time ssh $cloudhost /bin/true
real 0m5.066s
user 0m0.045s
sys 0m0.004s

## ssh in and disable updates-available so it wont run at all
$ ssh $cloudhost sudo chmod -x /etc/update-motd.d/90-updates-available
$ time ssh $cloudhost /bin/true
real 0m2.459s
user 0m0.026s
sys 0m0.015s

## so disabling that bought us 2.5 seconds back.

## How about entirely disabling update-motd
$ ssh $cloudhost "sudo cp /etc/pam.d/sshd /etc/pam.d/sshd.dist"
$ ssh $cloudhost "sudo sed -i '/^[^#].*pam_motd/s/^/#/' /etc/pam.d/sshd"
$ time ssh $cloudhost /bin/true
real 0m0.650s
user 0m0.041s
sys 0m0.004s

Note, that these things happen even for non-interactive logins, where the message of the day is completely swallowed. Automation tools suffer this ever time they ssh to a ubuntu system.

ProblemType: Bug
DistroRelease: Ubuntu 14.10
Package: update-notifier-common 3.157
ProcVersionSignature: User Name 3.16.0-30.40-generic 3.16.7-ckt3
Uname: Linux 3.16.0-30-generic x86_64
ApportVersion: 2.14.7-0ubuntu8.2
Architecture: amd64
Date: Thu Feb 26 16:27:02 2015
PackageArchitecture: all
ProcEnviron:
 TERM=xterm
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: update-notifier
UpgradeStatus: No upgrade log present (probably fresh install)

[Steve Langasek (vorlon)]

I agree that it doesn't make sense to block the login updating the motd when we aren't actually going to display the motd. I'm not sure if this should be solved in pam_motd, or in openssh - perhaps openssh should be using different a separate PAM profile for interactive vs. non-interactive sessions; or maybe openssh needs to be calling pam_open_session() with the PAM_SILENT flag for non-interactive sessions. Either of these changes would let us avoid wasteful calls to update-motd when it won't be displayed. But pam, at least, is working as designed here.

And the inclusion of package information in the dynamic output is as originally requested by the server team. If the server team no longer wants this information in the motd then we can drop this, but then it should be dropped across the board.

As for the actual delays you're seeing: this would be expected the first time you sshed to a machine, but after the first run, the apt check results are aggressively cached. Is the behavior you're describing reproducible on subsequent ssh calls to the same machine? If so there's certainly a bug in update-notifier.

For reference:

$ time ssh localhost true

real 0m0.301s
user 0m0.030s
sys 0m0.002s
$

[Scott Moser (smoser)]

this particular system (in azure) has io that rivals that of floppy disks, so the times are more dramatic.
you're correct in that the information is cached, and that is in general a good thing, but if I ssh to a system only once a day, then every time i do it, i hit this penalty.

thats in addition to the fact that often times the only reason anyone would ever ssh to a system is to to investigate failure/load issues. And in that case, i've massively added to the load by ssh'ing in :(.