Ubuntu
openssh package

upgrading openssh-server always shows error: rescue-ssh.target is a disabled or a static unit not running, not starting it.

Bug #2047082 reported by Martin Pitt on 2023-12-21

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	openssh (Ubuntu)	Fix Released	Low	Colin Watson

Bug Description

In our project we regularly build Ubuntu VM images for current 23.10 (stable). In https://github.com/cockpit-project/bots/issues/5691 we ran into an upgrade failure of openssh-server. It starts with the current cloud image and then apt upgrades it, with "DEBIAN_FRONTEND=noninteractive". openssh was updated a few days ago indeed:

  Setting up openssh-server (1:9.3p1-1ubuntu3.1) ...
  Creating SSH2 ECDSA key; this may take some time ...
  256 SHA256:UqrRSpQNM7SIixVivYP/WwZRjt7Sv89P31W/Gxaf+Z8 root@ubuntu (ECDSA)
  Creating SSH2 ED25519 key; this may take some time ...
  256 SHA256:hy9AEDydfnZeY9nf9P4Sb90kx39Oqr101A6tz5j4RQw root@ubuntu (ED25519)
  rescue-ssh.target is a disabled or a static unit not running, not starting it.
  Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 145.
  dpkg: error processing package openssh-server (--configure):
   installed openssh-server package post-installation script subprocess returned error exit status 1

I.e. of course that security update itself [1] didn't introduce the regression, but earlier VM builds just didn't have a pending openssh update -- looks like this has been a luring upgrade trap in the release already.

As a first naïve reproducer I tried

apt update
DEBIAN_FRONTEND=noninteractive apt update openssh-server

on our current VM (with the release version 1:9.3p1-1ubuntu3), and that worked fine. Same with installing all 9 available packages. rescue.target is loaded/inactive/static, as it should be. Updating without DEBIAN_FRONTEND does show me a conffile prompt about /etc/ssh/sshd_config, which is justified as we do modify the config:

  # Allow root login with password
  sed -i 's/^[# ]*PermitRootLogin .*/PermitRootLogin yes/' /etc/ssh/sshd_config
  # Prevent SSH from hanging for a long time when no external network access
  echo 'UseDNS no' >> /etc/ssh/sshd_config

this also leads to a merge conflict. However, I suppose all of that is tangential to the rescue-ssh.target issue. In all my interactive upgrades, it seemed to handle that just fine:

Setting up openssh-server (1:9.3p1-1ubuntu3.1) ...
rescue-ssh.target is a disabled or a static unit not running, not starting it.

So this seems to be related to the first-time installation of openssh-server -- it is part of the cloud image, but it does the host key generation during our image builds.

So reproducing this is a bit tricky, but aside from that: Why does it even do this in the first place?

# Automatically added by dh_installsystemd/13.11.6ubuntu1
if [ "$1" = "configure" ] || [ "$1" = "abort-upgrade" ] || [ "$1" = "abort-deconfigure" ] || [ "$1" = "abort-remove" ] ; then
        if [ -d /run/systemd/system ]; then
                systemctl --system daemon-reload >/dev/null || true
                if [ -n "$2" ]; then
                        _dh_action=restart
                else
                        _dh_action=start
                fi
                deb-systemd-invoke $_dh_action 'rescue-ssh.target' >/dev/null || true
        fi
fi

It feels like the postinst should *never* try to start rescue-ssh.target. That's an alternative boot mode, and should never run un multi-user.target, isn't it?

[1] https://launchpad.net/ubuntu/+source/openssh/1:9.3p1-1ubuntu3.1

DistroRelease: Ubuntu 23.10
PackageVersion: openssh-server 1:9.3p1-1ubuntu3.1

Tags:

CVE References

Revision history for this message

Martin Pitt (pitti) wrote on 2023-12-21:

Fun, this isn't even reliable. The first atttempt failed:

https://cockpit-logs.us-east-1.linodeobjects.com/image-refresh-logs/ubuntu-stable-20231219-223939.log

I retried the build now, no package or environment changes. Only daytime and timing (race conditions). Perhaps some interaction with cloud-init?

Revision history for this message

Martin Pitt (pitti) wrote on 2023-12-21:

Argh -- I missed the alternative truth in that rescue-ssh.target shell code. So this message should pretty much *always* appear -- it's nonsense to actually try and restart rescue-ssh.target in the postinst, *always*.

But it is a red herring due to the || true. The upgrade failed on something else but didn't print any error message. So there is no remaining evidence what happens. So let's dedicate this bug report to dropping that deb-system-invoke for rescue-ssh.target.

summary:	- upgrading openssh-server failed: rescue-ssh.target is a disabled or a - static unit not running, not starting it. + upgrading openssh-server always shows error: rescue-ssh.target is a + disabled or a static unit not running, not starting it.
Changed in openssh (Ubuntu):
importance:	Undecided → Low

Christian Ehrhardt  (paelzer) on 2024-01-02

tags:

added: server-todo

Revision history for this message

Colin Watson (cjwatson) wrote on 2024-01-03:

https://salsa.debian.org/ssh-team/openssh/-/commit/da06b7ef32c20de4dd18cd578025d96a9221984b

Changed in openssh (Ubuntu):
assignee:	nobody → Colin Watson (cjwatson)
status:	New → Fix Committed

Revision history for this message

Launchpad Janitor (janitor) wrote on 2024-02-02:

Download full text (9.8 KiB)

This bug was fixed in the package openssh - 1:9.6p1-3ubuntu1

---------------
openssh (1:9.6p1-3ubuntu1) noble; urgency=medium

  * Merge with Debian unstable (LP: #2040406). Remaining changes:
    - debian/rules: modify dh_installsystemd invocations for
      socket-activated sshd.
    - debian/openssh-server.postinst: handle migration of sshd_config
      options to systemd socket options on upgrade.
    - debian/README.Debian: document systemd socket activation.
    - debian/patches/socket-activation-documentation.patch: Document
      in sshd_config(5) that ListenAddress and Port no longer work.
    - debian/openssh-server.templates: include debconf prompt
      explaining when migration cannot happen due to multiple
      ListenAddress values.
    - debian/.gitignore: drop file.
    - debian/openssh-server.postrm: remove systemd drop-ins for
      socket-activated sshd on purge.
    - debian/openssh-server.ucf-md5sum: update for Ubuntu delta
    - debian/openssh-server.tmpfile,debian/systemd/ssh.service: Move
      /run/sshd creation out of the systemd unit to a tmpfile config
      so that sshd can be run manually if necessary without having to
      create this directory by hand.
    - debian/patches/systemd-socket-activation.patch: Fix sshd
      re-execution behavior when socket activation is used.
    - debian/tests/systemd-socket-activation: Add autopkgtest
      for systemd socket activation functionality.
    - d/p/test-set-UsePAM-no-on-some-tests.patch: set UsePAM=no
      for some tests.
  * Dropped changes, fixed upstream:
    - d/p/fix-ftbfs-with-zlib13.patch: fix ftbfs when using zlib 1.3
      (LP #2049552)

openssh (1:9.6p1-3) unstable; urgency=medium

  * Allow passing extra ssh-agent arguments via
    "/usr/lib/openssh/agent-launch start", making it possible to override
    things like identity lifetime using a systemd drop-in unit (closes:
    #1059639).
  * Don't try to start rescue-ssh.target in postinst (LP: #2047082).

openssh (1:9.6p1-2) unstable; urgency=medium

  * Improve detection of broken -fzero-call-used-regs=used (see
    https://bugzilla.mindrot.org/show_bug.cgi?id=3645; fixes build on
    ppc64/ppc64el).

openssh (1:9.6p1-1) unstable; urgency=medium

This bug was fixed in the package openssh - 1:9.6p1-3ubuntu1

---------------
openssh (1:9.6p1-3ubuntu1) noble; urgency=medium

openssh (1:9.6p1-3) unstable; urgency=medium

openssh (1:9.6p1-2) unstable; urgency=medium

* Improve detection of broken -fzero-call-used-regs=used (see
    https://bugzilla.mindrot.org/show_bug.cgi?id=3645; fixes build on
    ppc64/ppc64el).

openssh (1:9.6p1-1) unstable; urgency=medium

* Use single quotes in suggested ssh-keygen commands (closes: #1057835).
  * Debconf translations:
    - Catalan (thanks, Pablo Huguet; closes: #1049995).
  * New upstream release (https://www.openssh.com/releasenotes.html#9.6p1):
    - [CVE-2023-48795] ssh(1), sshd(8): implement protocol extensions to
      thwart the so-called "Terrapin attack" discovered by Fabian Bäumer,
      Marcus Brinkmann and Jörg Schwenk. This attack allows a MITM to effect
      a limited break of the integrity of the early encrypted SSH transport
      protocol by sending extra messages prior to the commencement of
      encryption, and deleting an equal number of consecutive messages
      immediately after encryption starts. A peer SSH client/server would
      not be able to detect that messages were deleted.
    - [CVE-2023-51384] ssh-agent(1): when adding PKCS#11-hosted private keys
      while specifying destination constraints, if the PKCS#11 token
      returned multiple keys then only the first key had the constraints
      applied. Use of regular private keys, FIDO tokens and unconstrained
      keys are unaffected.
    - [CVE-2023-51385] ssh(1): if an invalid user or hostname that contained
      shell metacharacters was passed to ssh(1), and a ProxyCommand,
      LocalCommand directive or "match exec" predicate referenced the user
      or hostname via %u, %h or similar expansion token, then an attacker
      who could supply arbitrary user/hostnames to ssh(1) could potentially
      perform command injection depending on what quoting was present in the
      user-supplied ssh_config(5) directive. OpenSSH 9.6 now bans most shell
      metacharacters from user and hostnames supplied via the command-line.
    - ssh(1), sshd(8): the RFC4254 connection/channels protocol provides a
      TCP-like window mechanism that limits the amount of data that can be
      sent without acceptance from the peer. In cases where this limit was
      exceeded by a non-conforming peer SSH implementation, ssh(1)/sshd(8)
      previously discarded the extra data. From OpenSSH 9.6, ssh(1)/sshd(8)
      will now terminate the connection if a peer exceeds the window limit
      by more than a small grace factor. This change should have no effect
      of SSH implementations that follow the specification.
    - ssh(1): add a %j token that expands to the configured ProxyJump
      hostname (or the empty string if this option is not being used) that
      can be used in a number of ssh_config(5) keywords.
    - ssh(1): add ChannelTimeout support to the client, mirroring the same
      option in the server and allowing ssh(1) to terminate quiescent
      channels.
    - ssh(1), sshd(8), ssh-add(1), ssh-keygen(1): add support for reading
      ED25519 private keys in PEM PKCS8 format. Previously only the OpenSSH
      private key format was supported.
    - ssh(1), sshd(8): introduce a protocol extension to allow renegotiation
      of acceptable signature algorithms for public key authentication after
      the server has learned the username being used for authentication.
      This allows varying sshd_config(5) PubkeyAcceptedAlgorithms in a
      "Match user" block.
    - ssh-add(1), ssh-agent(1): add an agent protocol extension to allow
      specifying certificates when loading PKCS#11 keys. This allows the use
      of certificates backed by PKCS#11 private keys in all OpenSSH tools
      that support ssh-agent(1). Previously only ssh(1) supported this
      use-case.
    - ssh(1): when deciding whether to enable the keystroke timing
      obfuscation, enable it only if a channel with a TTY is active.
    - ssh(1): switch mainloop from poll(3) to ppoll(3) and mask signals
      before checking flags set in signal handler. Avoids potential race
      condition between signaling ssh to exit and polling.
    - ssh(1): when connecting to a destination with both the AddressFamily
      and CanonicalizeHostname directives in use, the AddressFamily
      directive could be ignored.
    - sftp(1): correct handling of the limits@openssh.com option when the
      server returned an unexpected message.
    - ssh(1): release GSS OIDs only at end of authentication, avoiding
      unnecessary init/cleanup cycles.
    - ssh_config(5): mention "none" is a valid argument to IdentityFile in
      the manual.
    - scp(1): improved debugging for paths from the server rejected for not
      matching the client's glob(3) pattern in old SCP/RCP protocol mode.
    - ssh-agent(1): refuse signing operations on destination-constrained
      keys if a previous session-bind operation has failed. This may prevent
      a fail-open situation in future if a user uses a mismatched ssh(1)
      client and ssh-agent(1) where the client supports a key type that the
      agent does not support.
  * debian/run-tests: Supply absolute paths to tools.
  * debian/run-tests: Enable interop tests for Dropbear.

openssh (1:9.5p1-2) unstable; urgency=medium

* Upload to unstable.

openssh (1:9.5p1-1) experimental; urgency=medium

* New upstream release (https://www.openssh.com/releasenotes.html#9.5p1):
    - ssh-keygen(1): generate Ed25519 keys by default. Ed25519 public keys
      are very convenient due to their small size. Ed25519 keys are
      specified in RFC 8709 and OpenSSH has supported them since version 6.5
      (January 2014).
    - sshd(8): the Subsystem directive now accurately preserves quoting of
      subsystem commands and arguments. This may change behaviour for exotic
      configurations, but the most common subsystem configuration
      (sftp-server) is unlikely to be affected.
    - ssh(1): add keystroke timing obfuscation to the client. This attempts
      to hide inter-keystroke timings by sending interactive traffic at
      fixed intervals (default: every 20ms) when there is only a small
      amount of data being sent. It also sends fake "chaff" keystrokes for a
      random interval after the last real keystroke. These are controlled by
      a new ssh_config ObscureKeystrokeTiming keyword.
    - ssh(1), sshd(8): Introduce a transport-level ping facility. This adds
      a pair of SSH transport protocol messages SSH2_MSG_PING/PONG to
      implement a ping capability. These messages use numbers in the "local
      extensions" number space and are advertised using a "ping@openssh.com"
      ext-info message with a string version number of "0".
    - sshd(8): allow override of Subsystem directives in sshd Match blocks.
    - scp(1): fix scp in SFTP mode recursive upload and download of
      directories that contain symlinks to other directories. In scp mode,
      the links would be followed, but in SFTP mode they were not.
    - ssh-keygen(1): handle cr+lf (instead of just cr) line endings in
      sshsig signature files.
    - ssh(1): interactive mode for ControlPersist sessions if they
      originally requested a tty.
    - sshd(8): make PerSourceMaxStartups first-match-wins.
    - sshd(8): limit artificial login delay to a reasonable maximum (5s) and
      don't delay at all for the "none" authentication mechanism.
    - sshd(8): Log errors in kex_exchange_identification() with level
      verbose instead of error to reduce preauth log spam. All of those get
      logged with a more generic error message by sshpkt_fatal().
    - sshd(8): correct math for ClientAliveInterval that caused the probes
      to be sent less frequently than configured.
    - ssh(1): fix regression in OpenSSH 9.4 (mux.c r1.99) that caused
      multiplexed sessions to ignore SIGINT under some circumstances.
  * Build-depend on dh-sequence-movetousr.
  * Report DebianBanner setting in "sshd -G/-T" output (thanks, Rasmus
    Villemoes; closes: #1053555).

-- Miriam España Acebal <miriam.espana@canonical.com>  Mon, 29 Jan 2024 11:16:31 +0100

Changed in openssh (Ubuntu):
status:	Fix Committed → Fix Released

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

openssh-portable-bugs #3645 Edit

Bug watches keep track of this bug in other bug trackers.

Ubuntuopenssh package

upgrading openssh-server always shows error: rescue-ssh.target is a disabled or a static unit not running, not starting it.

Bug Description

CVE References

Other bug subscribers

Remote bug watches

Ubuntu
openssh package