PasswordAuthenticaion in sshd_config.d

[Gerard Weatherby]

[Impact]

When using the "Match" phrase in sshd_config.d files, the configuration does not apply. This leads to failures in user-specific configurations such as with PasswordAuthentication.

The fix for this issue should be added to Focal to allow users to use Match as expected.

The bug is fixed by backporting an upstream commit that includes custom config files then runs all matches provided. It updates the function for reading in config files with checks for matches, and, if the correct flags are marked, the match will then be handled accordingly.

[Test Plan]

$ lxc launch images:ubuntu/focal test-ssh-focal
$ lxc exec test-ssh-focal bash

# apt update && apt upgrade -y
# apt install openssh-server
# adduser user

> ssh into container from another terminal to show pw auth is available by default. You can get the ip through 'ip addr' in the container or 'lxc list' outside.

$ ssh user@<container-ip>
user@<container-ip>'s password:

# cat <<EOF >/etc/ssh/sshd_config.d/user.conf
Match User user
  PasswordAuthentication No
Match All
EOF

# systemctl restart sshd

> Check again in other terminal

$ ssh user@<container-ip>

> Before the fix, it will show:
user@<container-ip>'s password:

> After, it will show
user@<container-ip>: Permission denied (publickey).

[Where problems could occur]

If problems were to occur, they would be in the interpretation of configuration files. All changes from this fix exist in servconf.c. The largest part of this change is a move from the inc_flags variable being an integer to an integer pointer, so problems could show up through changes to the flags in the pass by reference. Going over the change to pointer usage visually, all instances within the process_server_config_line_depth function are modified properly, along with the two calls to the function.

[Other Info]

This issue has already been fixed in Jammy and later, as it was fixed in upstream version 8.4.

To use the PPA containing this fix, you can run:

$ sudo apt install -y software-properties-common
$ sudo add-apt-repository -y ppa:lvoytek/openssh-fix-passwordauthentication-config
$ sudo apt update
$ sudo apt upgrade -y
$ sudo systemctl restart sshd

[Original Description]

The stanza
Match User <username>
 PasswordAuthentication no

in /etc/ssh/sshd_config works as expected.

The same stanza in /etc/ssh/sshd_config.d/username.conf does not work.

The Include in /etc/ssh/sshd_config is not commented out, and

/usr/sbin/sshd -D -ddd

shows the username.config file being parsed.

ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: openssh-server 1:8.2p1-4ubuntu0.5
ProcVersionSignature: Ubuntu 5.4.0-131.147-generic 5.4.210
Uname: Linux 5.4.0-131-generic x86_64
NonfreeKernelModules: falcon_lsm_serviceable falcon_nf_netcontain falcon_kal falcon_lsm_pinned_14713
ApportVersion: 2.20.11-0ubuntu27.25
Architecture: amd64
CasperMD5CheckResult: skip
Date: Mon Mar 20 13:34:14 2023
InstallationDate: Installed on 2022-11-04 (136 days ago)
InstallationMedia:

SSHDConfig: Error: command ['pkexec', '/usr/sbin/sshd', '-T'] failed with exit code 127: pkexec must be setuid root
SourcePackage: openssh
UpgradeStatus: No upgrade log present (probably fresh install)

[Lucas Kanashiro (lucaskanashiro)]

Thanks for taking the time to report this bug and trying to make Ubuntu better.

Could you please share your config files (anonymizing any necessary data)? The config file might be loaded but depending on the ordering and the content inside the config files the option you are setting can be overriden.

I am setting this bug to Incomplete until you provided the requested information. Once that's done please set the bug status back to New.

[Gerard Weatherby (gweatherby)]

Failing sshd_config

[Gerard Weatherby (gweatherby)]

sshd_config.d file that doesn't work

[Gerard Weatherby (gweatherby)]

working sshd_config with Match block inline.

[Lena Voytek (lvoytek)]

Thanks for the update! I managed to reproduce this on my end:

$ lxc launch images:ubuntu/focal test-ssh-focal
$ lxc exec test-ssh-focal bash

# apt update && apt upgrade -y
# apt install openssh-server
# adduser user

> ssh into container from another terminal to show pw auth is available by default:

$ ssh user@<container-ip>
user@<container-ip>'s password:

> Update /etc/ssh/sshd_config with the following in first terminal

Match User user
  PasswordAuthentication No
Match All

# systemctl restart sshd

> Check again in other terminal

$ ssh user@<container-ip>
user@<container-ip>: Permission denied (publickey).

> Now remove the lines from /etc/ssh/sshd_config and add them to /etc/ssh/sshd_config.d/username.conf

# systemctl restart sshd

> Check if the other config worked in other terminal (it does not)

$ ssh user@<container-ip>
user@<container-ip>'s password:

I did, however, confirm this works for Jammy and Kinetic, so this is a Focal-specific issue. Marking as such

[Christian Ehrhardt  (paelzer)]

FYI: might be related (or even dup) of bug 2002994

[Gerard Weatherby (gweatherby)]

I don't think it's a duplicate of 2002994.

I tried moving the Include line in the main sshd_config to the end of the file and the behavior was unaffected.

[Paride Legovini (paride)]

I think I found a relevant upstream bug [0], fixed by [1]. If the patch applies fairly cleanly to the version in Focal this could be good SRU material.

[0] https://bugzilla.mindrot.org/show_bug.cgi?id=3122
[1] https://github.com/openssh/openssh-portable/commit/7af1e92cd289b7eaa9a683e9a6f2fddd98f37a01

[Lena Voytek (lvoytek)]

Thanks for finding the commit Paride. I ported it to Focal in a PPA here: https://launchpad.net/~lvoytek/+archive/ubuntu/openssh-fix-passwordauthentication-config

After testing I can confirm this worked for me, if you would like to test it though you can run the following commands:

sudo add-apt-repository ppa:lvoytek/openssh-fix-passwordauthentication-config
sudo apt update
sudo apt upgrade

I'll assign this bug to myself and work to get it added to 20.04

[Gerard Weatherby (gweatherby)]

@lvoytek

PPA works as expected, thank you.

For completeness, I'll note the affected packages appear to be:

sudo apt-get install -y openssh-client openssh-server openssh-sftp-server

(We manage our systems such that we don't apply non-security updates during system lifecycle so our users have a consistent environment).

[Gerard Weatherby (gweatherby)]

I'm not familiar with the Ubuntu update timeline -- is there an ETA for when we will see this in the mainline repository?

[Lena Voytek (lvoytek)]

Thanks for the confirmation Gerard. This fix won't be marked as a security update, but should show up in the mainline Ubuntu focal-updates repository soon once its uploaded and accepted. I don't have an exact eta, but it should happen within the next week or two.

[Steve Langasek (vorlon)]

Hello Gerard, or anyone else affected,

Accepted openssh into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/openssh/1:8.2p1-4ubuntu0.7 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Gerard Weatherby (gweatherby)]

Tested openssh-server/focal-proposed,now 1:8.2p1-4ubuntu0.7 amd64 [installed]

creating /etc/ssh/sshd_config.d/<username>.conf:

Match User <username>
  PasswordAuthentication No

worked as expected: prohibited password login while allow publickey authentication

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted openssh (1:8.2p1-4ubuntu0.7) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

gvfs/1.44.1-1ubuntu1.2 (amd64, ppc64el)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#openssh

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Gerard Weatherby (gweatherby)]

Do we have an estimate for when this will go mainline?

[Robie Basak (racb)]

Please could someone verify that with openssh-server 1:8.2p1-4ubuntu0.7 installed in its default configuration, ssh works as expected? If you could detail the exact steps you performed to verify this and copy and paste the output of eg. "dpkg -l|grep openssh" so there is no doubt as to what was tested and what version was tested, then I can release this to focal-updates.

Perhaps you already did this Gerard - it's specified as the first step in the Test Plan - but it isn't clear from your report whether you followed the Test Plan exactly as written or not. It would be a disaster for Ubuntu users to accidentally release a regression if this wasn't checked - particularly for openssh where a breakage might prevent users from being able to install a regression fix.

No need to do it again if it's already done, but I'd appreciate an explicit statement that this has been verified and that it works.

Once done, this is ready to release to focal-updates.

Thanks!

[Lena Voytek (lvoytek)]

Here's the result of me running through the test with the new version:

$ lxc launch images:ubuntu/focal test-ssh-focal
$ lxc exec test-ssh-focal bash

# cat <<EOF >/etc/apt/sources.list.d/ubuntu-$(lsb_release -cs)-proposed.list
# Enable Ubuntu proposed archive
deb http://archive.ubuntu.com/ubuntu/ $(lsb_release -cs)-proposed restricted main multiverse universe
EOF

# apt update && apt upgrade -y
# apt install openssh-server
# adduser user

> In a separate terminal:

$ lxc list
+-----------------+---------+---------------------+
| NAME | STATE | IPV4 |
+-----------------+---------+---------------------+
| test-ssh-focal | RUNNING | 10.190.23.31 (eth0) |
+-----------------+---------+---------------------+

$ ssh user@10.190.23.31
user@10.190.23.31's password:

> Back in the container

# cat <<EOF >/etc/ssh/sshd_config.d/user.conf
Match User user
  PasswordAuthentication No
Match All
EOF

# systemctl restart sshd

> Check again in other terminal

$ ssh user@10.190.23.31
user@10.190.23.31: Permission denied (publickey).

[Robie Basak (racb)]

Thanks Lena!

[Launchpad Janitor (janitor)]

This bug was fixed in the package openssh - 1:8.2p1-4ubuntu0.7

---------------
openssh (1:8.2p1-4ubuntu0.7) focal; urgency=medium

  * d/p/lp2012298-upstream-fix-match-in-d-config.patch: Allow ssh_config.d/
    configuration files to correctly update the PasswordAuthentication setting
    (LP: #2012298)

 -- Lena Voytek <email address hidden> Mon, 03 Apr 2023 15:47:13 -0700

[Robie Basak (racb)]

The verification of the Stable Release Update for openssh has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.