sshd.service ssh.socket systemd-tmpfiles-setup:Before= - Missing privilege separation directory: /run/sshd
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
openssh (Ubuntu) |
Fix Released
|
High
|
Nick Rosbrook | ||
Mantic |
New
|
Low
|
Unassigned |
Bug Description
Symptom: ssh.service is not running and not accepting new SSH connections and returns socket closed error upon attempted connection. ssh.service shows a failed with error message "Missing privilege separation directory: /run/sshd".
The following line is needed in the SystemD Unit configuration of the ssh.service file for the openssh-server package to fix what appears to be a race condition when ssh.service is sometimes started before systemd-
There might be more depth or an alternative reason to why /run/sshd/ still didn't exist after using ```systemctl reset-failed ssh.service```. So more investigation is needed by someone with more background in systemd-
= Error Messages =
```
$journalctl -u ssh.service -b-1
Mar 02 21:30:56 server systemd[1]: Starting ssh.service - OpenBSD Secure Shell server...
Mar 02 21:30:56 server sshd[1271]: Missing privilege separation directory: /run/sshd
Mar 02 21:30:56 server systemd[1]: ssh.service: Control process exited, code=exited, status=
Mar 02 21:30:56 server systemd[1]: ssh.service: Failed with result 'exit-code'.
Mar 02 21:30:56 server systemd[1]: Failed to start ssh.service - OpenBSD Secure Shell server.
Mar 02 21:30:56 server systemd[1]: ssh.service: Scheduled restart job, restart counter is at 1.
Mar 02 21:30:56 server systemd[1]: Stopped ssh.service - OpenBSD Secure Shell server.
Mar 02 21:30:56 server systemd[1]: Starting ssh.service - OpenBSD Secure Shell server...
Mar 02 21:30:56 server sshd[1369]: Missing privilege separation directory: /run/sshd
Mar 02 21:30:56 server systemd[1]: ssh.service: Control process exited, code=exited, status=
Mar 02 21:30:56 server systemd[1]: ssh.service: Failed with result 'exit-code'.
Mar 02 21:30:56 server systemd[1]: Failed to start ssh.service - OpenBSD Secure Shell server.
Mar 02 21:30:56 server systemd[1]: ssh.service: Scheduled restart job, restart counter is at 2.
Mar 02 21:30:56 server systemd[1]: Stopped ssh.service - OpenBSD Secure Shell server.
Mar 02 21:30:56 server systemd[1]: Starting ssh.service - OpenBSD Secure Shell server...
Mar 02 21:30:56 server sshd[1454]: Missing privilege separation directory: /run/sshd
Mar 02 21:30:56 server systemd[1]: ssh.service: Control process exited, code=exited, status=
Mar 02 21:30:56 server systemd[1]: ssh.service: Failed with result 'exit-code'.
Mar 02 21:30:56 server systemd[1]: Failed to start ssh.service - OpenBSD Secure Shell server.
Mar 02 21:30:57 server systemd[1]: ssh.service: Scheduled restart job, restart counter is at 3.
Mar 02 21:30:57 server systemd[1]: Stopped ssh.service - OpenBSD Secure Shell server.
Mar 02 21:30:57 server systemd[1]: Starting ssh.service - OpenBSD Secure Shell server...
Mar 02 21:30:57 server sshd[1465]: Missing privilege separation directory: /run/sshd
Mar 02 21:30:57 server systemd[1]: ssh.service: Control process exited, code=exited, status=
Mar 02 21:30:57 server systemd[1]: ssh.service: Failed with result 'exit-code'.
Mar 02 21:30:57 server systemd[1]: Failed to start ssh.service - OpenBSD Secure Shell server.
Mar 02 21:30:57 server systemd[1]: ssh.service: Scheduled restart job, restart counter is at 4.
Mar 02 21:30:57 server systemd[1]: Stopped ssh.service - OpenBSD Secure Shell server.
Mar 02 21:30:57 server systemd[1]: Starting ssh.service - OpenBSD Secure Shell server...
Mar 02 21:30:57 server sshd[1475]: Missing privilege separation directory: /run/sshd
Mar 02 21:30:57 server systemd[1]: ssh.service: Control process exited, code=exited, status=
Mar 02 21:30:57 server systemd[1]: ssh.service: Failed with result 'exit-code'.
Mar 02 21:30:57 server systemd[1]: Failed to start ssh.service - OpenBSD Secure Shell server.
Mar 02 21:30:57 server systemd[1]: ssh.service: Scheduled restart job, restart counter is at 5.
Mar 02 21:30:57 server systemd[1]: Stopped ssh.service - OpenBSD Secure Shell server.
Mar 02 21:30:57 server systemd[1]: ssh.service: Start request repeated too quickly.
Mar 02 21:30:57 server systemd[1]: ssh.service: Failed with result 'exit-code'.
Mar 02 21:30:57 server systemd[1]: Failed to start ssh.service - OpenBSD Secure Shell server.
Mar 02 22:19:25 server systemd[1]: Starting ssh.service - OpenBSD Secure Shell server...
Mar 02 22:19:25 server sshd[47238]: Missing privilege separation directory: /run/sshd
Mar 02 22:19:25 server systemd[1]: ssh.service: Control process exited, code=exited, status=
Mar 02 22:19:25 server systemd[1]: ssh.service: Failed with result 'exit-code'.
Mar 02 22:19:25 server systemd[1]: Failed to start ssh.service - OpenBSD Secure Shell server.
Mar 02 22:19:25 server systemd[1]: ssh.service: Scheduled restart job, restart counter is at 1.
Mar 02 22:19:25 server systemd[1]: Stopped ssh.service - OpenBSD Secure Shell server.
Mar 02 22:19:25 server systemd[1]: Starting ssh.service - OpenBSD Secure Shell server...
Mar 02 22:19:25 server sshd[47240]: Missing privilege separation directory: /run/sshd
Mar 02 22:19:25 server systemd[1]: ssh.service: Control process exited, code=exited, status=
Mar 02 22:19:25 server systemd[1]: ssh.service: Failed with result 'exit-code'.
Mar 02 22:19:25 server systemd[1]: Failed to start ssh.service - OpenBSD Secure Shell server.
Mar 02 22:19:26 server systemd[1]: ssh.service: Scheduled restart job, restart counter is at 2.
Mar 02 22:19:26 server systemd[1]: Stopped ssh.service - OpenBSD Secure Shell server.
Mar 02 22:19:26 server systemd[1]: Starting ssh.service - OpenBSD Secure Shell server...
Mar 02 22:19:26 server sshd[47241]: Missing privilege separation directory: /run/sshd
Mar 02 22:19:26 server systemd[1]: ssh.service: Control process exited, code=exited, status=
Mar 02 22:19:26 server systemd[1]: ssh.service: Failed with result 'exit-code'.
Mar 02 22:19:26 server systemd[1]: Failed to start ssh.service - OpenBSD Secure Shell server.
Mar 02 22:19:26 server systemd[1]: ssh.service: Scheduled restart job, restart counter is at 3.
Mar 02 22:19:26 server systemd[1]: Stopped ssh.service - OpenBSD Secure Shell server.
Mar 02 22:19:26 server systemd[1]: Starting ssh.service - OpenBSD Secure Shell server...
Mar 02 22:19:26 server sshd[47242]: Missing privilege separation directory: /run/sshd
Mar 02 22:19:26 server systemd[1]: ssh.service: Control process exited, code=exited, status=
Mar 02 22:19:26 server systemd[1]: ssh.service: Failed with result 'exit-code'.
Mar 02 22:19:26 server systemd[1]: Failed to start ssh.service - OpenBSD Secure Shell server.
Mar 02 22:19:26 server systemd[1]: ssh.service: Scheduled restart job, restart counter is at 4.
Mar 02 22:19:26 server systemd[1]: Stopped ssh.service - OpenBSD Secure Shell server.
Mar 02 22:19:26 server systemd[1]: Starting ssh.service - OpenBSD Secure Shell server...
Mar 02 22:19:26 server sshd[47243]: Missing privilege separation directory: /run/sshd
Mar 02 22:19:26 server systemd[1]: ssh.service: Control process exited, code=exited, status=
Mar 02 22:19:26 server systemd[1]: ssh.service: Failed with result 'exit-code'.
Mar 02 22:19:26 server systemd[1]: Failed to start ssh.service - OpenBSD Secure Shell server.
Mar 02 22:19:26 server systemd[1]: ssh.service: Scheduled restart job, restart counter is at 5.
Mar 02 22:19:26 server systemd[1]: Stopped ssh.service - OpenBSD Secure Shell server.
Mar 02 22:19:26 server systemd[1]: ssh.service: Start request repeated too quickly.
Mar 02 22:19:26 server systemd[1]: ssh.service: Failed with result 'exit-code'.
Mar 02 22:19:26 server systemd[1]: Failed to start ssh.service - OpenBSD Secure Shell server.
Mar 02 22:19:37 server systemd[1]: Starting ssh.service - OpenBSD Secure Shell server...
Mar 02 22:19:37 server sshd[47253]: Missing privilege separation directory: /run/sshd
Mar 02 22:19:37 server systemd[1]: ssh.service: Control process exited, code=exited, status=
Mar 02 22:19:37 server systemd[1]: ssh.service: Failed with result 'exit-code'.
Mar 02 22:19:37 server systemd[1]: Failed to start ssh.service - OpenBSD Secure Shell server.
Mar 02 22:19:37 server systemd[1]: ssh.service: Scheduled restart job, restart counter is at 1.
Mar 02 22:19:37 server systemd[1]: Stopped ssh.service - OpenBSD Secure Shell server.
Mar 02 22:19:37 server systemd[1]: Starting ssh.service - OpenBSD Secure Shell server...
Mar 02 22:19:37 server sshd[47254]: Missing privilege separation directory: /run/sshd
Mar 02 22:19:37 server systemd[1]: ssh.service: Control process exited, code=exited, status=
Mar 02 22:19:37 server systemd[1]: ssh.service: Failed with result 'exit-code'.
Mar 02 22:19:37 server systemd[1]: Failed to start ssh.service - OpenBSD Secure Shell server.
Mar 02 22:19:37 server systemd[1]: ssh.service: Scheduled restart job, restart counter is at 2.
Mar 02 22:19:37 server systemd[1]: Stopped ssh.service - OpenBSD Secure Shell server.
Mar 02 22:19:37 server systemd[1]: Starting ssh.service - OpenBSD Secure Shell server...
Mar 02 22:19:37 server sshd[47255]: Missing privilege separation directory: /run/sshd
Mar 02 22:19:37 server systemd[1]: ssh.service: Control process exited, code=exited, status=
Mar 02 22:19:37 server systemd[1]: ssh.service: Failed with result 'exit-code'.
Mar 02 22:19:37 server systemd[1]: Failed to start ssh.service - OpenBSD Secure Shell server.
Mar 02 22:19:38 server systemd[1]: ssh.service: Scheduled restart job, restart counter is at 3.
Mar 02 22:19:38 server systemd[1]: Stopped ssh.service - OpenBSD Secure Shell server.
Mar 02 22:19:38 server systemd[1]: Starting ssh.service - OpenBSD Secure Shell server...
Mar 02 22:19:38 server sshd[47256]: Missing privilege separation directory: /run/sshd
Mar 02 22:19:38 server systemd[1]: ssh.service: Control process exited, code=exited, status=
Mar 02 22:19:38 server systemd[1]: ssh.service: Failed with result 'exit-code'.
Mar 02 22:19:38 server systemd[1]: Failed to start ssh.service - OpenBSD Secure Shell server.
Mar 02 22:19:38 server systemd[1]: ssh.service: Scheduled restart job, restart counter is at 4.
Mar 02 22:19:38 server systemd[1]: Stopped ssh.service - OpenBSD Secure Shell server.
Mar 02 22:19:38 server systemd[1]: Starting ssh.service - OpenBSD Secure Shell server...
Mar 02 22:19:38 server sshd[47257]: Missing privilege separation directory: /run/sshd
Mar 02 22:19:38 server systemd[1]: ssh.service: Control process exited, code=exited, status=
Mar 02 22:19:38 server systemd[1]: ssh.service: Failed with result 'exit-code'.
Mar 02 22:19:38 server systemd[1]: Failed to start ssh.service - OpenBSD Secure Shell server.
Mar 02 22:19:38 server systemd[1]: ssh.service: Scheduled restart job, restart counter is at 5.
Mar 02 22:19:38 server systemd[1]: Stopped ssh.service - OpenBSD Secure Shell server.
Mar 02 22:19:38 server systemd[1]: ssh.service: Start request repeated too quickly.
Mar 02 22:19:38 server systemd[1]: ssh.service: Failed with result 'exit-code'.
Mar 02 22:19:38 server systemd[1]: Failed to start ssh.service - OpenBSD Secure Shell server.
Mar 02 22:19:45 server systemd[1]: ssh.service: Start request repeated too quickly.
Mar 02 22:19:45 server systemd[1]: ssh.service: Failed with result 'exit-code'.
Mar 02 22:19:45 server systemd[1]: Failed to start ssh.service - OpenBSD Secure Shell server.
```
Note: Errors showed up a total of 3x5 times because the ```systemctl reset-failed ssh.service``` command was used and it still showed up.
= Add to Package SystemD Unit File =
```
''/lib/
[Unit]
After=systemd-
```
= System Configuration =
```
''lsb_release -rd''
No LSB modules are available.
Description: Ubuntu 23.10
Release: 23.10
```
```
''cat /etc/lsb-release''
DISTRIB_ID=Ubuntu
DISTRIB_
DISTRIB_
DISTRIB_
```
```
''apt-cache policy openssh-server''
openssh-server:
Installed: 1:9.3p1-1ubuntu3.2
Candidate: 1:9.3p1-1ubuntu3.2
Version table:
*** 1:9.3p1-1ubuntu3.2 500
500 http://
500 http://
100 /var/lib/
1:
500 http://
```
= Package Configuration Problems =
```
''systemctl --property=After show ssh.service''
After=network.
```
```
''systemctl --no-pager --property=Before show systemd-
Before=ssh.service man-db.service logrotate.service systemd-
```
**Note: ssh.service needs to be in the Before= clause above to fix the race condition of the ssh.service starting before the /usr/lib/
```
''cat /usr/lib/
#Type Path Mode UID GID Age Arguments
D /run/sshd 0755 root root - -
```
= Temporary Solution =
Create the directory and the Unit file with the After= clause to populate the Before= clause for systemd-
```
''mkdir -p /etc/systemd/
cat >/etc/systemd/
[Unit]
After=systemd-
EOF
```
Verify the file.
```
''cat /etc/systemd/
[Unit]
After=systemd-
```
```
Reload SystemD config
```
systemctl daemon-reload
```
Verify new active config.
```
''systemctl --no-pager --property=Before show systemd-
Before=... ssh.service ...
```
```
''systemctl --property=After show ssh.service''
After=... systemd-
=== End of Bug Report ===
ProblemType: Bug
DistroRelease: Ubuntu 23.10
Package: openssh-server 1:9.3p1-1ubuntu3.2
ProcVersionSign
Uname: Linux 6.5.0-21-generic x86_64
NonfreeKernelMo
ApportVersion: 2.27.0-0ubuntu5
Architecture: amd64
CasperMD5CheckR
Date: Sun Mar 3 02:22:39 2024
InstallationDate: Installed on 2020-12-14 (1174 days ago)
InstallationMedia: Ubuntu 20.04.1 LTS "Focal Fossa" - Release amd64 (20200731)
ProcEnviron:
LANG=en_US.UTF-8
PATH=(custom, no user)
SHELL=/bin/bash
TERM=putty
SourcePackage: openssh
UpgradeStatus: Upgraded to mantic on 2024-02-26 (6 days ago)
tags: | added: packaging systemd-boot |
tags: | added: rls-nn-incoming |
Changed in openssh (Ubuntu): | |
assignee: | nobody → Nick Rosbrook (enr0n) |
Changed in openssh (Ubuntu Mantic): | |
importance: | Undecided → Low |
Changed in openssh (Ubuntu): | |
status: | New → Triaged |
importance: | Undecided → High |
tags: |
added: foundations-todo removed: rls-nn-incoming |
Changed in openssh (Ubuntu): | |
status: | Triaged → Fix Committed |
tags: | removed: foundations-todo |
This is a follow-up to the problems reported in the following thread that already discussed an issue with socket activation for the ssh.service.
SSHd now uses socket-based activation (Ubuntu 22.10 and later) /discourse. ubuntu. com/t/sshd- now-uses- socket- based-activatio n-ubuntu- 22-10-and- later/30189/ 4
https:/