| 2017-04-27 08:01:02 |
Frank Heimes |
bug |
|
|
added bug |
| 2017-04-27 08:11:16 |
Frank Heimes |
bug task added |
|
openssh (Ubuntu) |
|
| 2017-04-27 13:06:50 |
Dimitri John Ledkov |
openssh (Ubuntu): status |
New |
Triaged |
|
| 2017-04-27 13:06:51 |
Dimitri John Ledkov |
openssh (Ubuntu): importance |
Undecided |
High |
|
| 2017-04-27 13:06:53 |
Dimitri John Ledkov |
openssh (Ubuntu): assignee |
|
Dimitri John Ledkov (xnox) |
|
| 2017-04-27 13:06:56 |
Dimitri John Ledkov |
openssh (Ubuntu): milestone |
|
ubuntu-17.05 |
|
| 2017-04-27 13:07:01 |
Dimitri John Ledkov |
nominated for series |
|
Ubuntu Artful |
|
| 2017-04-27 13:07:01 |
Dimitri John Ledkov |
bug task added |
|
openssh (Ubuntu Artful) |
|
| 2017-04-27 13:07:01 |
Dimitri John Ledkov |
nominated for series |
|
Ubuntu Zesty |
|
| 2017-04-27 13:07:01 |
Dimitri John Ledkov |
bug task added |
|
openssh (Ubuntu Zesty) |
|
| 2017-04-27 13:29:10 |
Frank Heimes |
ubuntu-z-systems: status |
New |
Triaged |
|
| 2017-05-02 15:45:25 |
Dimitri John Ledkov |
openssh (Ubuntu Artful): status |
Triaged |
Fix Released |
|
| 2017-05-02 15:45:33 |
Dimitri John Ledkov |
openssh (Ubuntu Zesty): assignee |
|
Dimitri John Ledkov (xnox) |
|
| 2017-05-02 15:45:35 |
Dimitri John Ledkov |
openssh (Ubuntu Zesty): milestone |
|
zesty-updates |
|
| 2017-05-02 15:45:40 |
Dimitri John Ledkov |
openssh (Ubuntu Zesty): status |
New |
Triaged |
|
| 2017-05-02 15:45:42 |
Dimitri John Ledkov |
openssh (Ubuntu Zesty): importance |
Undecided |
High |
|
| 2017-05-03 15:24:39 |
Dimitri John Ledkov |
description |
short:
after investigations the following commits are needed by openssh-server version 7.4p1 that is part of 17.04:
- 5f1596e11d55539678c41f68aed358628d33d86f
- 9e96b41682aed793fadbea5ccd472f862179fb02
on master branch in https://github.com/openssh/openssh-portable
that belong to openssh 7.5 release notes statement: "sshd(8): Avoid sandbox errors for Linux S390 systems using an ICA crypto coprocessor."
__________
long:
enable z hw crypto support for openssh on an Ubuntu host (zlin42) on s390x like this:
sudo apt-get install openssh-ibmca libica-utils libica2
sudo tee -a /etc/ssl/openssl.cnf < /usr/share/doc/openssl-ibmca/examples/openssl.cnf.sample
sudo sed -i 's/^\(openssl_conf = openssl_def.*$\)/# \1/g' /etc/ssl/openssl.cnf
sudo sed -i '10i openssl_cnf = openssl_def' /etc/ssl/openssl.cnf
afterwards ssh login attempts fail:
$ ssh ubuntu@zlin42
ubuntu@zlin42's password:
Connection to zlin42 closed by remote host.
Connection to zlin42 closed.
the normal logs don't provide any interesting details:
mit log:
Apr 24 12:37:52 zlin42 kernel: [933567.994312] audit: type=1326 audit(1493051872.112:29): auid=4294967295 uid=107 gid=65534 ses=4294967295 pid=25105 comm="sshd" exe="/usr/sbin/sshd" sig=31 arch=80000016 syscall=201 compat=0 ip=0x3ffb8a3fb32 code=0x0
Verbose:
OpenSSH_7.2p2 Ubuntu-4ubuntu2.1, OpenSSL 1.0.2g 1 Mar 2016
debug1: Reading configuration data /home/fheimes/.ssh/config
debug1: /home/fheimes/.ssh/config line 6: Deprecated option "useroaming"
debug1: /home/fheimes/.ssh/config line 7: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to 10.245.208.7 [10.245.208.7] port 22.
debug1: Connection established.
debug1: identity file /home/fheimes/.ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.1
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4p1 Ubuntu-10
debug1: match: OpenSSH_7.4p1 Ubuntu-10 pat OpenSSH* compat 0x04000000
debug1: Authenticating to 10.245.208.7:22 as 'ubuntu'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256@libssh.org
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:ss9j12+jMMKL9u2vxNeb3XjOeH0E9lw24IG5LxUeJXk
debug1: Host '10.245.208.7' is known and matches the ECDSA host key.
debug1: Found key in /home/fheimes/.ssh/known_hosts:87
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /home/fheimes/.ssh/id_rsa
debug1: Authentications that can continue: publickey,password
debug1: Trying private key: /home/fheimes/.ssh/id_dsa
debug1: Trying private key: /home/fheimes/.ssh/id_ecdsa
debug1: Trying private key: /home/fheimes/.ssh/id_ed25519
debug1: Next authentication method: password
ubuntu@10.245.208.7's password:
debug1: Authentication succeeded (password).
Authenticated to 10.245.208.7 ([10.245.208.7]:22).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: pledge: network
debug1: channel 0: free: client-session, nchannels 1
Connection to 10.245.208.7 closed by remote host.
Connection to 10.245.208.7 closed.
Transferred: sent 2084, received 1596 bytes, in 0.0 seconds
Bytes per second: sent 10518567.4, received 8055486.4
debug1: Exit status -1
but loglevel verbose points to this issue:
"fatal: privsep_preauth: preauth child terminated by signal 31"
syslog:
Apr 26 12:39:18 s1lp15 kernel: [12676.655977] audit: type=1326 audit(1493224758.414:99): auid=4294967295 uid=107 gid=65534 ses=4294967295 pid=12380 comm="sshd" exe="/usr/sbin/sshd" sig=31 arch=80000016 syscall=201 compat=0 ip=0x3ff850bfb32 code=0x0
authlog:
Apr 26 12:38:40 s1lp15 sshd[12323]: Connection from 10.172.194.66 port 51512 on 10.245.236.15 port 22
Apr 26 12:38:40 s1lp15 sshd[12323]: Failed publickey for ubuntu from 10.172.194.66 port 51512 ssh2: RSA SHA256:joGsdfW7NbJRkg17sRyXaegoR0iZEdDWdR9Hpbc2KIw
Apr 26 12:38:43 s1lp15 sshd[12323]: Accepted password for ubuntu from 10.172.194.66 port 51512 ssh2
Apr 26 12:38:43 s1lp15 sshd[12323]: fatal: privsep_preauth: preauth child terminated by signal 31
Apr 26 12:39:15 s1lp15 sshd[12379]: Connection from 10.172.194.66 port 51534 on 10.245.236.15 port 22
Apr 26 12:39:16 s1lp15 sshd[12379]: Failed publickey for ubuntu from 10.172.194.66 port 51534 ssh2: RSA SHA256:joGsdfW7NbJRkg17sRyXaegoR0iZEdDWdR9Hpbc2KIw
Apr 26 12:39:18 s1lp15 sshd[12379]: Accepted password for ubuntu from 10.172.194.66 port 51534 ssh2
Apr 26 12:39:18 s1lp15 sshd[12379]: fatal: privsep_preauth: preauth child terminated by signal 31
compared to a system with hw cryto disabled (means ssh working):
syslog:
Apr 26 12:42:04 s1lp15 systemd[1]: Started Session 30 of user ubuntu.
authlog:
Apr 26 12:42:01 s1lp15 sshd[12542]: Connection from 10.172.194.66 port 51658 on 10.245.236.15 port 22
Apr 26 12:42:02 s1lp15 sshd[12542]: Failed publickey for ubuntu from 10.172.194.66 port 51658 ssh2: RSA SHA256:joGsdfW7NbJRkg17sRyXaegoR0iZEdDWdR9Hpbc2KIw
Apr 26 12:42:04 s1lp15 sshd[12542]: Accepted password for ubuntu from 10.172.194.66 port 51658 ssh2
Apr 26 12:42:04 s1lp15 sshd[12542]: pam_unix(sshd:session): session opened for user ubuntu by (uid=0)
Apr 26 12:42:04 s1lp15 systemd-logind[1167]: New session 30 of user ubuntu.
Apr 26 12:42:09 s1lp15 sshd[12542]: User child is on pid 12605
Apr 26 12:42:09 s1lp15 sshd[12605]: Starting session: shell on pts/5 for ubuntu from 10.172.194.66 port 51658 id 0
Workaround:
in /etc/ssh/sshd_config
change:
#UsePrivilegeSeparation sandbox
to:
UsePrivilegeSeparation yes
So it's an issue with the sandbox / seccomp
that got fixed in openssh 7.5
release notes: "sshd(8): Avoid sandbox errors for Linux S390 systems using an ICA crypto coprocessor."
corresponding patches/commits:
master branch https://github.com/openssh/openssh-portable
- 5f1596e11d55539678c41f68aed358628d33d86f
- 9e96b41682aed793fadbea5ccd472f862179fb02 |
[ Impact ]
* Unable to ssh into Ubuntu, using default sshd configuration, when hw acceleration is enabled in openssl.
[ Proposed solution ]
* Cherrypick upstream fixes for:
- sandboxing code on big endian
- allowing hw accel iocls in the sandbox
short:
after investigations the following commits are needed by openssh-server version 7.4p1 that is part of 17.04:
- 5f1596e11d55539678c41f68aed358628d33d86f
- 9e96b41682aed793fadbea5ccd472f862179fb02
on master branch in https://github.com/openssh/openssh-portable
that belong to openssh 7.5 release notes statement: "sshd(8): Avoid sandbox errors for Linux S390 systems using an ICA crypto coprocessor."
__________
[Test case]
long:
enable z hw crypto support for openssh on an Ubuntu host (zlin42) on s390x like this:
sudo apt-get install openssl-ibmca libica-utils libica2
sudo tee -a /etc/ssl/openssl.cnf < /usr/share/doc/openssl-ibmca/examples/openssl.cnf.sample
sudo sed -i 's/^\(openssl_conf = openssl_def.*$\)/# \1/g' /etc/ssl/openssl.cnf
sudo sed -i '10i openssl_cnf = openssl_def' /etc/ssl/openssl.cnf
afterwards ssh login attempts fail:
$ ssh ubuntu@zlin42
ubuntu@zlin42's password:
Connection to zlin42 closed by remote host.
Connection to zlin42 closed.
the normal logs don't provide any interesting details:
mit log:
Apr 24 12:37:52 zlin42 kernel: [933567.994312] audit: type=1326 audit(1493051872.112:29): auid=4294967295 uid=107 gid=65534 ses=4294967295 pid=25105 comm="sshd" exe="/usr/sbin/sshd" sig=31 arch=80000016 syscall=201 compat=0 ip=0x3ffb8a3fb32 code=0x0
Verbose:
OpenSSH_7.2p2 Ubuntu-4ubuntu2.1, OpenSSL 1.0.2g 1 Mar 2016
debug1: Reading configuration data /home/fheimes/.ssh/config
debug1: /home/fheimes/.ssh/config line 6: Deprecated option "useroaming"
debug1: /home/fheimes/.ssh/config line 7: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to 10.245.208.7 [10.245.208.7] port 22.
debug1: Connection established.
debug1: identity file /home/fheimes/.ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.1
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4p1 Ubuntu-10
debug1: match: OpenSSH_7.4p1 Ubuntu-10 pat OpenSSH* compat 0x04000000
debug1: Authenticating to 10.245.208.7:22 as 'ubuntu'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256@libssh.org
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:ss9j12+jMMKL9u2vxNeb3XjOeH0E9lw24IG5LxUeJXk
debug1: Host '10.245.208.7' is known and matches the ECDSA host key.
debug1: Found key in /home/fheimes/.ssh/known_hosts:87
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /home/fheimes/.ssh/id_rsa
debug1: Authentications that can continue: publickey,password
debug1: Trying private key: /home/fheimes/.ssh/id_dsa
debug1: Trying private key: /home/fheimes/.ssh/id_ecdsa
debug1: Trying private key: /home/fheimes/.ssh/id_ed25519
debug1: Next authentication method: password
ubuntu@10.245.208.7's password:
debug1: Authentication succeeded (password).
Authenticated to 10.245.208.7 ([10.245.208.7]:22).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: pledge: network
debug1: channel 0: free: client-session, nchannels 1
Connection to 10.245.208.7 closed by remote host.
Connection to 10.245.208.7 closed.
Transferred: sent 2084, received 1596 bytes, in 0.0 seconds
Bytes per second: sent 10518567.4, received 8055486.4
debug1: Exit status -1
but loglevel verbose points to this issue:
"fatal: privsep_preauth: preauth child terminated by signal 31"
syslog:
Apr 26 12:39:18 s1lp15 kernel: [12676.655977] audit: type=1326 audit(1493224758.414:99): auid=4294967295 uid=107 gid=65534 ses=4294967295 pid=12380 comm="sshd" exe="/usr/sbin/sshd" sig=31 arch=80000016 syscall=201 compat=0 ip=0x3ff850bfb32 code=0x0
authlog:
Apr 26 12:38:40 s1lp15 sshd[12323]: Connection from 10.172.194.66 port 51512 on 10.245.236.15 port 22
Apr 26 12:38:40 s1lp15 sshd[12323]: Failed publickey for ubuntu from 10.172.194.66 port 51512 ssh2: RSA SHA256:joGsdfW7NbJRkg17sRyXaegoR0iZEdDWdR9Hpbc2KIw
Apr 26 12:38:43 s1lp15 sshd[12323]: Accepted password for ubuntu from 10.172.194.66 port 51512 ssh2
Apr 26 12:38:43 s1lp15 sshd[12323]: fatal: privsep_preauth: preauth child terminated by signal 31
Apr 26 12:39:15 s1lp15 sshd[12379]: Connection from 10.172.194.66 port 51534 on 10.245.236.15 port 22
Apr 26 12:39:16 s1lp15 sshd[12379]: Failed publickey for ubuntu from 10.172.194.66 port 51534 ssh2: RSA SHA256:joGsdfW7NbJRkg17sRyXaegoR0iZEdDWdR9Hpbc2KIw
Apr 26 12:39:18 s1lp15 sshd[12379]: Accepted password for ubuntu from 10.172.194.66 port 51534 ssh2
Apr 26 12:39:18 s1lp15 sshd[12379]: fatal: privsep_preauth: preauth child terminated by signal 31
compared to a system with hw cryto disabled (means ssh working):
syslog:
Apr 26 12:42:04 s1lp15 systemd[1]: Started Session 30 of user ubuntu.
authlog:
Apr 26 12:42:01 s1lp15 sshd[12542]: Connection from 10.172.194.66 port 51658 on 10.245.236.15 port 22
Apr 26 12:42:02 s1lp15 sshd[12542]: Failed publickey for ubuntu from 10.172.194.66 port 51658 ssh2: RSA SHA256:joGsdfW7NbJRkg17sRyXaegoR0iZEdDWdR9Hpbc2KIw
Apr 26 12:42:04 s1lp15 sshd[12542]: Accepted password for ubuntu from 10.172.194.66 port 51658 ssh2
Apr 26 12:42:04 s1lp15 sshd[12542]: pam_unix(sshd:session): session opened for user ubuntu by (uid=0)
Apr 26 12:42:04 s1lp15 systemd-logind[1167]: New session 30 of user ubuntu.
Apr 26 12:42:09 s1lp15 sshd[12542]: User child is on pid 12605
Apr 26 12:42:09 s1lp15 sshd[12605]: Starting session: shell on pts/5 for ubuntu from 10.172.194.66 port 51658 id 0
Workaround:
in /etc/ssh/sshd_config
change:
#UsePrivilegeSeparation sandbox
to:
UsePrivilegeSeparation yes
So it's an issue with the sandbox / seccomp
that got fixed in openssh 7.5
release notes: "sshd(8): Avoid sandbox errors for Linux S390 systems using an ICA crypto coprocessor."
corresponding patches/commits:
master branch https://github.com/openssh/openssh-portable
- 5f1596e11d55539678c41f68aed358628d33d86f
- 9e96b41682aed793fadbea5ccd472f862179fb02 |
|
| 2017-05-04 10:04:18 |
Dimitri John Ledkov |
openssh (Ubuntu Zesty): status |
Triaged |
In Progress |
|
| 2017-05-04 12:20:24 |
Frank Heimes |
ubuntu-z-systems: status |
Triaged |
In Progress |
|
| 2017-05-04 12:29:36 |
bugproxy |
tags |
s390x |
architecture-s39064 bugnameltc-153940 s390x severity-high targetmilestone-inin1704 |
|
| 2017-05-04 19:58:26 |
Brian Murray |
openssh (Ubuntu Zesty): status |
In Progress |
Fix Committed |
|
| 2017-05-04 19:58:29 |
Brian Murray |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2017-05-04 19:58:35 |
Brian Murray |
bug |
|
|
added subscriber SRU Verification |
| 2017-05-04 19:58:38 |
Brian Murray |
tags |
architecture-s39064 bugnameltc-153940 s390x severity-high targetmilestone-inin1704 |
architecture-s39064 bugnameltc-153940 s390x severity-high targetmilestone-inin1704 verification-needed |
|
| 2017-05-05 07:21:10 |
Frank Heimes |
ubuntu-z-systems: status |
In Progress |
Fix Committed |
|
| 2017-05-05 12:27:05 |
Dimitri John Ledkov |
tags |
architecture-s39064 bugnameltc-153940 s390x severity-high targetmilestone-inin1704 verification-needed |
architecture-s39064 bugnameltc-153940 s390x severity-high targetmilestone-inin1704 verification-failed |
|
| 2017-05-05 12:53:45 |
Dimitri John Ledkov |
openssh (Ubuntu Artful): status |
Fix Released |
Triaged |
|
| 2017-05-09 09:40:21 |
bugproxy |
attachment added |
|
patch to enable geteuid syscall in sshd sandbox on s390 https://bugs.launchpad.net/bugs/1686618/+attachment/4873913/+files/0004-Add-geteuid-syscall-for-Linux-s390.patch |
|
| 2017-05-10 06:39:35 |
bugproxy |
attachment added |
|
Upstream patch to enable geteuid syscall for Linux on s390 https://bugs.launchpad.net/bugs/1686618/+attachment/4874469/+files/0001-Permit-geteuid-syscall-for-Linux-on-s390.patch |
|
| 2017-05-22 12:24:24 |
Dimitri John Ledkov |
openssh (Ubuntu Artful): status |
Triaged |
Fix Committed |
|
| 2017-05-22 12:24:26 |
Dimitri John Ledkov |
openssh (Ubuntu Zesty): status |
Fix Committed |
In Progress |
|
| 2017-05-22 18:32:34 |
Launchpad Janitor |
openssh (Ubuntu Artful): status |
Fix Committed |
Fix Released |
|
| 2017-07-21 09:29:29 |
Dimitri John Ledkov |
openssh (Ubuntu Artful): status |
Fix Released |
Triaged |
|
| 2017-07-21 09:29:31 |
Dimitri John Ledkov |
openssh (Ubuntu Artful): importance |
High |
Critical |
|
| 2017-07-21 09:29:34 |
Dimitri John Ledkov |
openssh (Ubuntu Zesty): status |
In Progress |
Confirmed |
|
| 2017-07-21 09:29:36 |
Dimitri John Ledkov |
openssh (Ubuntu Zesty): importance |
High |
Critical |
|
| 2017-07-21 09:29:38 |
Dimitri John Ledkov |
ubuntu-z-systems: status |
Fix Committed |
Triaged |
|
| 2017-07-21 09:29:41 |
Dimitri John Ledkov |
ubuntu-z-systems: importance |
High |
Critical |
|
| 2017-07-28 14:10:08 |
Dimitri John Ledkov |
openssh (Ubuntu Artful): status |
Triaged |
Fix Committed |
|
| 2017-07-28 14:47:02 |
Frank Heimes |
ubuntu-z-systems: status |
Triaged |
In Progress |
|
| 2017-07-28 15:29:59 |
bugproxy |
attachment added |
|
Upstream patch to enable geteuid syscall for Linux on s390 https://bugs.launchpad.net/bugs/1686618/+attachment/4923114/+files/0001-Permit-geteuid-syscall-for-Linux-on-s390.patch |
|
| 2017-07-28 20:09:41 |
bugproxy |
bug watch added |
|
https://bugzilla.mindrot.org/show_bug.cgi?id=2752 |
|
| 2017-07-30 06:57:38 |
Launchpad Janitor |
openssh (Ubuntu Artful): status |
Fix Committed |
Fix Released |
|
| 2017-09-29 17:26:58 |
Francis Ginther |
tags |
architecture-s39064 bugnameltc-153940 s390x severity-high targetmilestone-inin1704 verification-failed |
architecture-s39064 bugnameltc-153940 id-59a6de69fde9c920947b3d4b s390x severity-high targetmilestone-inin1704 verification-failed |
|
| 2017-09-29 17:58:43 |
Francis Ginther |
tags |
architecture-s39064 bugnameltc-153940 id-59a6de69fde9c920947b3d4b s390x severity-high targetmilestone-inin1704 verification-failed |
architecture-s39064 bugnameltc-153940 id-597a835aabb9be94fe80eb45 id-59a6de69fde9c920947b3d4b s390x severity-high targetmilestone-inin1704 verification-failed |
|
| 2018-01-17 13:44:59 |
Frank Heimes |
openssh (Ubuntu Zesty): status |
Confirmed |
Invalid |
|
| 2018-01-17 13:45:03 |
Frank Heimes |
ubuntu-z-systems: status |
In Progress |
Fix Released |
|
| 2018-01-17 14:00:03 |
bugproxy |
tags |
architecture-s39064 bugnameltc-153940 id-597a835aabb9be94fe80eb45 id-59a6de69fde9c920947b3d4b s390x severity-high targetmilestone-inin1704 verification-failed |
architecture-s39064 bugnameltc-153940 id-597a835aabb9be94fe80eb45 id-59a6de69fde9c920947b3d4b s390x severity-high targetmilestone-inin1704 |
|
| 2019-03-14 11:26:01 |
Frank Heimes |
description |
[ Impact ]
* Unable to ssh into Ubuntu, using default sshd configuration, when hw acceleration is enabled in openssl.
[ Proposed solution ]
* Cherrypick upstream fixes for:
- sandboxing code on big endian
- allowing hw accel iocls in the sandbox
short:
after investigations the following commits are needed by openssh-server version 7.4p1 that is part of 17.04:
- 5f1596e11d55539678c41f68aed358628d33d86f
- 9e96b41682aed793fadbea5ccd472f862179fb02
on master branch in https://github.com/openssh/openssh-portable
that belong to openssh 7.5 release notes statement: "sshd(8): Avoid sandbox errors for Linux S390 systems using an ICA crypto coprocessor."
__________
[Test case]
long:
enable z hw crypto support for openssh on an Ubuntu host (zlin42) on s390x like this:
sudo apt-get install openssl-ibmca libica-utils libica2
sudo tee -a /etc/ssl/openssl.cnf < /usr/share/doc/openssl-ibmca/examples/openssl.cnf.sample
sudo sed -i 's/^\(openssl_conf = openssl_def.*$\)/# \1/g' /etc/ssl/openssl.cnf
sudo sed -i '10i openssl_cnf = openssl_def' /etc/ssl/openssl.cnf
afterwards ssh login attempts fail:
$ ssh ubuntu@zlin42
ubuntu@zlin42's password:
Connection to zlin42 closed by remote host.
Connection to zlin42 closed.
the normal logs don't provide any interesting details:
mit log:
Apr 24 12:37:52 zlin42 kernel: [933567.994312] audit: type=1326 audit(1493051872.112:29): auid=4294967295 uid=107 gid=65534 ses=4294967295 pid=25105 comm="sshd" exe="/usr/sbin/sshd" sig=31 arch=80000016 syscall=201 compat=0 ip=0x3ffb8a3fb32 code=0x0
Verbose:
OpenSSH_7.2p2 Ubuntu-4ubuntu2.1, OpenSSL 1.0.2g 1 Mar 2016
debug1: Reading configuration data /home/fheimes/.ssh/config
debug1: /home/fheimes/.ssh/config line 6: Deprecated option "useroaming"
debug1: /home/fheimes/.ssh/config line 7: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to 10.245.208.7 [10.245.208.7] port 22.
debug1: Connection established.
debug1: identity file /home/fheimes/.ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.1
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4p1 Ubuntu-10
debug1: match: OpenSSH_7.4p1 Ubuntu-10 pat OpenSSH* compat 0x04000000
debug1: Authenticating to 10.245.208.7:22 as 'ubuntu'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256@libssh.org
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:ss9j12+jMMKL9u2vxNeb3XjOeH0E9lw24IG5LxUeJXk
debug1: Host '10.245.208.7' is known and matches the ECDSA host key.
debug1: Found key in /home/fheimes/.ssh/known_hosts:87
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /home/fheimes/.ssh/id_rsa
debug1: Authentications that can continue: publickey,password
debug1: Trying private key: /home/fheimes/.ssh/id_dsa
debug1: Trying private key: /home/fheimes/.ssh/id_ecdsa
debug1: Trying private key: /home/fheimes/.ssh/id_ed25519
debug1: Next authentication method: password
ubuntu@10.245.208.7's password:
debug1: Authentication succeeded (password).
Authenticated to 10.245.208.7 ([10.245.208.7]:22).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: pledge: network
debug1: channel 0: free: client-session, nchannels 1
Connection to 10.245.208.7 closed by remote host.
Connection to 10.245.208.7 closed.
Transferred: sent 2084, received 1596 bytes, in 0.0 seconds
Bytes per second: sent 10518567.4, received 8055486.4
debug1: Exit status -1
but loglevel verbose points to this issue:
"fatal: privsep_preauth: preauth child terminated by signal 31"
syslog:
Apr 26 12:39:18 s1lp15 kernel: [12676.655977] audit: type=1326 audit(1493224758.414:99): auid=4294967295 uid=107 gid=65534 ses=4294967295 pid=12380 comm="sshd" exe="/usr/sbin/sshd" sig=31 arch=80000016 syscall=201 compat=0 ip=0x3ff850bfb32 code=0x0
authlog:
Apr 26 12:38:40 s1lp15 sshd[12323]: Connection from 10.172.194.66 port 51512 on 10.245.236.15 port 22
Apr 26 12:38:40 s1lp15 sshd[12323]: Failed publickey for ubuntu from 10.172.194.66 port 51512 ssh2: RSA SHA256:joGsdfW7NbJRkg17sRyXaegoR0iZEdDWdR9Hpbc2KIw
Apr 26 12:38:43 s1lp15 sshd[12323]: Accepted password for ubuntu from 10.172.194.66 port 51512 ssh2
Apr 26 12:38:43 s1lp15 sshd[12323]: fatal: privsep_preauth: preauth child terminated by signal 31
Apr 26 12:39:15 s1lp15 sshd[12379]: Connection from 10.172.194.66 port 51534 on 10.245.236.15 port 22
Apr 26 12:39:16 s1lp15 sshd[12379]: Failed publickey for ubuntu from 10.172.194.66 port 51534 ssh2: RSA SHA256:joGsdfW7NbJRkg17sRyXaegoR0iZEdDWdR9Hpbc2KIw
Apr 26 12:39:18 s1lp15 sshd[12379]: Accepted password for ubuntu from 10.172.194.66 port 51534 ssh2
Apr 26 12:39:18 s1lp15 sshd[12379]: fatal: privsep_preauth: preauth child terminated by signal 31
compared to a system with hw cryto disabled (means ssh working):
syslog:
Apr 26 12:42:04 s1lp15 systemd[1]: Started Session 30 of user ubuntu.
authlog:
Apr 26 12:42:01 s1lp15 sshd[12542]: Connection from 10.172.194.66 port 51658 on 10.245.236.15 port 22
Apr 26 12:42:02 s1lp15 sshd[12542]: Failed publickey for ubuntu from 10.172.194.66 port 51658 ssh2: RSA SHA256:joGsdfW7NbJRkg17sRyXaegoR0iZEdDWdR9Hpbc2KIw
Apr 26 12:42:04 s1lp15 sshd[12542]: Accepted password for ubuntu from 10.172.194.66 port 51658 ssh2
Apr 26 12:42:04 s1lp15 sshd[12542]: pam_unix(sshd:session): session opened for user ubuntu by (uid=0)
Apr 26 12:42:04 s1lp15 systemd-logind[1167]: New session 30 of user ubuntu.
Apr 26 12:42:09 s1lp15 sshd[12542]: User child is on pid 12605
Apr 26 12:42:09 s1lp15 sshd[12605]: Starting session: shell on pts/5 for ubuntu from 10.172.194.66 port 51658 id 0
Workaround:
in /etc/ssh/sshd_config
change:
#UsePrivilegeSeparation sandbox
to:
UsePrivilegeSeparation yes
So it's an issue with the sandbox / seccomp
that got fixed in openssh 7.5
release notes: "sshd(8): Avoid sandbox errors for Linux S390 systems using an ICA crypto coprocessor."
corresponding patches/commits:
master branch https://github.com/openssh/openssh-portable
- 5f1596e11d55539678c41f68aed358628d33d86f
- 9e96b41682aed793fadbea5ccd472f862179fb02 |
[ Impact ]
* Unable to ssh into Ubuntu, using default sshd configuration, when hw acceleration is enabled in openssl.
[ Proposed solution ]
* Cherrypick upstream fixes for:
- sandboxing code on big endian
- allowing hw accel iocls in the sandbox
short:
after investigations the following commits are needed by openssh-server version 7.4p1 that is part of 17.04:
- 5f1596e11d55539678c41f68aed358628d33d86f
- 9e96b41682aed793fadbea5ccd472f862179fb02
on master branch in https://github.com/openssh/openssh-portable
that belong to openssh 7.5 release notes statement: "sshd(8): Avoid sandbox errors for Linux S390 systems using an ICA crypto coprocessor."
__________
[Test case]
long:
enable z hw crypto support for openssh on an Ubuntu host (zlin42) on s390x like this:
sudo apt-get install openssl-ibmca libica-utils libica2
sudo tee -a /etc/ssl/openssl.cnf < /usr/share/doc/openssl-ibmca/examples/openssl.cnf.sample
sudo sed -i 's/^\(openssl_conf = openssl_def.*$\)/# \1/g' /etc/ssl/openssl.cnf
sudo sed -i '10i openssl_conf = openssl_def' /etc/ssl/openssl.cnf
afterwards ssh login attempts fail:
$ ssh ubuntu@zlin42
ubuntu@zlin42's password:
Connection to zlin42 closed by remote host.
Connection to zlin42 closed.
the normal logs don't provide any interesting details:
mit log:
Apr 24 12:37:52 zlin42 kernel: [933567.994312] audit: type=1326 audit(1493051872.112:29): auid=4294967295 uid=107 gid=65534 ses=4294967295 pid=25105 comm="sshd" exe="/usr/sbin/sshd" sig=31 arch=80000016 syscall=201 compat=0 ip=0x3ffb8a3fb32 code=0x0
Verbose:
OpenSSH_7.2p2 Ubuntu-4ubuntu2.1, OpenSSL 1.0.2g 1 Mar 2016
debug1: Reading configuration data /home/fheimes/.ssh/config
debug1: /home/fheimes/.ssh/config line 6: Deprecated option "useroaming"
debug1: /home/fheimes/.ssh/config line 7: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to 10.245.208.7 [10.245.208.7] port 22.
debug1: Connection established.
debug1: identity file /home/fheimes/.ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/fheimes/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.1
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4p1 Ubuntu-10
debug1: match: OpenSSH_7.4p1 Ubuntu-10 pat OpenSSH* compat 0x04000000
debug1: Authenticating to 10.245.208.7:22 as 'ubuntu'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256@libssh.org
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:ss9j12+jMMKL9u2vxNeb3XjOeH0E9lw24IG5LxUeJXk
debug1: Host '10.245.208.7' is known and matches the ECDSA host key.
debug1: Found key in /home/fheimes/.ssh/known_hosts:87
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /home/fheimes/.ssh/id_rsa
debug1: Authentications that can continue: publickey,password
debug1: Trying private key: /home/fheimes/.ssh/id_dsa
debug1: Trying private key: /home/fheimes/.ssh/id_ecdsa
debug1: Trying private key: /home/fheimes/.ssh/id_ed25519
debug1: Next authentication method: password
ubuntu@10.245.208.7's password:
debug1: Authentication succeeded (password).
Authenticated to 10.245.208.7 ([10.245.208.7]:22).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: pledge: network
debug1: channel 0: free: client-session, nchannels 1
Connection to 10.245.208.7 closed by remote host.
Connection to 10.245.208.7 closed.
Transferred: sent 2084, received 1596 bytes, in 0.0 seconds
Bytes per second: sent 10518567.4, received 8055486.4
debug1: Exit status -1
but loglevel verbose points to this issue:
"fatal: privsep_preauth: preauth child terminated by signal 31"
syslog:
Apr 26 12:39:18 s1lp15 kernel: [12676.655977] audit: type=1326 audit(1493224758.414:99): auid=4294967295 uid=107 gid=65534 ses=4294967295 pid=12380 comm="sshd" exe="/usr/sbin/sshd" sig=31 arch=80000016 syscall=201 compat=0 ip=0x3ff850bfb32 code=0x0
authlog:
Apr 26 12:38:40 s1lp15 sshd[12323]: Connection from 10.172.194.66 port 51512 on 10.245.236.15 port 22
Apr 26 12:38:40 s1lp15 sshd[12323]: Failed publickey for ubuntu from 10.172.194.66 port 51512 ssh2: RSA SHA256:joGsdfW7NbJRkg17sRyXaegoR0iZEdDWdR9Hpbc2KIw
Apr 26 12:38:43 s1lp15 sshd[12323]: Accepted password for ubuntu from 10.172.194.66 port 51512 ssh2
Apr 26 12:38:43 s1lp15 sshd[12323]: fatal: privsep_preauth: preauth child terminated by signal 31
Apr 26 12:39:15 s1lp15 sshd[12379]: Connection from 10.172.194.66 port 51534 on 10.245.236.15 port 22
Apr 26 12:39:16 s1lp15 sshd[12379]: Failed publickey for ubuntu from 10.172.194.66 port 51534 ssh2: RSA SHA256:joGsdfW7NbJRkg17sRyXaegoR0iZEdDWdR9Hpbc2KIw
Apr 26 12:39:18 s1lp15 sshd[12379]: Accepted password for ubuntu from 10.172.194.66 port 51534 ssh2
Apr 26 12:39:18 s1lp15 sshd[12379]: fatal: privsep_preauth: preauth child terminated by signal 31
compared to a system with hw cryto disabled (means ssh working):
syslog:
Apr 26 12:42:04 s1lp15 systemd[1]: Started Session 30 of user ubuntu.
authlog:
Apr 26 12:42:01 s1lp15 sshd[12542]: Connection from 10.172.194.66 port 51658 on 10.245.236.15 port 22
Apr 26 12:42:02 s1lp15 sshd[12542]: Failed publickey for ubuntu from 10.172.194.66 port 51658 ssh2: RSA SHA256:joGsdfW7NbJRkg17sRyXaegoR0iZEdDWdR9Hpbc2KIw
Apr 26 12:42:04 s1lp15 sshd[12542]: Accepted password for ubuntu from 10.172.194.66 port 51658 ssh2
Apr 26 12:42:04 s1lp15 sshd[12542]: pam_unix(sshd:session): session opened for user ubuntu by (uid=0)
Apr 26 12:42:04 s1lp15 systemd-logind[1167]: New session 30 of user ubuntu.
Apr 26 12:42:09 s1lp15 sshd[12542]: User child is on pid 12605
Apr 26 12:42:09 s1lp15 sshd[12605]: Starting session: shell on pts/5 for ubuntu from 10.172.194.66 port 51658 id 0
Workaround:
in /etc/ssh/sshd_config
change:
#UsePrivilegeSeparation sandbox
to:
UsePrivilegeSeparation yes
So it's an issue with the sandbox / seccomp
that got fixed in openssh 7.5
release notes: "sshd(8): Avoid sandbox errors for Linux S390 systems using an ICA crypto coprocessor."
corresponding patches/commits:
master branch https://github.com/openssh/openssh-portable
- 5f1596e11d55539678c41f68aed358628d33d86f
- 9e96b41682aed793fadbea5ccd472f862179fb02 |
|
