StrongSwan with GCM and large packet sizes produces unstable behavior

Bug #1828035 reported by bugproxy
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu on IBM z Systems
Fix Released
High
Canonical Server
linux (Ubuntu)
Fix Released
Undecided
Unassigned
strongswan (Ubuntu)
Invalid
Undecided
Skipper Bug Screeners

Bug Description

StrongSwan with GCM and large packet sizes produces unstable behavior when used in Linux native environment.

---uname output---
Linux ubu01 4.15.0-42-generic #45-Ubuntu SMP Thu Nov 15 19:29:11 UTC 2018 s390x s390x s390x GNU/Linux

Machine Type = 3906 / M04 (z14), LPAR (dedicated)

---Debugger---
A debugger is not configured

---Steps to Reproduce---
On two separate machines (Linux native), install StrongSwan and on both machines, configure the encryption with aes128gcm8 for IPsec.

Then run the following command from one of the machine:

```
$# ping <other_machine_ip> -s 1024
```

Small packet sizes are working as expected. However, anything large (around 1024 bytes or more) are sometimes returning wrong values, or the packets are getting lost. This problem does not occur for ciphers not involving GCM.

Userspace tool common name: StrongSwan

The userspace tool has the following bit modes: both

Userspace package: StrongSwan

Userspace tool obtained from project website: na

-Attach ltrace and strace of userspace application.

Revision history for this message
bugproxy (bugproxy) wrote : [l|s]trace ping <ip> -s 1300 # ltrace and strace of the ping command

Default Comment by Bridge

tags: added: architecture-s39064 bugnameltc-177435 severity-high targetmilestone-inin18041
Changed in ubuntu:
assignee: nobody → Skipper Bug Screeners (skipper-screen-team)
affects: ubuntu → strongswan (Ubuntu)
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
importance: Undecided → High
assignee: nobody → Canonical Server Team (canonical-server)
status: New → Triaged
Revision history for this message
Dimitri John Ledkov (xnox) wrote :

What version of strongswan was used?
Have you been able to reproduce it on Eoan with strongswan 5.7.2?
Have you reported this upstream at https://wiki.strongswan.org/projects/strongswan/issues ?

Changed in strongswan (Ubuntu):
status: New → Incomplete
Revision history for this message
Tobias Brunner (tobias-strongswan) wrote :

It's unlikely that this is a strongSwan issue as IPsec is handled by the Linux kernel. It's more likely a kernel bug related to that particular architecture.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Thanks Tobias for the advise, adding a kernel Task to pull in the kernel team as well (confirmed to stop the bot from asking for logs).

@bugproxy:
"configure the encryption with aes128gcm8 for IPsec" can probably be done in more than one way. To avoid wasting effort could you just provide the config files you used on both ends
This is -not- overriding the question of xnox if it works in a newer strongswan.

@bugproxy:
As the kernel might be important (see the comment above), another check is to verify if the same occurs with the HWE kernel "linux-generic-hwe-18.04" currently at 4.18.0.18.68.
Furthermore you should try 5.1 from [1]
This is -not- overriding the question of xnox if it works in a newer strongswan either.

[1]: https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.1/

Changed in linux (Ubuntu):
status: New → Confirmed
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: Triaged → Confirmed
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2019-05-08 09:20 EDT-------
A little bit more information:

```
$# ipsec version
Linux strongSwan U5.6.2/K4.15.0-42-generic
Institute for Internet Technologies and Applications
University of Applied Sciences Rapperswil, Switzerland
See 'ipsec --copyright' for copyright information.
```

```
$# cat /etc/ipsec.conf
config setup
conn main<ip>
type=transport
authby=psk
left=%any #<ip>
leftsubnet=<ip>
right=%any
rightsubnet=<ip>
auto=route
compress=no
esp=aes128gcm8
fragmentation=yes
keyexchange=ikev2
```

Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-05-24 07:01 EDT-------
Does the problem goes away when you unload the aes_s390 module and also prevent loading it. (maybe use a blacklist statement in modprobe.conf)

Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-05-27 04:43 EDT-------
Can you kindly give me a command do so? I get the following error when I try:

```
$# rmmod -f aes_s390
rmmod: ERROR: ../libkmod/libkmod-module.c:793 kmod_module_remove_module() could not remove 'aes_s390': Resource temporarily unavailable
rmmod: ERROR: could not remove module aes_s390: Resource temporarily unavailable
```

Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: Confirmed → Incomplete
Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-05-29 06:55 EDT-------
Can you create a file
/etc/modprobe.d/blacklist.conf

with
blacklist aes_s390
as content
and then reboot?

Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-06-26 08:06 EDT-------
When we remove the aes_s390 module, the GCM works and the error with the ping command is no longer visible.

------- Comment From <email address hidden> 2019-06-26 08:09 EDT-------
I think this is then fixed with https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/arch/s390/crypto?id=bef9f0ba300a55d79a69aa172156072182176515

commit bef9f0ba300a55d79a69aa172156072182176515

s390/crypto: fix gcm-aes-s390 selftest failures

The current kernel uses improved crypto selftests. These
tests showed that the current implementation of gcm-aes-s390
is not able to deal with chunks of output buffers which are
not a multiple of 16 bytes. This patch introduces a rework
of the gcm aes s390 scatter walk handling which now is able
to handle any input and output scatter list chunk sizes
correctly.

Code has been verified by the crypto selftests, the tcrypt
kernel module and additional tests ran via the af_alg interface.

Cc: <email address hidden>
Reported-by: Julian Wiedmann <email address hidden>
Reviewed-by: Patrick Steuer <email address hidden>
Signed-off-by: Harald Freudenberger <email address hidden>
Signed-off-by: Heiko Carstens <email address hidden>

------- Comment From <email address hidden> 2019-06-26 08:10 EDT-------
*** Bug 177436 has been marked as a duplicate of this bug. ***

Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-06-26 11:06 EDT-------
Canonical: any chance to get a test kernel build with the outlined patch applied?

Revision history for this message
Frank Heimes (fheimes) wrote :

Since there is a kernel SRU request in progress to include that patch anyway,
( addressed in LP 1832623 '[UBUNTU] kernel: Fix gcm-aes-s390 wrong scatter-gather list processing' - https://bugs.launchpad.net/ubuntu-z-systems/+bug/1832623 )
I suggest to wait until this hits -proposed and is ready for verification - if that is okay for you?!

Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-06-28 05:04 EDT-------
Is there an outlook when this SRU will be available in proposed?

Revision history for this message
Frank Heimes (fheimes) wrote :

A rough outlook is: in ~ less than 2 weeks - it's planned to come with the next proposed kernel.
(If everything runs smoothly with the kernel SRU, but it looks good - there is already one Ack.)

Revision history for this message
Frank Heimes (fheimes) wrote :

According to LP 1832623 (especially https://bugs.launchpad.net/ubuntu-z-systems/+bug/1832623/comments/6) the latest kernel available in bionic-proposed (4.15.0-55) includes the 'gcm-aes-s390' patch and is ready for test.

See also bionic kernel changelog: https://launchpad.net/ubuntu/+source/linux/+changelog and search for 'gcm-aes-s390' or 1832623)

Changed in strongswan (Ubuntu):
status: Incomplete → In Progress
status: In Progress → Incomplete
Changed in ubuntu-z-systems:
status: Incomplete → Confirmed
Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-07-09 04:27 EDT-------
With the new bionic-proposed kernel (4.15.0-55), the ping errors with GCM are eliminated and things look fine.

Revision history for this message
Frank Heimes (fheimes) wrote :

Thx for testing and confirming - this kernel (4.15.0-55) will soon be rolled out and copied from the proposed to the update pocket, which then will fix this issue - hence changing status to Fix Committed.

Changed in linux (Ubuntu):
status: Confirmed → Fix Committed
Changed in ubuntu-z-systems:
status: Confirmed → Fix Committed
Revision history for this message
Frank Heimes (fheimes) wrote :

The 'gcm-aes-s390' patch landed today in bionic's release pocket.
The work was done based on the kernel SRU of LP 1832623, which is now Fix Released.
Hence I mark this ticket as Fix Released as well.

Changed in linux (Ubuntu):
status: Fix Committed → Fix Released
Changed in ubuntu-z-systems:
status: Fix Committed → Fix Released
Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-07-24 05:52 EDT-------
IBM bugzilla status -> closed, Fix Released by Canonical

Changed in strongswan (Ubuntu):
status: Incomplete → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.