udev race condition with qeth device and bridge_role

Bug #1618463 reported by Andrew McLeod on 2016-08-30
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu on IBM z Systems
Undecided
Dimitri John Ledkov
s390-tools (Ubuntu)
Undecided
Dimitri John Ledkov
Xenial
Undecided
Dimitri John Ledkov
Yakkety
Undecided
Dimitri John Ledkov
Zesty
Undecided
Dimitri John Ledkov

Bug Description

[Impact]

 * bridge_role property setting is racy on boot

 * This results in incorrect bridge mode set on the devices, sometimes, which leads to lack of desired connectivity (e.g. bridging internet to containers)

 * The fix for this issue is to set bridge_role, only after the device is online

 * Unfortunately the udev rules are not regenerated, therefore affected systemd must manually remove and recreate chzdev rules

[Test Case]

 * Remove qeth udev rules from /etc/udev/rules.d/
 * Enable qeth device using chzdev with a non-default bridge_role setting, e.g.:
   chzdev --no-root-update -pVe c003 bridge_role=primary;
 * Reboot and check that bridge_role setting is correctly set in the sysfs, e.g.:
   /sys/devices/qeth/0.0.c003/bridge_role

[Regression Potential]

 * Minimal, the generated udev rules remain the same; the only difference in the generated udev rules is the ordering in setting the bridge_role attribute

[Other Info]

 * Original bug report:

Attempting to set bridge_role = primary with the following command in preseed:

in-target chzdev --no-root-update -pVe c003 bridge_role=primary;

...works, and generates the following udev rule for this device:

https://pastebin.canonical.com/164271/

However, after reboot:

systemd-udevd[2634]: error opening ATTR{/sys/devices/qeth/0.0.c003/bridge_role} for writing: Permission denied

More logging:

https://pastebin.canonical.com/164272/

after the system has booted, we are able to write to the file and set bridge_role to primary:

root@10-13-3-10:/var/log# cat /sys/devices/qeth/0.0.c003/bridge_role
none
root@10-13-3-10:/var/log# echo primary > /sys/devices/qeth/0.0.c003/bridge_role
root@10-13-3-10:/var/log# cat /sys/devices/qeth/0.0.c003/bridge_role
primary

tags: added: s390x
Andrew McLeod (admcleod) on 2016-09-09
tags: added: uosci
Changed in s390-tools (Ubuntu):
assignee: nobody → Dimitri John Ledkov (xnox)
Changed in s390-tools (Ubuntu):
status: New → In Progress
Dimitri John Ledkov (xnox) wrote :

sudo add-apt-repository ppa:ci-train-ppa-service/2038
sudo apt update
sudo apt dist-upgrade

After that please regenerate persistent configuration for your qeth device, e.g.:

$ sudo chzdev -p -d 600
$ sudo chzdev -p -e 600 bridge_role=primary
$ sudo update-initramfs -u

undo any other manual changes to set bridge_role to primary, and finally reboot to test the packages with a proposed fix for your issue.

Please note packages are pending publication, however should be available shortly.

Changed in s390-tools (Ubuntu):
status: In Progress → Fix Committed
description: updated
Changed in s390-tools (Ubuntu Yakkety):
assignee: nobody → Dimitri John Ledkov (xnox)
Changed in s390-tools (Ubuntu Xenial):
assignee: nobody → Dimitri John Ledkov (xnox)
Changed in s390-tools (Ubuntu Zesty):
milestone: none → ubuntu-17.03
Changed in s390-tools (Ubuntu Yakkety):
milestone: none → yakkety-updates
Changed in s390-tools (Ubuntu Xenial):
milestone: none → xenial-updates
Changed in s390-tools (Ubuntu Yakkety):
status: New → In Progress
Changed in s390-tools (Ubuntu Xenial):
status: New → In Progress
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package s390-tools - 1.37.0-0ubuntu2

---------------
s390-tools (1.37.0-0ubuntu2) zesty; urgency=medium

  * chzdev/qeth: set bridge_role after the device is online to avoid a
    race condition when setting qeth attributes on boot. LP: #1618463.

 -- Dimitri John Ledkov <email address hidden> Tue, 14 Mar 2017 09:26:12 +0000

Changed in s390-tools (Ubuntu Zesty):
status: Fix Committed → Fix Released

Hello Andrew, or anyone else affected,

Accepted s390-tools into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/s390-tools/1.36.1-0ubuntu2.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in s390-tools (Ubuntu Yakkety):
status: In Progress → Fix Committed
tags: added: verification-needed
Changed in s390-tools (Ubuntu Xenial):
status: In Progress → Fix Committed
Brian Murray (brian-murray) wrote :

Hello Andrew, or anyone else affected,

Accepted s390-tools into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/s390-tools/1.34.0-0ubuntu8.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in ubuntu-z-systems:
status: New → Fix Committed
assignee: nobody → Dimitri John Ledkov (xnox)
Andrew McLeod (admcleod) wrote :

Hi Brian,

I have tested "s390-tools 1.34.0-0ubuntu8.3" in Xenial and can confirm it fixes the bug in Xenial. I am not currently able to test in Yakkety or Zesty

Thanks!

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers