| 2016-11-28 13:44:58 |
Amad Ali |
bug |
|
|
added bug |
| 2016-11-28 13:46:16 |
Amad Ali |
affects |
qemu (Ubuntu) |
ebtables (Ubuntu) |
|
| 2016-11-29 12:31:29 |
Amad Ali |
attachment added |
|
debdiff-ebtables2.0.10.4-3ubuntu2 https://bugs.launchpad.net/ubuntu/+source/ebtables/+bug/1645324/+attachment/4784670/+files/debdiff-ebtables2.0.10.4-3ubuntu2 |
|
| 2016-11-29 12:45:53 |
Amad Ali |
information type |
Public |
Public Security |
|
| 2016-11-29 12:46:24 |
Amad Ali |
ebtables (Ubuntu): milestone |
|
trusty-updates |
|
| 2016-11-29 12:47:06 |
Amad Ali |
ebtables (Ubuntu): milestone |
trusty-updates |
|
|
| 2016-11-29 12:50:43 |
Amad Ali |
tags |
|
sts-sponsor |
|
| 2016-11-29 16:29:17 |
Ubuntu Foundations Team Bug Bot |
tags |
sts-sponsor |
patch sts-sponsor |
|
| 2016-11-29 16:29:27 |
Ubuntu Foundations Team Bug Bot |
bug |
|
|
added subscriber Ubuntu Security Sponsors Team |
| 2016-12-01 16:32:59 |
Tyler Hicks |
information type |
Public Security |
Public |
|
| 2016-12-05 08:51:53 |
Christian Ehrhardt |
bug |
|
|
added subscriber Ubuntu Server Team |
| 2016-12-06 13:33:01 |
Louis Bouchard |
nominated for series |
|
Ubuntu Trusty |
|
| 2016-12-06 13:33:01 |
Louis Bouchard |
bug task added |
|
ebtables (Ubuntu Trusty) |
|
| 2016-12-11 03:21:43 |
Mathew Hodson |
ebtables (Ubuntu): importance |
Undecided |
Medium |
|
| 2016-12-11 03:21:45 |
Mathew Hodson |
ebtables (Ubuntu Trusty): importance |
Undecided |
Medium |
|
| 2016-12-19 21:31:39 |
Nish Aravamudan |
bug watch added |
|
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=813760 |
|
| 2016-12-19 21:31:39 |
Nish Aravamudan |
bug task added |
|
ebtables (Debian) |
|
| 2016-12-19 21:31:58 |
Nish Aravamudan |
ebtables (Ubuntu): status |
New |
Fix Released |
|
| 2016-12-19 22:13:34 |
Bug Watch Updater |
ebtables (Debian): status |
Unknown |
Fix Released |
|
| 2017-01-01 18:24:26 |
Dominique Poulain |
bug |
|
|
added subscriber Dominique Poulain |
| 2017-01-04 12:42:20 |
Marc Deslauriers |
bug |
|
|
added subscriber Ubuntu Sponsors Team |
| 2017-01-04 12:42:22 |
Marc Deslauriers |
removed subscriber Ubuntu Security Sponsors Team |
|
|
|
| 2017-01-05 10:27:41 |
Michael Cunningham |
bug |
|
|
added subscriber Michael Cunningham |
| 2017-01-11 22:40:23 |
Amad Ali |
attachment added |
|
debdiff-ebtables2.0.10.4-3ubuntu1.1 https://bugs.launchpad.net/ubuntu/+source/ebtables/+bug/1645324/+attachment/4803098/+files/debdiff-ebtables2.0.10.4-3ubuntu1.1 |
|
| 2017-01-27 11:00:40 |
Louis Bouchard |
ebtables (Ubuntu Trusty): status |
New |
Triaged |
|
| 2017-01-27 11:00:45 |
Louis Bouchard |
ebtables (Ubuntu Trusty): status |
Triaged |
In Progress |
|
| 2017-02-08 10:58:54 |
Robie Basak |
removed subscriber Ubuntu Sponsors Team |
|
|
|
| 2017-02-08 10:59:02 |
Robie Basak |
bug |
|
|
added subscriber Robie Basak |
| 2017-02-08 11:03:36 |
Robie Basak |
ebtables (Ubuntu Trusty): status |
In Progress |
Incomplete |
|
| 2017-02-23 10:54:18 |
Louis Bouchard |
tags |
patch sts-sponsor |
patch |
|
| 2017-03-06 21:38:56 |
Dragan S. |
attachment added |
|
fix_ds-lp1645324.2.debdiff https://bugs.launchpad.net/ubuntu/+source/ebtables/+bug/1645324/+attachment/4832573/+files/fix_ds-lp1645324.2.debdiff |
|
| 2017-03-06 23:41:07 |
Dragan S. |
description |
[Impact]
ebtables uses a lock file, if it is started with --concurrent parameter.
Everytime ebtables is run in trusty, a lock file is created in /var/lib/ebtables/.
If the system crashes the lock file might exist on startup. This makes libvirtd and other services to hang after startup that depends on ebtables.
Lockfile should probably be moved to /run instead, especially that the initscript is not taking care of cleaning that lockfile at startup.
[Test Case]
1.Create a lockfile manually with "sudo touch /var/lib/ebtables/lock"
2.reboot
3.libvird hanging, try to connect to qemu:///system
testcase2:
1. $ while true; do /usr/sbin/ebtables --concurrent -L; done
2. hard reboot VM
3. likely that the lock file is present under /var/lib/ebtables
4. libvird hanging, try to connect to qemu:///system
[Regression Potential]
There should be minimal regression potential in this patch
[Original Text]
libvirtd is hanging after startup due to ebtables lock file -from an earlier run- remains intact when the system reboots.
Same issue is happening than it is reported here: https://bugzilla.redhat.com/show_bug.cgi?id=1290327 when the system boots.
After booting the system, It's not possible connect to the qemu-service.
- libvirt daemon tried to obtain a lock:
[pid 20966] read(24, "Trying to obtain lock /var/lib/e"..., 1024) = 45
[pid 20966] poll([{fd=22, events=POLLIN}, {fd=24, events=POLLIN}], 2, 4294967295) = 1 ([{fd=24, revents=POLLIN}])
[pid 20966] read(24, "Trying to obtain lock /var/lib/e"..., 1024) = 45
[pid 20966] poll([{fd=22, events=POLLIN}, {fd=24, events=POLLIN}], 2, 4294967295) = 1 ([{fd=24, revents=POLLIN}])
[pid 20966] read(24, "Trying to obtain lock /var/lib/e"..., 1024) = 45
[pid 20966] poll([{fd=22, events=POLLIN}, {fd=24, events=POLLIN}], 2, 4294967295^CProcess 20916 detached
- there was a file named 'lock' in /var/lib/ebtables directory with timestamp 14:54
- ebtables was configured:
* Ebtables support available, number of installed rules [ OK ]
(other nodes appeared to be in the same state from ebtables point of view, but without the lock file)
- I removed the lock file and libvirt started to work instantly - the lock obtain messages have disappeared from the trace and virsh commands are working
- at 14:54 the host was booting up. According to the logs, there were other reboots after that one, but the lock file remained intact (at least the timestamp was not updated).
Could you please suggest a solution to be sure that ebtables lock file is removed during boot? |
[Impact]
* ebtables uses creation of a file with an exclusive flag
as a lock to synchronize table modification when used
with --concurrent parameter.
* If ebtables crashes it will leave a stale lock file.
This will prevent another copy of ebtables from running,
and indirectly any other service that depends on ebtables
will also be affected.
* This change adds support for real locks that get
cleaned up if a process exits or crashes.
[Test Case]
* Test Case1:
1. $ sudo touch /var/lib/ebtables/lock"
2. $ sudo ebtables --concurrent -L
3. ebtables can't acquire a lock
* Test Case 2:
1. $ while true; do /usr/sbin/ebtables --concurrent -L; done
2. hard reboot VM
3. likely that the lock file is present under /var/lib/ebtables
4. libvird hanging, try to connect to qemu:///system
[Regression Potential]
* Normal Use:
There is no regression potential during normal use and
operation of ebtables.
* Package Upgrade:
There is a very very small regression potential during the package
upgrade to the latest version. Once the package is upgraded that
potential is gone. It is a very small potential because several
things have to happen in a very small time frame and in an exact
order since ebtables is not a resident program like a daemon:
1. ebtables is launched during package upgrade AND
2. new ebtables binary has not yet been written to disk AND
3. it is launched with --concurent switch AND
4. another ebtables with new binary is launched AND
5. it is launched with --concurent switch AND
6. the first ebtables copy hasn't exited yet AND
7. both copies of ebtables are launched with a WRITE command AND
8. both copies of ebtables are manipulating the same resource.
Then one of the binaries could potentially fail, but once the old
binary exits the potential is gone so subsequent re-runs of
ebtables will succeed.
[Original Text]
libvirtd is hanging after startup due to ebtables lock file -from an earlier run- remains intact when the system reboots.
Same issue is happening than it is reported here: https://bugzilla.redhat.com/show_bug.cgi?id=1290327 when the system boots.
After booting the system, It's not possible connect to the qemu-service.
- libvirt daemon tried to obtain a lock:
[pid 20966] read(24, "Trying to obtain lock /var/lib/e"..., 1024) = 45
[pid 20966] poll([{fd=22, events=POLLIN}, {fd=24, events=POLLIN}], 2, 4294967295) = 1 ([{fd=24, revents=POLLIN}])
[pid 20966] read(24, "Trying to obtain lock /var/lib/e"..., 1024) = 45
[pid 20966] poll([{fd=22, events=POLLIN}, {fd=24, events=POLLIN}], 2, 4294967295) = 1 ([{fd=24, revents=POLLIN}])
[pid 20966] read(24, "Trying to obtain lock /var/lib/e"..., 1024) = 45
[pid 20966] poll([{fd=22, events=POLLIN}, {fd=24, events=POLLIN}], 2, 4294967295^CProcess 20916 detached
- there was a file named 'lock' in /var/lib/ebtables directory with timestamp 14:54
- ebtables was configured:
* Ebtables support available, number of installed rules [ OK ]
(other nodes appeared to be in the same state from ebtables point of view, but without the lock file)
- I removed the lock file and libvirt started to work instantly - the lock obtain messages have disappeared from the trace and virsh commands are working
- at 14:54 the host was booting up. According to the logs, there were other reboots after that one, but the lock file remained intact (at least the timestamp was not updated).
Could you please suggest a solution to be sure that ebtables lock file is removed during boot? |
|
| 2017-03-06 23:42:07 |
Dragan S. |
ebtables (Ubuntu Trusty): assignee |
|
Dragan S. (dragan-s) |
|
| 2017-03-06 23:42:17 |
Dragan S. |
ebtables (Ubuntu Trusty): status |
Incomplete |
In Progress |
|
| 2017-03-20 15:06:13 |
Louis Bouchard |
tags |
patch |
patch sts-sponsor sts-sru |
|
| 2017-03-23 11:48:50 |
Edward Hope-Morley |
tags |
patch sts-sponsor sts-sru |
patch sts-sponsor sts-sru-needed |
|
| 2017-03-31 15:38:24 |
Louis Bouchard |
tags |
patch sts-sponsor sts-sru-needed |
patch sts-sponsor sts-sponsor-caribou sts-sru-needed |
|
| 2017-04-03 09:10:57 |
Louis Bouchard |
tags |
patch sts-sponsor sts-sponsor-caribou sts-sru-needed |
patch sts-sru-needed |
|
| 2017-04-18 22:41:41 |
Dan Streetman |
nominated for series |
|
Ubuntu Zesty |
|
| 2017-04-19 00:23:25 |
Dragan S. |
attachment added |
|
Zesty Patch https://bugs.launchpad.net/ubuntu/+source/ebtables/+bug/1645324/+attachment/4864509/+files/zesty-fix-real-locking.debdiff |
|
| 2017-04-19 12:49:14 |
Robie Basak |
summary |
ebtables: Lock file should be moved from /var/lib/ebtables to /run |
ebtables: Lock file handling has races |
|
| 2017-04-19 12:49:18 |
Robie Basak |
ebtables (Ubuntu): status |
Fix Released |
Triaged |
|
| 2017-04-19 14:01:28 |
Dragan S. |
bug watch added |
|
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=860590 |
|
| 2017-04-19 14:53:10 |
Eric Desrochers |
bug task added |
|
ebtables (Ubuntu Zesty) |
|
| 2017-04-19 14:54:11 |
Dragan S. |
ebtables (Ubuntu Zesty): assignee |
|
Dragan S. (dragan-s) |
|
| 2017-05-02 20:10:25 |
Dragan S. |
ebtables (Debian): status |
Fix Released |
Unknown |
|
| 2017-05-02 20:10:25 |
Dragan S. |
ebtables (Debian): remote watch |
Debian Bug tracker #813760 |
Debian Bug tracker #860590 |
|
| 2017-05-03 17:57:07 |
Bug Watch Updater |
ebtables (Debian): status |
Unknown |
New |
|
| 2017-05-16 01:31:12 |
Eric Desrochers |
nominated for series |
|
Ubuntu Artful |
|
| 2017-05-16 01:31:12 |
Eric Desrochers |
bug task added |
|
ebtables (Ubuntu Artful) |
|
| 2017-05-16 01:31:30 |
Eric Desrochers |
ebtables (Ubuntu Artful): assignee |
Dragan S. (dragan-s) |
|
|
| 2017-05-16 01:39:23 |
Eric Desrochers |
ebtables (Ubuntu Artful): assignee |
|
Eric Desrochers (slashd) |
|
| 2017-05-16 01:39:27 |
Eric Desrochers |
ebtables (Ubuntu Artful): status |
Triaged |
In Progress |
|
| 2017-05-16 02:02:17 |
Eric Desrochers |
attachment added |
|
artful_lp1645324.debdiff https://bugs.launchpad.net/ubuntu/+source/ebtables/+bug/1645324/+attachment/4877194/+files/artful_lp1645324.debdiff |
|
| 2017-05-16 13:14:17 |
Eric Desrochers |
bug |
|
|
added subscriber Ubuntu Sponsors Team |
| 2017-05-16 13:17:50 |
Eric Desrochers |
description |
[Impact]
* ebtables uses creation of a file with an exclusive flag
as a lock to synchronize table modification when used
with --concurrent parameter.
* If ebtables crashes it will leave a stale lock file.
This will prevent another copy of ebtables from running,
and indirectly any other service that depends on ebtables
will also be affected.
* This change adds support for real locks that get
cleaned up if a process exits or crashes.
[Test Case]
* Test Case1:
1. $ sudo touch /var/lib/ebtables/lock"
2. $ sudo ebtables --concurrent -L
3. ebtables can't acquire a lock
* Test Case 2:
1. $ while true; do /usr/sbin/ebtables --concurrent -L; done
2. hard reboot VM
3. likely that the lock file is present under /var/lib/ebtables
4. libvird hanging, try to connect to qemu:///system
[Regression Potential]
* Normal Use:
There is no regression potential during normal use and
operation of ebtables.
* Package Upgrade:
There is a very very small regression potential during the package
upgrade to the latest version. Once the package is upgraded that
potential is gone. It is a very small potential because several
things have to happen in a very small time frame and in an exact
order since ebtables is not a resident program like a daemon:
1. ebtables is launched during package upgrade AND
2. new ebtables binary has not yet been written to disk AND
3. it is launched with --concurent switch AND
4. another ebtables with new binary is launched AND
5. it is launched with --concurent switch AND
6. the first ebtables copy hasn't exited yet AND
7. both copies of ebtables are launched with a WRITE command AND
8. both copies of ebtables are manipulating the same resource.
Then one of the binaries could potentially fail, but once the old
binary exits the potential is gone so subsequent re-runs of
ebtables will succeed.
[Original Text]
libvirtd is hanging after startup due to ebtables lock file -from an earlier run- remains intact when the system reboots.
Same issue is happening than it is reported here: https://bugzilla.redhat.com/show_bug.cgi?id=1290327 when the system boots.
After booting the system, It's not possible connect to the qemu-service.
- libvirt daemon tried to obtain a lock:
[pid 20966] read(24, "Trying to obtain lock /var/lib/e"..., 1024) = 45
[pid 20966] poll([{fd=22, events=POLLIN}, {fd=24, events=POLLIN}], 2, 4294967295) = 1 ([{fd=24, revents=POLLIN}])
[pid 20966] read(24, "Trying to obtain lock /var/lib/e"..., 1024) = 45
[pid 20966] poll([{fd=22, events=POLLIN}, {fd=24, events=POLLIN}], 2, 4294967295) = 1 ([{fd=24, revents=POLLIN}])
[pid 20966] read(24, "Trying to obtain lock /var/lib/e"..., 1024) = 45
[pid 20966] poll([{fd=22, events=POLLIN}, {fd=24, events=POLLIN}], 2, 4294967295^CProcess 20916 detached
- there was a file named 'lock' in /var/lib/ebtables directory with timestamp 14:54
- ebtables was configured:
* Ebtables support available, number of installed rules [ OK ]
(other nodes appeared to be in the same state from ebtables point of view, but without the lock file)
- I removed the lock file and libvirt started to work instantly - the lock obtain messages have disappeared from the trace and virsh commands are working
- at 14:54 the host was booting up. According to the logs, there were other reboots after that one, but the lock file remained intact (at least the timestamp was not updated).
Could you please suggest a solution to be sure that ebtables lock file is removed during boot? |
[Impact]
* ebtables uses creation of a file with an exclusive flag
as a lock to synchronize table modification when used
with --concurrent parameter.
* If ebtables crashes it will leave a stale lock file.
This will prevent another copy of ebtables from running,
and indirectly any other service that depends on ebtables
will also be affected.
* This change adds support for real locks that get
cleaned up if a process exits or crashes.
[Test Case]
* Test Case1:
1. $ sudo touch /var/lib/ebtables/lock"
2. $ sudo ebtables --concurrent -L
3. ebtables can't acquire a lock
* Test Case 2:
1. $ while true; do /usr/sbin/ebtables --concurrent -L; done
2. hard reboot VM
3. likely that the lock file is present under /var/lib/ebtables
4. libvird hanging, try to connect to qemu:///system
[Regression Potential]
* Normal Use:
There is no regression potential during normal use and
operation of ebtables.
* Package Upgrade:
There is a very very small regression potential during the package
upgrade to the latest version. Once the package is upgraded that
potential is gone. It is a very small potential because several
things have to happen in a very small time frame and in an exact
order since ebtables is not a resident program like a daemon:
1. ebtables is launched during package upgrade AND
2. new ebtables binary has not yet been written to disk AND
3. it is launched with --concurent switch AND
4. another ebtables with new binary is launched AND
5. it is launched with --concurent switch AND
6. the first ebtables copy hasn't exited yet AND
7. both copies of ebtables are launched with a WRITE command AND
8. both copies of ebtables are manipulating the same resource.
Then one of the binaries could potentially fail, but once the old
binary exits the potential is gone so subsequent re-runs of
ebtables will succeed.
* Dragan's patch has been submitted to Debian via :
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=860590
* Note that the ebtables upstream project is nearly dead. Nowadays, all the development is now happening in nft project which is intended to be replacement.
[Original Text]
libvirtd is hanging after startup due to ebtables lock file -from an earlier run- remains intact when the system reboots.
Same issue is happening than it is reported here: https://bugzilla.redhat.com/show_bug.cgi?id=1290327 when the system boots.
After booting the system, It's not possible connect to the qemu-service.
- libvirt daemon tried to obtain a lock:
[pid 20966] read(24, "Trying to obtain lock /var/lib/e"..., 1024) = 45
[pid 20966] poll([{fd=22, events=POLLIN}, {fd=24, events=POLLIN}], 2, 4294967295) = 1 ([{fd=24, revents=POLLIN}])
[pid 20966] read(24, "Trying to obtain lock /var/lib/e"..., 1024) = 45
[pid 20966] poll([{fd=22, events=POLLIN}, {fd=24, events=POLLIN}], 2, 4294967295) = 1 ([{fd=24, revents=POLLIN}])
[pid 20966] read(24, "Trying to obtain lock /var/lib/e"..., 1024) = 45
[pid 20966] poll([{fd=22, events=POLLIN}, {fd=24, events=POLLIN}], 2, 4294967295^CProcess 20916 detached
- there was a file named 'lock' in /var/lib/ebtables directory with timestamp 14:54
- ebtables was configured:
* Ebtables support available, number of installed rules [ OK ]
(other nodes appeared to be in the same state from ebtables point of view, but without the lock file)
- I removed the lock file and libvirt started to work instantly - the lock obtain messages have disappeared from the trace and virsh commands are working
- at 14:54 the host was booting up. According to the logs, there were other reboots after that one, but the lock file remained intact (at least the timestamp was not updated).
Could you please suggest a solution to be sure that ebtables lock file is removed during boot? |
|
| 2017-05-29 15:46:59 |
Eric Desrochers |
ebtables (Ubuntu Trusty): assignee |
Dragan S. (dragan-s) |
Eric Desrochers (slashd) |
|
| 2017-05-29 15:52:20 |
Eric Desrochers |
bug |
|
|
added subscriber Szilard Cserey |
| 2017-05-29 17:49:24 |
Launchpad Janitor |
ebtables (Ubuntu Artful): status |
In Progress |
Fix Released |
|
| 2017-05-29 18:22:01 |
Eric Desrochers |
nominated for series |
|
Ubuntu Xenial |
|
| 2017-05-29 18:22:01 |
Eric Desrochers |
bug task added |
|
ebtables (Ubuntu Xenial) |
|
| 2017-05-29 18:22:26 |
Eric Desrochers |
nominated for series |
|
Ubuntu Yakkety |
|
| 2017-05-29 18:22:26 |
Eric Desrochers |
bug task added |
|
ebtables (Ubuntu Yakkety) |
|
| 2017-05-29 18:55:56 |
Eric Desrochers |
ebtables (Ubuntu Zesty): status |
Triaged |
In Progress |
|
| 2017-05-29 18:55:59 |
Eric Desrochers |
ebtables (Ubuntu Yakkety): status |
New |
In Progress |
|
| 2017-05-29 18:56:04 |
Eric Desrochers |
ebtables (Ubuntu Zesty): assignee |
Dragan S. (dragan-s) |
Eric Desrochers (slashd) |
|
| 2017-05-29 18:56:10 |
Eric Desrochers |
ebtables (Ubuntu Yakkety): assignee |
|
Eric Desrochers (slashd) |
|
| 2017-05-29 18:56:12 |
Eric Desrochers |
ebtables (Ubuntu Yakkety): importance |
Undecided |
Medium |
|
| 2017-05-29 18:56:25 |
Eric Desrochers |
ebtables (Ubuntu Xenial): assignee |
|
Dragan S. (dragan-s) |
|
| 2017-05-29 18:56:27 |
Eric Desrochers |
ebtables (Ubuntu Xenial): importance |
Undecided |
Medium |
|
| 2017-05-29 19:02:22 |
Eric Desrochers |
tags |
patch sts-sru-needed |
patch sts-sru-done |
|
| 2017-05-29 20:27:36 |
Eric Desrochers |
ebtables (Ubuntu Xenial): status |
New |
In Progress |
|
| 2017-05-29 20:33:16 |
Eric Desrochers |
removed subscriber Ubuntu Sponsors Team |
|
|
|
| 2017-05-29 20:33:25 |
Eric Desrochers |
tags |
patch sts-sru-done |
sts sts-sru-done |
|
| 2017-06-01 10:28:15 |
Łukasz Zemczak |
ebtables (Ubuntu Zesty): status |
In Progress |
Fix Committed |
|
| 2017-06-01 10:28:20 |
Łukasz Zemczak |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2017-06-01 10:28:23 |
Łukasz Zemczak |
bug |
|
|
added subscriber SRU Verification |
| 2017-06-01 10:28:31 |
Łukasz Zemczak |
tags |
sts sts-sru-done |
sts sts-sru-done verification-needed |
|
| 2017-06-01 10:30:28 |
Łukasz Zemczak |
ebtables (Ubuntu Yakkety): status |
In Progress |
Fix Committed |
|
| 2017-06-01 10:33:58 |
Łukasz Zemczak |
ebtables (Ubuntu Xenial): status |
In Progress |
Fix Committed |
|
| 2017-06-01 10:35:59 |
Łukasz Zemczak |
ebtables (Ubuntu Trusty): status |
In Progress |
Fix Committed |
|
| 2017-06-02 11:51:00 |
Szilard Cserey |
description |
[Impact]
* ebtables uses creation of a file with an exclusive flag
as a lock to synchronize table modification when used
with --concurrent parameter.
* If ebtables crashes it will leave a stale lock file.
This will prevent another copy of ebtables from running,
and indirectly any other service that depends on ebtables
will also be affected.
* This change adds support for real locks that get
cleaned up if a process exits or crashes.
[Test Case]
* Test Case1:
1. $ sudo touch /var/lib/ebtables/lock"
2. $ sudo ebtables --concurrent -L
3. ebtables can't acquire a lock
* Test Case 2:
1. $ while true; do /usr/sbin/ebtables --concurrent -L; done
2. hard reboot VM
3. likely that the lock file is present under /var/lib/ebtables
4. libvird hanging, try to connect to qemu:///system
[Regression Potential]
* Normal Use:
There is no regression potential during normal use and
operation of ebtables.
* Package Upgrade:
There is a very very small regression potential during the package
upgrade to the latest version. Once the package is upgraded that
potential is gone. It is a very small potential because several
things have to happen in a very small time frame and in an exact
order since ebtables is not a resident program like a daemon:
1. ebtables is launched during package upgrade AND
2. new ebtables binary has not yet been written to disk AND
3. it is launched with --concurent switch AND
4. another ebtables with new binary is launched AND
5. it is launched with --concurent switch AND
6. the first ebtables copy hasn't exited yet AND
7. both copies of ebtables are launched with a WRITE command AND
8. both copies of ebtables are manipulating the same resource.
Then one of the binaries could potentially fail, but once the old
binary exits the potential is gone so subsequent re-runs of
ebtables will succeed.
* Dragan's patch has been submitted to Debian via :
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=860590
* Note that the ebtables upstream project is nearly dead. Nowadays, all the development is now happening in nft project which is intended to be replacement.
[Original Text]
libvirtd is hanging after startup due to ebtables lock file -from an earlier run- remains intact when the system reboots.
Same issue is happening than it is reported here: https://bugzilla.redhat.com/show_bug.cgi?id=1290327 when the system boots.
After booting the system, It's not possible connect to the qemu-service.
- libvirt daemon tried to obtain a lock:
[pid 20966] read(24, "Trying to obtain lock /var/lib/e"..., 1024) = 45
[pid 20966] poll([{fd=22, events=POLLIN}, {fd=24, events=POLLIN}], 2, 4294967295) = 1 ([{fd=24, revents=POLLIN}])
[pid 20966] read(24, "Trying to obtain lock /var/lib/e"..., 1024) = 45
[pid 20966] poll([{fd=22, events=POLLIN}, {fd=24, events=POLLIN}], 2, 4294967295) = 1 ([{fd=24, revents=POLLIN}])
[pid 20966] read(24, "Trying to obtain lock /var/lib/e"..., 1024) = 45
[pid 20966] poll([{fd=22, events=POLLIN}, {fd=24, events=POLLIN}], 2, 4294967295^CProcess 20916 detached
- there was a file named 'lock' in /var/lib/ebtables directory with timestamp 14:54
- ebtables was configured:
* Ebtables support available, number of installed rules [ OK ]
(other nodes appeared to be in the same state from ebtables point of view, but without the lock file)
- I removed the lock file and libvirt started to work instantly - the lock obtain messages have disappeared from the trace and virsh commands are working
- at 14:54 the host was booting up. According to the logs, there were other reboots after that one, but the lock file remained intact (at least the timestamp was not updated).
Could you please suggest a solution to be sure that ebtables lock file is removed during boot? |
[Impact]
* ebtables uses creation of a file with an exclusive flag
as a lock to synchronize table modification when used
with --concurrent parameter.
* If ebtables crashes it will leave a stale lock file.
This will prevent another copy of ebtables from running,
and indirectly any other service that depends on ebtables
will also be affected.
* This change adds support for real locks that get
cleaned up if a process exits or crashes.
[Test Case]
* Test Case1:
1. $ sudo touch /var/lib/ebtables/lock"
2. $ sudo ebtables --concurrent -L
3. ebtables can't acquire a lock
* Test Case 2:
1. $ while true; do /usr/sbin/ebtables --concurrent -L; done
2. hard reboot VM
3. likely that the lock file is present under /var/lib/ebtables
4. libvird hanging, try to connect to qemu:///system
[Regression Potential]
* Normal Use:
There is no regression potential during normal use and
operation of ebtables.
* Package Upgrade:
There is a very very small regression potential during the package
upgrade to the latest version. Once the package is upgraded that
potential is gone. It is a very small potential because several
things have to happen in a very small time frame and in an exact
order since ebtables is not a resident program like a daemon:
1. ebtables is launched during package upgrade AND
2. new ebtables binary has not yet been written to disk AND
3. it is launched with --concurent switch AND
4. another ebtables with new binary is launched AND
5. it is launched with --concurent switch AND
6. the first ebtables copy hasn't exited yet AND
7. both copies of ebtables are launched with a WRITE command AND
8. both copies of ebtables are manipulating the same resource.
Then one of the binaries could potentially fail, but once the old
binary exits the potential is gone so subsequent re-runs of
ebtables will succeed.
* Dragan's patch has been submitted to Debian via :
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=860590
* Note that the ebtables upstream project is nearly dead. Nowadays, all the development is now happening in nft project which is intended to be replacement.
[Original Text]
libvirtd is hanging after startup due to ebtables lock file -from an earlier run- remains intact when the system reboots.
Same issue is happening than it is reported here: https://bugzilla.redhat.com/show_bug.cgi?id=1290327 when the system boots.
After booting the system, It's not possible connect to the qemu-service.
- libvirt daemon tried to obtain a lock:
[pid 20966] read(24, "Trying to obtain lock /var/lib/e"..., 1024) = 45
[pid 20966] poll([{fd=22, events=POLLIN}, {fd=24, events=POLLIN}], 2, 4294967295) = 1 ([{fd=24, revents=POLLIN}])
[pid 20966] read(24, "Trying to obtain lock /var/lib/e"..., 1024) = 45
[pid 20966] poll([{fd=22, events=POLLIN}, {fd=24, events=POLLIN}], 2, 4294967295) = 1 ([{fd=24, revents=POLLIN}])
[pid 20966] read(24, "Trying to obtain lock /var/lib/e"..., 1024) = 45
[pid 20966] poll([{fd=22, events=POLLIN}, {fd=24, events=POLLIN}], 2, 4294967295^CProcess 20916 detached
- there was a file named 'lock' in /var/lib/ebtables directory with timestamp 14:54
- ebtables was configured:
* Ebtables support available, number of installed rules [ OK ]
(other nodes appeared to be in the same state from ebtables point of view, but without the lock file)
- I removed the lock file and libvirt started to work instantly - the lock obtain messages have disappeared from the trace and virsh commands are working
- at 14:54 the host was booting up. According to the logs, there were other reboots after that one, but the lock file remained intact (at least the timestamp was not updated).
Could you please suggest a solution to be sure that ebtables lock file is removed during boot? |
|
| 2017-06-02 12:38:42 |
Szilard Cserey |
tags |
sts sts-sru-done verification-needed |
sts sts-sru-done verification-done |
|
| 2017-06-02 13:39:25 |
Eric Desrochers |
ebtables (Ubuntu Xenial): assignee |
Dragan S. (dragan-s) |
Eric Desrochers (slashd) |
|
| 2017-06-09 07:47:56 |
Łukasz Zemczak |
tags |
sts sts-sru-done verification-done |
sts sts-sru-done verification-done-trusty verification-done-xenial verification-done-yakkety verification-done-zesty |
|
| 2017-06-13 14:52:45 |
Launchpad Janitor |
ebtables (Ubuntu Zesty): status |
Fix Committed |
Fix Released |
|
| 2017-06-13 14:52:49 |
Łukasz Zemczak |
removed subscriber Ubuntu Stable Release Updates Team |
|
|
|
| 2017-06-13 14:53:01 |
Launchpad Janitor |
ebtables (Ubuntu Yakkety): status |
Fix Committed |
Fix Released |
|
| 2017-06-13 14:53:24 |
Launchpad Janitor |
ebtables (Ubuntu Xenial): status |
Fix Committed |
Fix Released |
|
| 2017-06-13 14:53:44 |
Launchpad Janitor |
ebtables (Ubuntu Trusty): status |
Fix Committed |
Fix Released |
|
| 2019-03-08 16:38:56 |
Bug Watch Updater |
ebtables (Debian): status |
New |
Fix Released |
|
