Add SIGPIPE handler to subprocess execution in rootwrap and utils.execute

Bug #1053364 reported by Thierry Carrez on 2012-09-20
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Cinder
Medium
Thierry Carrez
Folsom
Medium
Chuck Short
OpenStack Compute (nova)
Medium
Thierry Carrez
Folsom
Medium
Chuck Short
neutron
Medium
Thierry Carrez
Folsom
Medium
Chuck Short
cinder (Ubuntu)
Quantal
Undecided
Unassigned
nova (Ubuntu)
Quantal
Undecided
Unassigned
quantum (Ubuntu)
Medium
Unassigned
Quantal
Undecided
Unassigned

Bug Description

As explained here: http://www.chiark.greenend.org.uk/ucgi/~cjwatson/blosxom/2009-07-02-python-sigpipe.html

Python ignores SIGPIPE on startup, because it prefers to check every write and raise an IOError exception rather than taking the signal. This is all well and good for Python itself, but most Unix subprocesses don't expect to work this way. Thus, when you are creating subprocesses from Python, it is very important to set SIGPIPE back to the default action. Before I realised this was necessary, I wrote code that caused serious data loss due to a child process carrying on out of control after its parent process died!

Our execution stuff should make use of that.

Thierry Carrez (ttx) on 2012-09-20
Changed in nova:
importance: High → Medium
Changed in cinder:
status: New → Triaged
importance: Undecided → Medium
Changed in quantum:
status: New → Triaged
importance: Undecided → Medium

Fix proposed to branch: master
Review: https://review.openstack.org/13346

Changed in nova:
status: Triaged → In Progress

Fix proposed to branch: master
Review: https://review.openstack.org/13347

Changed in cinder:
assignee: nobody → Thierry Carrez (ttx)
status: Triaged → In Progress
Thierry Carrez (ttx) on 2012-09-20
summary: - Add SIGPIPE handler to subprocess execution
+ Add SIGPIPE handler to subprocess execution in rootwrap and
+ utils.execute
Thierry Carrez (ttx) wrote :

Sigh, Quantum makes use of subprocess.Popen all over the place. Looks like it should be cleaned up to use a common function before we apply this.

Same for Nova in
nova/virt/baremetal/tilera.py
nova/virt/hyperv/volumeutils.py
and plenty of things under plugins/xenserver

Fix proposed to branch: master
Review: https://review.openstack.org/13348

Changed in quantum:
assignee: nobody → Thierry Carrez (ttx)
status: Triaged → In Progress
Thierry Carrez (ttx) wrote :

Created bug 1053381 (Quantum) and bug 1053382 (Nova) to make sure all subprocess.Popen calls are routed through the same function(s)

Reviewed: https://review.openstack.org/13348
Committed: http://github.com/openstack/quantum/commit/30fe8a4e3c53438bbfb24ba882b3a37c65440e72
Submitter: Jenkins
Branch: master

commit 30fe8a4e3c53438bbfb24ba882b3a37c65440e72
Author: Thierry Carrez <email address hidden>
Date: Thu Sep 20 14:42:53 2012 +0200

    Restore SIGPIPE default action for subprocesses

    Python ignores SIGPIPE on startup, because it prefers to check every
    write and raise an IOError exception rather than taking the signal. Most
    Unix subprocesses don't expect to work this way. This patch (adapted
    from Colin Watson's post at http://tinyurl.com/2a7mzh5) sets SIGPIPE
    back to the default action for quantum.agent.linux.utils.execute,
    quantum.common.utils.execute and quantum-rootwrap created subprocesses.

    Fixes bug 1053364

    Change-Id: Ib805f1f8846c245b75a5ea64278c840b823c1fb2

Changed in quantum:
status: In Progress → Fix Committed
dan wendlandt (danwent) wrote :

thanks ttx, I assume this should be targeted for RC2 in quantum?

Reviewed: https://review.openstack.org/13346
Committed: http://github.com/openstack/nova/commit/a694b9e5adec8236ce8b2cd4832f8dc4912de6fc
Submitter: Jenkins
Branch: master

commit a694b9e5adec8236ce8b2cd4832f8dc4912de6fc
Author: Thierry Carrez <email address hidden>
Date: Thu Sep 20 14:11:38 2012 +0200

    Restore SIGPIPE default action for subprocesses

    Python ignores SIGPIPE on startup, because it prefers to check every
    write and raise an IOError exception rather than taking the signal. Most
    Unix subprocesses don't expect to work this way. This patch (adapted
    from Colin Watson's post at http://tinyurl.com/2a7mzh5) sets SIGPIPE
    back to the default action for nova.utils.execute and nova-rootwrap
    created subprocesses.

    Fixes bug 1053364

    Change-Id: I17e1629bb4ef4268515c6734ddb6e12746739c52

Changed in nova:
status: In Progress → Fix Committed

Reviewed: https://review.openstack.org/13347
Committed: http://github.com/openstack/cinder/commit/730ebab71e11812b6b5990180e7bd9cabfe237c4
Submitter: Jenkins
Branch: master

commit 730ebab71e11812b6b5990180e7bd9cabfe237c4
Author: Thierry Carrez <email address hidden>
Date: Thu Sep 20 14:21:00 2012 +0200

    Restore SIGPIPE default action for subprocesses

    Python ignores SIGPIPE on startup, because it prefers to check every
    write and raise an IOError exception rather than taking the signal. Most
    Unix subprocesses don't expect to work this way. This patch (adapted
    from Colin Watson's post at http://tinyurl.com/2a7mzh5) sets SIGPIPE
    back to the default action for cinder.utils.execute and cinder-rootwrap
    created subprocesses.

    Fixes bug 1053364

    Change-Id: I4b3307bd2f0f5d0da529d8b7d80fabae28c57732

Changed in cinder:
status: In Progress → Fix Committed
tags: removed: folsom-rc-potential
Thierry Carrez (ttx) wrote :

Let's live with it a few weeks in Grizzly before backporting it

tags: added: folsom-backport-potential

Reviewed: https://review.openstack.org/14084
Committed: http://github.com/openstack/quantum/commit/3bbf281b15866fc6aecdf8283bca696e5c646300
Submitter: Jenkins
Branch: stable/folsom

commit 3bbf281b15866fc6aecdf8283bca696e5c646300
Author: Thierry Carrez <email address hidden>
Date: Thu Sep 20 14:42:53 2012 +0200

    Restore SIGPIPE default action for subprocesses

    Python ignores SIGPIPE on startup, because it prefers to check every
    write and raise an IOError exception rather than taking the signal. Most
    Unix subprocesses don't expect to work this way. This patch (adapted
    from Colin Watson's post at http://tinyurl.com/2a7mzh5) sets SIGPIPE
    back to the default action for quantum.agent.linux.utils.execute,
    quantum.common.utils.execute and quantum-rootwrap created subprocesses.

    Fixes bug 1053364

    Change-Id: Ib805f1f8846c245b75a5ea64278c840b823c1fb2

Reviewed: https://review.openstack.org/14085
Committed: http://github.com/openstack/nova/commit/d4d166503a93a458fabdcd919882d29fc590ad7b
Submitter: Jenkins
Branch: stable/folsom

commit d4d166503a93a458fabdcd919882d29fc590ad7b
Author: Thierry Carrez <email address hidden>
Date: Thu Sep 20 14:11:38 2012 +0200

    Restore SIGPIPE default action for subprocesses

    Python ignores SIGPIPE on startup, because it prefers to check every
    write and raise an IOError exception rather than taking the signal. Most
    Unix subprocesses don't expect to work this way. This patch (adapted
    from Colin Watson's post at http://tinyurl.com/2a7mzh5) sets SIGPIPE
    back to the default action for nova.utils.execute and nova-rootwrap
    created subprocesses.

    Fixes bug 1053364

    Change-Id: I17e1629bb4ef4268515c6734ddb6e12746739c52

Reviewed: https://review.openstack.org/14086
Committed: http://github.com/openstack/cinder/commit/d12d4b62076c2c73bab295a328d9d254c16d5569
Submitter: Jenkins
Branch: stable/folsom

commit d12d4b62076c2c73bab295a328d9d254c16d5569
Author: Thierry Carrez <email address hidden>
Date: Thu Sep 20 14:21:00 2012 +0200

    Restore SIGPIPE default action for subprocesses

    Python ignores SIGPIPE on startup, because it prefers to check every
    write and raise an IOError exception rather than taking the signal. Most
    Unix subprocesses don't expect to work this way. This patch (adapted
    from Colin Watson's post at http://tinyurl.com/2a7mzh5) sets SIGPIPE
    back to the default action for cinder.utils.execute and cinder-rootwrap
    created subprocesses.

    Fixes bug 1053364

    Change-Id: I4b3307bd2f0f5d0da529d8b7d80fabae28c57732

Chuck Short (zulcss) on 2012-10-24
tags: removed: folsom-backport-potential
Chuck Short (zulcss) on 2012-11-06
Changed in quantum (Ubuntu):
status: New → Fix Released
Changed in quantum (Ubuntu Precise):
status: New → Confirmed
Akihiro Motoki (amotoki) on 2012-11-13
tags: added: in-stable-folsom
Thierry Carrez (ttx) on 2012-11-21
Changed in nova:
milestone: none → grizzly-1
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2012-11-21
Changed in quantum:
milestone: none → grizzly-1
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2012-11-22
Changed in cinder:
milestone: none → grizzly-1
status: Fix Committed → Fix Released
no longer affects: cinder (Ubuntu Precise)
no longer affects: quantum (Ubuntu Precise)
Changed in quantum (Ubuntu Quantal):
status: New → Confirmed
James Page (james-page) on 2012-12-06
Changed in quantum (Ubuntu):
importance: Undecided → Medium

Hello Thierry, or anyone else affected,

Accepted cinder into quantal-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/cinder/2012.2.1-0ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in cinder (Ubuntu Quantal):
status: New → Fix Committed
tags: added: verification-needed
Changed in nova (Ubuntu):
status: New → Fix Committed
Changed in cinder (Ubuntu):
status: New → Fix Committed
Changed in nova (Ubuntu Quantal):
status: New → Fix Committed
Changed in quantum (Ubuntu Quantal):
status: Confirmed → Fix Committed
Clint Byrum (clint-fewbar) wrote :

Hello Thierry, or anyone else affected,

Accepted quantum into quantal-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/quantum/2012.2.1-0ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Clint Byrum (clint-fewbar) wrote :

Hello Thierry, or anyone else affected,

Accepted nova into quantal-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/nova/2012.2.1+stable-20121212-a99a802e-0ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Mark McLoughlin (markmc) on 2013-01-22
tags: removed: in-stable-folsom
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cinder - 2012.2.1-0ubuntu1

---------------
cinder (2012.2.1-0ubuntu1) quantal-proposed; urgency=low

  * Ubuntu updates:
    - Cinder should suggest ceph-common, not python-ceph (LP: #1065901):
      - debian/control: cinder-volume Suggests: python-ceph -> ceph-common
  * Resynchronize with stable/folsom (87d839a5) (LP: #1085255):
    - [f990ff0] Remove unused python-daemon dependency
    - [940f363] Detached and deleted RBD volumes remain associated with insance
      (LP: #1083818)
    - [7f34ba3] After folsom upgrade, instances can no longer access existing
      volumes. (LP: #1065702)
    - [1c99b24] Jenkins jobs fail because of incompatibility between sqlalchemy-
      migrate and the newest sqlalchemy-0.8.0b1 (LP: #1073569)
    - [d12d4b6] Add SIGPIPE handler to subprocess execution in rootwrap and
      utils.execute (LP: #1053364)
    - [ce5e002] Set defaultbranch in .gitreview to stable/folsom
 -- Adam Gandelman <email address hidden> Tue, 04 Dec 2012 09:19:29 -0800

Changed in cinder (Ubuntu Quantal):
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :
Download full text (8.3 KiB)

This bug was fixed in the package nova - 2012.2.1+stable-20121212-a99a802e-0ubuntu1

---------------
nova (2012.2.1+stable-20121212-a99a802e-0ubuntu1) quantal-proposed; urgency=low

  * Ubuntu updates:
    - debian/control: Ensure novaclient is upgraded with nova,
      require python-keystoneclient >= 1:2.9.0. (LP: #1073289)
    - d/p/avoid_setuptools_git_dependency.patch: Refresh.
  * Dropped patches, applied upstream:
    - debian/patches/CVE-2012-5625.patch: [a99a802]
  * Resynchronize with stable/folsom (b55014ca) (LP: #1085255):
    - [a99a802] create_lvm_image allocates dirty blocks (LP: #1070539)
    - [670b388] RPC exchange name defaults to 'openstack' (LP: #1083944)
    - [3ede373] disassociate_floating_ip with multi_host=True fails
      (LP: #1074437)
    - [22d7c3b] libvirt imagecache should handle shared image storage
      (LP: #1075018)
    - [e787786] Detached and deleted RBD volumes remain associated with insance
      (LP: #1083818)
    - [9265eb0] live_migration missing migrate_data parameter in Hyper-V driver
      (LP: #1066513)
    - [3d99848] use_single_default_gateway does not function correctly
      (LP: #1075859)
    - [65a2d0a] resize does not migrate DHCP host information (LP: #1065440)
    - [102c76b] Nova backup image fails (LP: #1065053)
    - [48a3521] Fix config-file overrides for nova-dhcpbridge
    - [69663ee] Cloudpipe in Folsom: no such option: cnt_vpn_clients
      (LP: #1069573)
    - [6e47cc8] DisassociateAddress can cause Internal Server Error
      (LP: #1080406)
    - [22c3d7b] API calls to dis-associate an auto-assigned floating IP should
      return proper warning (LP: #1061499)
    - [bd11d15] libvirt: if exception raised during volume_detach, volume state
      is inconsistent (LP: #1057756)
    - [dcb59c3] admin can't describe all images in ec2 api (LP: #1070138)
    - [78de622] Incorrect Exception raised during Create server when metadata
      over 255 characters (LP: #1004007)
    - [c313de4] Fixed IP isn't released before updating DHCP host file
      (LP: #1078718)
    - [f4ab42d] Enabling Return Reservation ID with XML create server request
      returns no body (LP: #1061124)
    - [3db2a38] 'BackupCreate' should accept rotation parameter greater than or
      equal to zero (LP: #1071168)
    - [f7e5dde] libvirt reboot sometimes fails to reattach volumes
      (LP: #1073720)
    - [ff776d4] libvirt: detaching volume may fail while terminating other
      instances on the same host concurrently (LP: #1060836)
    - [85a8bc2] Used instance uuid rather than id in remove-fixed-ip
    - [42a85c0] Fix error on invalid delete_on_termination value
    - [6a17579] xenapi migrations fail w/ swap (LP: #1064083)
    - [97649b8] attach-time field for volumes is not updated for detach volume
      (LP: #1056122)
    - [8f6a718] libvirt: rebuild is not using kernel and ramdisk associated with
      the new image (LP: #1060925)
    - [fbe835f] live-migration and volume host assignement (LP: #1066887)
    - [c2a9150] typo prevents volume_tmp_dir flag from working (LP: #1071536)
    - [93efa21] Instances deleted during spawn leak network allocations
      (LP: #1068716)
    - [ebabd02] After restartin...

Read more...

Changed in nova (Ubuntu Quantal):
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :
Download full text (3.8 KiB)

This bug was fixed in the package quantum - 2012.2.1-0ubuntu1

---------------
quantum (2012.2.1-0ubuntu1) quantal-proposed; urgency=low

  * Resynchronize with stable/folsom (1e774867) (LP: #1085255):
    - [aeabb42] There are routing problems when the dnsmasq port does not come
      first in the routing table (LP: #1083238)
    - [04aab72] Quantum linux bridge not optimized with libvirt (LP: #1078210)
    - [ca7fc10] getting quotas from database has severe performance implications
      (LP: #1075369)
    - [66605e8] failed to update an external network into non external network
      (LP: #1083387)
    - [c60051a] Quantum test suite leaks memory like a sieve (LP: #1065276)
    - [3179dfc] clear_db() does incomplete db teardown (LP: #1080988)
    - [c1e19d7] Unauthorized command: cat /proc/None/cmdline (LP: #1077651)
    - [af9e076] At times a instance will not receive an IP address from the DHCP
      agent (LP: #1081664)
    - [e0d1a7d] allow multiple floating-ip on single port if they use different
      fixed ips and/or external nets (LP: #1057844)
    - [8471d79] Delete port fails to gateway ip (LP: #1079980)
    - [aca8b4a] fixed_ip allocation which is not included within
      allocation_pools makes error when delete port or re-create port
      (LP: #1077292)
    - [eacc9d3] Mapping same bridge to different phyiscal networks succeed
      (LP: #1067669)
    - [51b4c82] python-quantum: not region aware (LP: #1080793)
    - [6f0a486] delete floatingip should be in one transaction to delete port
      (LP: #1080516)
    - [db6cda7] Remove qpid configuration variables no longer supported
    - [a112840] Allow NVP plugin to use per-tenant quota extension
    - [82b1a55] Quantum service does not restart after reboot (LP: #1073999)
    - [c01a839] There are some cases that L3 API with an invalid parameter
      returns 500. (LP: #1064765)
    - [26b383f] external network can be plugged also as internal network for one
      router (LP: #1053633)
    - [49f649c] There is a lot of cases that API with an invalid parameter
      returns 500. (LP: #1062046)
    - [4546a18] When create subnet, you con set up the value as cidr (the value
      isn't cidr form). (LP: #1067959)
    - [9ba453a] killfilter should handle updated/deleted executables
      (LP: #1073768)
    - [7c8a55c] a port which is not able to delete is made when floatingip
      create fails. (LP: #1064748)
    - [c9b84cf] Linux bridge port update causes exception (LP: #1072713)
    - [cb57932] I can't add interface to router, if there is another port in
      non-shared network of other tenant (LP: #1057558)
    - [574e278] Ryu plugin does not support Security Groups (LP: #1059393)
    - [607f486] tap device added to integration bridge without tag
      (LP: #1064070)
    - [21a0fdf] L3 agent external network flag (LP: #1056720)
    - [5cbaff4] router create with external_gateway_info fails with 500 always.
      (LP: #1064235)
    - [63b81f6] l3 db operations failed in multiple transactions (LP: #1070335)
    - [bff17fb] Ensure that the SqlSoup import is still supported.
    - [e091a29] l3_nat_agent was renamed to l3_agent
    - [9030969] remove default value of 'local_ip' of 10...

Read more...

Changed in quantum (Ubuntu Quantal):
status: Fix Committed → Fix Released
tags: removed: verification-needed
Thierry Carrez (ttx) on 2013-04-04
Changed in quantum:
milestone: grizzly-1 → 2013.1
Thierry Carrez (ttx) on 2013-04-04
Changed in nova:
milestone: grizzly-1 → 2013.1
Thierry Carrez (ttx) on 2013-04-04
Changed in cinder:
milestone: grizzly-1 → 2013.1
James Page (james-page) on 2014-03-14
no longer affects: cinder (Ubuntu)
James Page (james-page) on 2014-08-01
no longer affects: nova (Ubuntu)
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers