Openstack services should support SIGHUP signal

Bug #1276694 reported by Bogdan Dobrelya
56
This bug affects 10 people
Affects Status Importance Assigned to Milestone
Cinder
Invalid
Undecided
Unassigned
Glance
Fix Released
Medium
Abhishek Kekane
Murano
Confirmed
Medium
Alexander Tivelkov
OpenStack Compute (nova)
Fix Released
Wishlist
Davanum Srinivas (DIMS)
OpenStack Heat
Fix Released
Medium
Unassigned
OpenStack Identity (keystone)
Invalid
Wishlist
Unassigned
Sahara
Fix Released
Medium
Sergey Lukjanov
neutron
Fix Released
Wishlist
Unassigned
oslo-incubator
Invalid
Undecided
Davanum Srinivas (DIMS)
oslo.config
Fix Released
Undecided
Unassigned
oslo.log
Fix Released
Undecided
Unassigned
oslo.service
Fix Released
Undecided
Unassigned

Bug Description

1)In order to more effectively manage the unlinked and open (lsof +L1) log files descriptors w/o restarting the services, SIGHUP signal should be accepted by every Openstack service.

That would allow, e.g. logrotate jobs to gracefully HUP services after their log files were rotated. The only option we have for now is to force the services restart, quite a poor option from the services continuous accessibility PoV.

Note: according to http://en.wikipedia.org/wiki/Unix_signal
SIGHUP
   ... Many daemons will reload their configuration files and reopen their logfiles instead of exiting when receiving this signal.

Currently Murano and Glance are out of sync with Oslo SIGHUP support.

There is also the following issue exists for some of the services of OS projects with synced SIGHUP support:
2)
heat-api-cfn, heat-api, heat-api-cloudwatch, keystone: looks like the synced code is never being executed, thus SIGHUP is not supported for them. Here is a simple test scenario:
2.1) modify <python-path>/site-packages/<foo-service-name>/openstack/common/service.py
def _sighup_supported():
+ LOG.warning("SIGHUP is supported: {0}".format(hasattr(signal, 'SIGHUP')))
    return hasattr(signal, 'SIGHUP')
2.2) restart service foo-service-name and check logs for "SIGHUP is supported", if service really supports it, the appropriate messages would be present in the logs.
2.3) issue kill -HUP <foo-service-pid> and check logs for "SIGHUP is supported" and "Caught SIGHUP", if service really supports it, the appropriate messages would be present in the logs. Besides that, the service should remain started and its main thread PID should not be changed.

e.g.
2.a) heat-engine supports HUPing:
#service openstack-heat-engine restart
<132>Apr 11 14:03:48 node-3 heat-heat.openstack.common.service WARNING: SIGHUP is supported: True

2.b)But heat-api don't know how to HUP:
#service openstack-heat-api restart
<134>Apr 11 14:06:22 node-3 heat-heat.api INFO: Starting Heat ReST API on 0.0.0.0:8004
<134>Apr 11 14:06:22 node-3 heat-eventlet.wsgi.server INFO: Starting single process server

2.c) HUPing heat-engine is OK
#pid=$(cat /var/run/heat/openstack-heat-engine.pid); kill -HUP $pid && echo $pid
16512
<134>Apr 11 14:12:15 node-3 heat-heat.openstack.common.service INFO: Caught SIGHUP, exiting
<132>Apr 11 14:12:15 node-3 heat-heat.openstack.common.service WARNING: SIGHUP is supported: True
<134>Apr 11 14:12:15 node-3 heat-heat.openstack.common.rpc.common INFO: Connected to AMQP server on ...
service openstack-heat-engine status
openstack-heat-engine (pid 16512) is running...

2.d) HUPed heat-api is dead now ;(
#kill -HUP $(cat /var/run/heat/openstack-heat-api.pid)
(no new logs)
# service openstack-heat-api status
openstack-heat-api dead but pid file exists

3)
nova-cert, nova-novncproxy, nova-objectstore, nova-consoleauth, nova-scheduler - unlike to case 2, after kill -HUP <foo-service-pid> command was issued, there would be a "Caught SIGHUP" message in the logs, BUT the associated service would have got dead anyway. Instead, the service should remain started and its main thread PID should not be changed (similar to the 2.c case).

So, looks like there are a lot of things still should be done to ensure POSIX standards abidance in Openstack :-)

Revision history for this message
Ben Nemec (bnemec) wrote :

This ability already exists in Oslo, but to my knowledge only Nova is currently using it. Other projects that should behave this way can be added as "Also affects" to track that they need to pick up the functionality.

Changed in oslo:
status: New → Invalid
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

Please make sure murano and glance could support SIGHUP as well (see the links above as a confirmation of the issue).

Changed in murano:
status: New → Confirmed
Changed in glance:
status: New → Confirmed
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

Ping pong update:
1) Glance and Murano projects are out of sync with Oslo SIGHUP support
2) For some projects synced from Oslo code for SIGHUP support are never being executed. See updated bug description.

description: updated
description: updated
Changed in heat:
status: New → Confirmed
Changed in nova:
status: New → Confirmed
Changed in keystone:
status: New → Confirmed
Changed in murano:
importance: Undecided → Medium
milestone: none → 0.6
ruhe (ruhe)
Changed in murano:
importance: Medium → Low
ruhe (ruhe)
Changed in murano:
milestone: 0.6 → juno-1
milestone: juno-1 → none
Changed in keystone:
importance: Undecided → Low
Changed in fuel:
assignee: nobody → MOS Oslo (mos-oslo)
milestone: none → 5.1
Dmitry Pyzhov (dpyzhov)
Changed in fuel:
importance: Undecided → Low
Changed in fuel:
status: New → Confirmed
Changed in murano:
assignee: nobody → Igor Yozhikov (iyozhikov)
milestone: none → juno-2
Dmitry Ilyin (idv1985)
summary: - Openstack services should support SIGHUP signal
+ [mos] Openstack services should support SIGHUP signal
ruhe (ruhe)
no longer affects: murano
summary: - [mos] Openstack services should support SIGHUP signal
+ Openstack services should support SIGHUP signal
no longer affects: fuel
Jeff Peeler (jpeeler-z)
Changed in heat:
assignee: nobody → Jeff Peeler (jpeeler-z)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to heat (master)

Fix proposed to branch: master
Review: https://review.openstack.org/111363

Changed in heat:
status: Confirmed → In Progress
Sean Dague (sdague)
Changed in nova:
importance: Undecided → Wishlist
Revision history for this message
Davanum Srinivas (DIMS) (dims-v) wrote :

Nova already supports SIGHUP from what i can tell.

Changed in nova:
status: Confirmed → Fix Committed
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

@Davanum, according to 3)
nova-cert, nova-novncproxy, nova-objectstore, nova-consoleauth, nova-scheduler - unlike to case 2, after kill -HUP <foo-service-pid> command was issued, there would be a "Caught SIGHUP" message in the logs, BUT the associated service would have got dead anyway. Instead, the service should remain started and its main thread PID should not be changed (similar to the 2.c case).

Not all Nova services handle SIGHUP as appropriate.

Changed in nova:
status: Fix Committed → Confirmed
Revision history for this message
Davanum Srinivas (DIMS) (dims-v) wrote :

Thanks for the information Bogdan!

Changed in glance:
importance: Undecided → Medium
milestone: none → kilo-1
assignee: nobody → Tushar Patil (tpatil)
Revision history for this message
Tushar Patil (tpatil) wrote :

Mark, Bodgan: In case of glance service, we had submitted patch [1] to allow SIGHUP signal to be handled for reloading of the configuration files. But it wasn't merged because it wasn't clear which parameters will be reloaded gracefully.

Thinking of fixing this bug from glance service Pov, Are you expecting to change log related parameters before sending SIGHUP signal?

[1] : https://review.openstack.org/#/c/117988/

Changed in glance:
assignee: Tushar Patil (tpatil) → Abhishek Kekane (abhishek-kekane)
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

@Tushar : I believe - I could be wrong, tho- it is expected by operators of any POSIX compatible operating system that SIGHUP should make the process to reread and re-apply any given configuration parameter of its config file as well as refresh its FDs...

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to heat (master)

Reviewed: https://review.openstack.org/111363
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=d795cfc5c89e9445148a48e9a0b9000485bc4ba3
Submitter: Jenkins
Branch: master

commit d795cfc5c89e9445148a48e9a0b9000485bc4ba3
Author: Jeff Peeler <email address hidden>
Date: Fri Aug 1 14:39:00 2014 -0400

    Fix signal handling for WSGI servers

    Previously, sending a SIGHUP to the parent process would close the
    shared socket used with all the WSGI servers (problematic when multiple
    workers are present). Also, exiting the wait loop in the parent would
    cause the children to be orphaned. Instead, make the parent process
    restart all the children servers upon receipt of SIGHUP. This behavior
    seems to make sense given that sending SIGHUP to a child restarts and
    creates a new server (due to the parent handling of the children).

    Another fix was to issue a SIGTERM upon Ctrl-C in order to not leave
    parentless children. SIGTERM was chosen instead of SIGINT simply to
    avoid worker tracebacks on the terminal.

    Minor unrelated change: the WSGI server in single worker mode has been
    changed to also respect the debug flag, exactly the same as with
    multiple workers.

    Note: the original bug report was examining lack of signal handling
    based on the Oslo service code. However, only the engine makes use of
    it.

    Change-Id: I8519cc6fedf72b38a8ce0762bb86cd70e58e0dc7
    Closes-bug: #1276694

Changed in heat:
status: In Progress → Fix Committed
Dolph Mathews (dolph)
Changed in keystone:
importance: Low → Wishlist
Thierry Carrez (ttx)
Changed in heat:
milestone: none → juno-rc1
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in heat:
milestone: juno-rc1 → 2014.2
Thierry Carrez (ttx)
Changed in glance:
milestone: kilo-1 → kilo-2
Changed in sahara:
status: New → Confirmed
assignee: nobody → Andrew Lazarev (alazarev)
Changed in sahara:
milestone: none → kilo-2
importance: Undecided → High
Changed in sahara:
status: Confirmed → In Progress
Changed in glance:
milestone: kilo-2 → kilo-3
Revision history for this message
Angus Salkeld (asalkeld) wrote :

Heat services just exit, they should reload config and reopen logfiles.

2015-02-03 10:38:06.889 INFO heat.openstack.common.service [-] Caught SIGHUP, exiting
2015-02-03 10:38:06.889 INFO heat.engine.service [-] Attempting to stop engine service...
2015-02-03 10:38:06.899 INFO heat.engine.service [req-5081e303-1f40-4614-9c2d-2ecce8733085 None None] Service 834c488a-d5c5-46c4-b01c-7b9c7fd83468 is deleted
2015-02-03 10:38:06.899 INFO heat.engine.service [req-5081e303-1f40-4614-9c2d-2ecce8733085 None None] All threads were gone, terminating engine

Changed in heat:
importance: Undecided → Medium
status: Fix Released → Confirmed
milestone: 2014.2 → kilo-3
assignee: Jeff Peeler (jpeeler-z) → nobody
Changed in sahara:
milestone: kilo-2 → kilo-3
Changed in neutron:
importance: Undecided → Wishlist
status: New → Confirmed
Changed in sahara:
importance: High → Medium
status: In Progress → Confirmed
Changed in nova:
assignee: nobody → Davanum Srinivas (DIMS) (dims-v)
Changed in oslo-incubator:
assignee: nobody → Davanum Srinivas (DIMS) (dims-v)
Changed in murano:
status: New → Confirmed
importance: Undecided → Medium
milestone: none → kilo-3
assignee: nobody → Andrew Pashkin (apashkin)
Elena Ezhova (eezhova)
Changed in neutron:
assignee: nobody → Elena Ezhova (eezhova)
Anant Patil (ananta)
Changed in heat:
assignee: nobody → Anant Patil (ananta)
Revision history for this message
Elena Ezhova (eezhova) wrote :

In case when heat-api is running in single process mode sending SIGHUP leads to its termination:

Hangup
h-api failed to start

Revision history for this message
Elena Ezhova (eezhova) wrote :

Additionally, sending SIGHUP doesn't lead to reloading of heat-api configuration

Elena Ezhova (eezhova)
Changed in neutron:
status: Confirmed → In Progress
Changed in sahara:
milestone: kilo-3 → liberty-1
Angus Salkeld (asalkeld)
Changed in heat:
milestone: kilo-3 → liberty-1
Changed in murano:
milestone: kilo-3 → none
Thierry Carrez (ttx)
Changed in glance:
milestone: kilo-3 → kilo-rc1
Revision history for this message
Davanum Srinivas (DIMS) (dims-v) wrote :

Tested nova-compute just now. kill -HUP does work and config files gets reloaded as well

Changed in nova:
status: Confirmed → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/167974

Anant Patil (ananta)
Changed in heat:
assignee: Anant Patil (ananta) → nobody
status: Confirmed → Fix Committed
Revision history for this message
Nikhil Komawar (nikhil-komawar) wrote :

To whomever concerned or the Assignee:

I'd like for us to mark this as "Fix Committed" in Glance for kilo-3. There are some related bugs that need review, we can link them here and keep working on them.

@ttx and the Assignee :
This does not seem like a blocker for RC1.

Revision history for this message
Nikhil Komawar (nikhil-komawar) wrote :

@ttx: This bug was split into multiple ones in Glance.

We are waiting on (review related to) https://bugs.launchpad.net/glance/+bug/1436275 to be merged so, this can be moved out of RC1 and I've marked the other one as RC1 blocker.

Changed in glance:
milestone: kilo-rc1 → none
status: Confirmed → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Related fix proposed to branch: master
Review: https://review.openstack.org/171178

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/171179

Kyle Mestery (mestery)
Changed in neutron:
milestone: none → liberty-1
Changed in murano:
milestone: none → kilo-rc1
assignee: Andrew Pashkin (apashkin) → nobody
milestone: kilo-rc1 → liberty-1
Thierry Carrez (ttx)
Changed in nova:
milestone: none → kilo-rc1
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in glance:
milestone: none → kilo-rc1
status: Fix Committed → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (master)

Reviewed: https://review.openstack.org/167974
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=f92d22d91b8a29a0088c69a1cf4940c822d38847
Submitter: Jenkins
Branch: master

commit f92d22d91b8a29a0088c69a1cf4940c822d38847
Author: Elena Ezhova <email address hidden>
Date: Thu Mar 26 15:33:36 2015 +0300

    Sync service from oslo-incubator

    This sync includes changes that are required to fix handling
    of SIGHUP in Neutron.

    The following changes and bugfixes are included:

    d24b658 Revert "Optimization of waiting subprocesses in ProcessLauncher"
    593005b ProcessLauncher: reload config file in parent process on SIGHUP
    f29e865 Store ProcessLauncher signal handlers on class level
    bf92010 Optimization of waiting subprocesses in ProcessLauncher

    Change-Id: If0aab4e8978422346f6ba4c9e6272cdaf39db6cb
    Closes-Bug: #1433142
    Related-Bug: #1276694

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Reviewed: https://review.openstack.org/171178
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=88510ef1b153fc548fc5bccc24e116a0882e66a8
Submitter: Jenkins
Branch: master

commit 88510ef1b153fc548fc5bccc24e116a0882e66a8
Author: Elena Ezhova <email address hidden>
Date: Tue Apr 7 14:54:45 2015 +0300

    Refactor socket ssl wrapping

    Move socket wrapping into a separate method in order to separate
    its logic from other action done in _get_socket. Now, ssl wrapping
    is applied to the socket returned by _get_socket method.
    Additionally checks for ssl config options are now performed during
    init and not each time wrap_socket is called.

    Added unit tests.

    Related-Bug: #1276694
    Change-Id: I706517ae351a7a681623ec91c9657a2f61cd2679

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/171179
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=d9251d6e35ac87b755b63aa58bb32da20496dfba
Submitter: Jenkins
Branch: master

commit d9251d6e35ac87b755b63aa58bb32da20496dfba
Author: Elena Ezhova <email address hidden>
Date: Tue Apr 7 14:55:50 2015 +0300

    Handle SIGHUP in dhcp and l3 agents

    All launchers implemented in common.service require each service to
    implement reset method because it is called in case a process
    receives a SIGHUP.

    This change adds the reset method to neutron.service.Service class
    which is used to start dhcp and l3 agents.

    Now dhcp and l3 agents don't die on receiving SIGHUP and support
    reloading policy_path and logging options in config.

    Partial-Bug: #1276694
    Change-Id: I96010e44928a665bea546865b2c81bde4ed0adf2

Thierry Carrez (ttx)
Changed in glance:
milestone: kilo-rc1 → 2015.1.0
Thierry Carrez (ttx)
Changed in nova:
milestone: kilo-rc1 → 2015.1.0
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (neutron-pecan)

Related fix proposed to branch: neutron-pecan
Review: https://review.openstack.org/185072

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/187174

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to heat (master)

Reviewed: https://review.openstack.org/185640
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=cd81f0d1c5e7c6d015ad0def2008d8ff35d6fa32
Submitter: Jenkins
Branch: master

commit cd81f0d1c5e7c6d015ad0def2008d8ff35d6fa32
Author: Tetiana Lashchova <email address hidden>
Date: Tue May 26 17:57:02 2015 +0300

    Reload logging options when receiving SIGHUP

    Related-Bug: #1276694
    Change-Id: I4c058654a094bb6ce3ca154d311f5d6aa31f6056

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/187174
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=ea5ad714cfcd336c0062ddeb13238c452117782a
Submitter: Jenkins
Branch: master

commit ea5ad714cfcd336c0062ddeb13238c452117782a
Author: Elena Ezhova <email address hidden>
Date: Mon Jun 1 14:55:13 2015 +0300

    Handle SIGHUP in ovs neutron agent

    This change allows ovs neutron agent not to die on receiving SIGHUP
    and reload its logging options.

    Note that this patch allows changing only logging options.
    All other config options are not handled explicitly and changing
    them using SIGHUP can lead to unpredictable circumstances.
    So, until changing other options is handled it is highly recommended
    to use SIGHUP for changing ONLY logging options.

    Change-Id: Ic0cf8a9ca7f3a16b556a6825e2979471ae136c33
    Partial-Bug: #1276694

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Reviewed: https://review.openstack.org/161732
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=6d0d72973152bb45587437c80d4ffe0fe7bba761
Submitter: Jenkins
Branch: master

commit 6d0d72973152bb45587437c80d4ffe0fe7bba761
Author: Elena Ezhova <email address hidden>
Date: Tue Apr 7 14:58:13 2015 +0300

    Handle SIGHUP: neutron-server (multiprocess) and metadata agent

    All launchers implemented in common.service require each service to
    implement reset method because it is called in case a process
    receives a SIGHUP.

    This change adds the reset method to neutron.service.RpcWorker and
    neutron.wsgi.WorkerService which are used to wrap rpc and api
    workers correspondingly.

    Now neutron-server running in multiprocess mode (api_workers > 0 and
    rpc_workers > 0) and metadata agent don't die on receiving SIGHUP and support
    reloading policy_path and logging options in config.

    Note that reset is called only in case a service is running in daemon mode.

    Other changes made in the scope of this patch that need to be mentioned:

    * Don't empty self._servers list in RpcWorker's stop method

      When a service is restarted all services are gracefully shutdowned,
      resetted and started again (see openstack.common.service code).
      As graceful shutdown implies calling service.stop() and then
      service.wait() we don't want to clean self._servers list because
      it would be impossible to wait for them to stop processing
      requests and cleaning up their resources.
      Otherwise, this would lead to problems with rpc after starting
      the rpc server again.

    * Create a duplicate socket each time WorkerService starts

      When api worker is stopped it kills the eventlet wsgi server
      which internally closes the wsgi server socket object. This server
      socket object becomes not usable which leads to "Bad file
      descriptor" errors on service restart.

    Added functional and unit tests.

    DocImpact
    Partial-Bug: #1276694
    Change-Id: I75b00946b7cae891c6eb192e853118e7d49e4a24

Changed in sahara:
milestone: liberty-1 → liberty-2
Thierry Carrez (ttx)
Changed in neutron:
milestone: liberty-1 → liberty-2
Thierry Carrez (ttx)
Changed in heat:
status: Fix Committed → Fix Released
Changed in sahara:
assignee: Andrew Lazarev (alazarev) → Sergey Lukjanov (slukjanov)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (feature/qos)

Fix proposed to branch: feature/qos
Review: https://review.openstack.org/196097

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (feature/qos)
Download full text (93.9 KiB)

Reviewed: https://review.openstack.org/196097
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=1cfed745d54a6ce9cb3dd4e6f454666d9e6676c2
Submitter: Jenkins
Branch: feature/qos

commit ba7d673d1ddd5bfa5aa1be5b26a59e9a8cd78a9f
Author: Kevin Benton <email address hidden>
Date: Thu Jun 25 18:31:38 2015 -0700

    Remove duplicated call to setup_coreplugin

    The test case for vlan_transparent was calling setup_coreplugin
    before calling the super setUp method which already calls
    setup_coreplugin. This was causing duplicate core plugin fixtures
    which resulted in patching the dhcp periodic check twice.

    Change-Id: Ide4efad42748e799d8e9c815480c8ffa94b27b38
    Partial-Bug: #1468998

commit e64062efa3b793f7c4ce4ab9e62918af4f1bfcc9
Author: Kevin Benton <email address hidden>
Date: Thu Jun 25 18:29:37 2015 -0700

    Remove double mock of dhcp agent periodic check

    The test case for the periodic check was patching a target
    that the core plugin fixture already patched out. This removes
    that and exposes the mock from the fixture so the test case
    can reference it.

    Change-Id: I3adee6a875c497e070db4198567b52aa16b81ce8
    Partial-Bug: #1468998

commit 25ae0429a713143d42f626dd59ed4514ba25820c
Author: Kevin Benton <email address hidden>
Date: Thu Jun 25 18:24:10 2015 -0700

    Remove double fanout mock

    The test_mech_driver was duplicating a fanout mock already setup
    in the setUp routine.

    Change-Id: I5b88dff13113d55c72241d3d5025791a76672ac2
    Partial-Bug: #1468998

commit 993771556332d9b6bbf7eb3f0300cf9d8a2cb464
Author: Kevin Benton <email address hidden>
Date: Thu Jun 25 17:55:16 2015 -0700

    Remove double callback manager mocks

    setup_test_registry_instance() in the base test case class gives
    each test its own registry by mocking out the get_callback_manager.
    The L3 agent test cases were duplicating this.

    Partial-Bug: #1468998
    Change-Id: I7356daa846524611e9f92365939e8ad15d1e1cd8

commit 0be1efad93734f11cd63fb3b7bd2983442ce1268
Author: Kevin Benton <email address hidden>
Date: Thu Jun 25 16:57:30 2015 -0700

    Remove ensure_dirs double-patch

    test_spawn_radvd called mock.patch on ensure_dirs after the
    setup method already patched it out. This causes issues when
    mock.patch.stopall() is called because the mocks are stored
    as a set and are unwound in a non-deterministic fashion.[1]
    So some of the time they will be undone correctly, but others
    will leave a monkey-patched in mock, causing the ensure_dir
    test to fail.

    1. http://bugs.python.org/issue21239

    Closes-Bug: #1467908
    Change-Id: I321b5fed71dc73bd19b5099311c6f43640726cd4

commit 0a2238e34e72c17ca8a75e36b1f56e41a3ece74e
Author: Sukhdev Kapur <email address hidden>
Date: Thu Jun 25 15:11:28 2015 -0700

    Fix tenant-id in Arista ML2 driver to support HA router

    When HA router is created, the framework creates a network and does
    not specify the tenant-id. This casuse Arista ML2 driver to fail.
    This patch sets the tenant-id when it is not passed explicitly by
    by the network_create() call from the HA r...

tags: added: in-feature-qos
Changed in murano:
milestone: liberty-1 → liberty-2
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (feature/pecan)

Fix proposed to branch: feature/pecan
Review: https://review.openstack.org/196701

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (feature/pecan)

Change abandoned by Kyle Mestery (<email address hidden>) on branch: feature/pecan
Review: https://review.openstack.org/196701
Reason: This is lacking the functional fix [1], so I'll propose a new merge commit which includes that one.

[1] https://review.openstack.org/#/c/196711/

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (feature/pecan)

Fix proposed to branch: feature/pecan
Review: https://review.openstack.org/196920

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (feature/pecan)
Download full text (171.5 KiB)

Reviewed: https://review.openstack.org/196920
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=7f759c077f8f860c13db92d2ea6b353ef6b70900
Submitter: Jenkins
Branch: feature/pecan

commit 8123144fadd7c5d5e6e56a76ea860512619a2cf6
Author: Moshe Levi <email address hidden>
Date: Sun Jun 28 14:37:14 2015 +0300

    Fix Consolidate sriov agent and driver code

    This patch add mising __init to mech_sriov/mech_driver/
    and update the setup.cfg to the new agent entrypoint

    Trivial Fix

    Change-Id: I53a527081feb78472f496675bbb3c5121d38a14a

commit 8942fccf02e6e179d47582fdb2792a1ca972da21
Author: Assaf Muller <email address hidden>
Date: Mon Jun 29 11:38:51 2015 -0400

    Remove failing SafeFixture tests

    The fixtures 1.3 release attempted to fix the fixtures resource
    leak issue, but failed to do so completely. Our own SafeFixture
    is still needed: The 1.3 release broke our SafeFixture tests,
    but not the usage of SafeFixture itself. This patch removes
    those failing tests for now to unbreak the gate. Jakub reported
    a bug on fixtures 1.3:
    https://bugs.launchpad.net/python-fixtures/+bug/1469759

    We will continue to use SafeFixture until that bug is fixed
    in fixtures, at which point we will be able to require
    fixtures > 1.3.

    Change-Id: I59457c3bb198ff86d5ad55a1e623d008f0034b8f
    Closes-Bug: #1469734

commit 71dffb0a2c1720cd8233a329d32958a0160dd6f5
Author: Kevin Benton <email address hidden>
Date: Mon Jun 29 08:27:41 2015 +0000

    Revert "Removed test_lib module"

    This reverts commit 9a6536de6e1a7fe9b2552adc142e254426b82b6f.

    We pulled all of the plugins out of the tree, many of which still inherit
    from neutron test classes. This change then stated that we no longer
    support testing other plugins. I think this is a bit premature and should
    have been discussed under the subject
    "Neutron plugins can't use neutron plugin unit tests" or something
    similar.

    Change-Id: I68318589f010b731574ea3bfa8df98492bab31fc

commit b20fd81dbd497e058384a0af065dd0f1fdc4c728
Author: Jakub Libosvar <email address hidden>
Date: Fri Jun 5 14:32:51 2015 +0000

    Refactor NetcatTester class

    Following capabilities were added:
       - used transport protocol is passed as a constant instead of bool
       - src port for testing was added
       - connection can be established explicitly
       - change constructor parameters of NetcatTester

    As a part of removing bool for protocol definition
    get_free_namespace_port() was also modified to match the behavior.

    Change-Id: Id2ec322e7f731c05a3754a65411c9a5d8b258126

commit 83e37980dcd0b2bad6d64dd2cb23bcd2891cafca
Author: jingliuqing <email address hidden>
Date: Sat Jun 27 13:41:54 2015 +0800

    Use REST rather than ReST

    Change-Id: I06c9deaab58c5ec13bfeec39fb8fd4b1fe21f42d

commit 1b60df85ba3ad442c2e4e7e52538e1b9a1bf9378
Author: Kevin Benton <email address hidden>
Date: Thu Jun 25 18:34:38 2015 -0700

    Add a double-mock guard to the base test case

    Use mock to patch mock with a check to prevent multiple active
    patches to the...

tags: added: in-feature-pecan
Changed in murano:
assignee: nobody → Ekaterina Chernova (efedorova)
importance: Medium → Wishlist
Changed in murano:
status: Confirmed → In Progress
Thierry Carrez (ttx)
Changed in sahara:
milestone: liberty-2 → liberty-3
Thierry Carrez (ttx)
Changed in neutron:
milestone: liberty-2 → liberty-3
Changed in sahara:
status: Confirmed → In Progress
Changed in murano:
milestone: liberty-2 → liberty-3
Changed in murano:
status: In Progress → Confirmed
assignee: Ekaterina Chernova (efedorova) → nobody
Changed in neutron:
status: In Progress → Fix Committed
status: Fix Committed → In Progress
Changed in sahara:
milestone: liberty-3 → liberty-rc1
Changed in neutron:
milestone: liberty-3 → liberty-rc1
Changed in murano:
milestone: liberty-3 → liberty-rc1
Changed in murano:
importance: Wishlist → Medium
Changed in murano:
assignee: nobody → Alexander Tivelkov (ativelkov)
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

Added the Oslo.config as there is at least one patch addressing this issue https://review.openstack.org/#/c/213062/

Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

Added Oslo.log because a service must reopen its log files upon receiving a SIGHUP. IIUC, Oslo.log should address this.

Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

Added Oslo.service according to this http://lists.openstack.org/pipermail/openstack-dev/2015-September/074745.html
"oslo.service is responsible for catching/handling signals"

Revision history for this message
Kyle Mestery (mestery) wrote :

Moving out of Liberty as we're in the RC.

Changed in neutron:
milestone: liberty-rc1 → mitaka-1
Changed in sahara:
milestone: liberty-rc1 → next
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (master)

Change abandoned by Kyle Mestery (<email address hidden>) on branch: master
Review: https://review.openstack.org/170905
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

Revision history for this message
Marian Horban (mhorban) wrote :

Clause 1) from Bug Description is not an issue any more because new feature that allow to recreate rotated log files for oslo.log was implemented

Revision history for this message
Marian Horban (mhorban) wrote :
Changed in murano:
milestone: liberty-rc1 → mitaka-1
Thierry Carrez (ttx)
Changed in heat:
milestone: liberty-1 → 5.0.0
Changed in neutron:
milestone: mitaka-1 → mitaka-2
Changed in murano:
milestone: mitaka-1 → mitaka-2
Revision history for this message
Elena Ezhova (eezhova) wrote :

oslo.service currently supports handling of SIGHUP

Changed in oslo.service:
status: New → Fix Released
Changed in keystone:
milestone: none → mitaka-2
Revision history for this message
Steve Martinelli (stevemar) wrote :

I think keystone gets this for free since we're using oslo.service (https://github.com/openstack/keystone/blob/master/requirements.txt#L32) for eventlet.

for the case of running keystone under httpd, i think most support SIGHUP: https://httpd.apache.org/docs/2.2/en/stopping.html

Changed in keystone:
status: Confirmed → Invalid
Changed in keystone:
milestone: mitaka-2 → none
Changed in neutron:
milestone: mitaka-2 → mitaka-3
Changed in murano:
milestone: mitaka-2 → mitaka-3
Changed in oslo.config:
status: New → Fix Released
Changed in oslo.log:
status: New → Fix Released
Revision history for this message
Michal Dulko (michal-dulko-f) wrote :

I'm not 100% sure that this is the intent of this bug, but I do believe that Cinder isn't dying when receiving SIGHUP signal [1], so I'm marking it as invalid.

[1] https://review.openstack.org/#/c/279039/

Changed in cinder:
status: New → Invalid
Revision history for this message
Vitalii Gridnev (vgridnev) wrote :

Sahara uses oslo.service for launching sahara. Moving to Fix Released because of that

Changed in sahara:
milestone: next → mitaka-3
status: In Progress → Fix Released
Changed in neutron:
milestone: mitaka-3 → mitaka-rc1
Changed in murano:
milestone: mitaka-3 → mitaka-rc1
Changed in neutron:
milestone: mitaka-rc1 → newton-1
Changed in murano:
milestone: mitaka-rc1 → newton-1
Changed in neutron:
milestone: newton-1 → newton-2
Changed in murano:
milestone: newton-1 → newton-2
Changed in neutron:
milestone: newton-2 → newton-3
Changed in murano:
milestone: newton-2 → newton-3
Changed in neutron:
milestone: newton-3 → newton-rc1
Changed in murano:
milestone: newton-3 → newton-rc1
Changed in neutron:
milestone: newton-rc1 → none
assignee: Elena Ezhova (eezhova) → nobody
status: In Progress → Incomplete
Changed in murano:
milestone: newton-rc1 → newton-rc2
Changed in murano:
milestone: newton-rc2 → ocata-1
Changed in murano:
milestone: 3.1.0 → pike-2
Changed in murano:
milestone: pike-2 → pike-rc1
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

Given that we have triaged https://bugs.launchpad.net/tripleo/+bug/1780139 we prolly can confirm this bug for Neutron now

Changed in neutron:
status: Incomplete → Confirmed
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :
Changed in neutron:
status: Confirmed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.