Service engine is launched with error due to updating oslo.messaging to 3.0.0

Bug #1521087 reported by Dmytro Dovbii
22
This bug affects 5 people
Affects Status Importance Assigned to Milestone
Murano
Fix Released
High
Kirill Zaitsev

Bug Description

1. Download murano from master
2. Install requirments
3. Confugure murano
4. Run murano-engine

You will see error in logs:

2015-11-30 10:09:23.688 19894 WARNING oslo_messaging.server [-] Possible hang: wait is waiting for stop to complete
2015-11-30 10:09:23.689 19894 DEBUG oslo_messaging.server [-] File "/home/ddovbii/work/murano/murano/cmd/engine.py", line 60, in <module>
    main()
  File "/home/ddovbii/work/murano/murano/cmd/engine.py", line 53, in main
    launcher.wait()
  File "/home/ddovbii/work/murano/.tox/venv/local/lib/python2.7/site-packages/oslo_service/service.py", line 284, in wait
    status, signo = self._wait_for_exit_or_signal(ready_callback)
  File "/home/ddovbii/work/murano/.tox/venv/local/lib/python2.7/site-packages/oslo_service/service.py", line 262, in _wait_for_exit_or_signal
    super(ServiceLauncher, self).wait()
  File "/home/ddovbii/work/murano/.tox/venv/local/lib/python2.7/site-packages/oslo_service/service.py", line 207, in wait
    self.services.wait()
  File "/home/ddovbii/work/murano/.tox/venv/local/lib/python2.7/site-packages/oslo_service/service.py", line 640, in wait
    service.wait()
  File "/home/ddovbii/work/murano/.tox/venv/local/lib/python2.7/site-packages/oslo_messaging/server.py", line 251, in wrapper
    log_after, timeout_timer)
  File "/home/ddovbii/work/murano/.tox/venv/local/lib/python2.7/site-packages/oslo_messaging/server.py", line 150, in wait_for_completion
    msg, log_after, timeout_timer)
  File "/home/ddovbii/work/murano/.tox/venv/local/lib/python2.7/site-packages/oslo_messaging/server.py", line 115, in _wait
    LOG.debug(''.join(traceback.format_stack()))
 _wait /home/ddovbii/work/murano/.tox/venv/local/lib/python2.7/site-packages/oslo_messaging/server.py:115

Changed in murano:
assignee: nobody → Nikolay Starodubtsev (starodubcevna)
importance: Undecided → High
Stan Lagun (slagun)
Changed in murano:
milestone: none → mitaka-1
status: New → Triaged
Changed in murano:
milestone: mitaka-1 → mitaka-2
Stan Lagun (slagun)
Changed in murano:
status: Triaged → Confirmed
assignee: Nikolay Starodubtsev (starodubcevna) → Stan Lagun (slagun)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to murano (master)

Fix proposed to branch: master
Review: https://review.openstack.org/255154

Changed in murano:
status: Confirmed → In Progress
Changed in sahara:
importance: Undecided → High
milestone: none → mitaka-2
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to murano (master)

Reviewed: https://review.openstack.org/255154
Committed: https://git.openstack.org/cgit/openstack/murano/commit/?id=05edab366164ebd4ac0743252776a9e00579650c
Submitter: Jenkins
Branch: master

commit 05edab366164ebd4ac0743252776a9e00579650c
Author: Stan Lagun <email address hidden>
Date: Wed Dec 9 12:30:49 2015 +0300

    Do not wait for MessageHandlingServer

    Previously oslo.messaging's MessageHandlingServer
    was used as as a murano-engine service in service launcher.
    As a result it's wait() method was called. But after recent
    changes it's wait method is supposed to be call only
    after stop() to wait for graceful message processing to end.
    As a result a warning was printed that stop() didn't finish
    after 30 sec from wait() invocation (because it was never called).

    This commit adds EngineService service implementation
    that has its own wait() and encapsulates MessageHandlingServer
    instance.

    Change-Id: Ie553e0b27cc1c261b963907b4f12f89795b99a12
    Closes-Bug: #1521087

Changed in murano:
status: In Progress → Fix Released
summary: - murano is launched with error due to updating oslo.messaging to 3.0.0
+ Service engine is launched with error due to updating oslo.messaging to
+ 3.0.0
Changed in sahara:
status: New → Confirmed
Revision history for this message
Sergey Reshetnyak (sreshetniak) wrote :

Not reproduced

Changed in sahara:
status: Confirmed → Incomplete
milestone: mitaka-2 → mitaka-3
Revision history for this message
Vitalii Gridnev (vgridnev) wrote :
Changed in sahara:
importance: High → Medium
status: Incomplete → Confirmed
Changed in sahara:
milestone: mitaka-3 → next
Stan Lagun (slagun)
tags: added: liberty-backport-potential
Revision history for this message
Kirill Zaitsev (kzaitsev) wrote :

Reopening the bug for murano. Still experiencing this on current master with all the latest requirements installed

2016-02-29 13:12:51.156 16663 DEBUG oslo_messaging.server [-] File ".tox/venv/bin/murano-api", line 10, in <module>
    sys.exit(main())
  File "/Users/teferi/murano/murano/murano/cmd/api.py", line 72, in main
    launcher.wait()
  File "/Users/teferi/murano/murano/.tox/venv/lib/python2.7/site-packages/oslo_service/service.py", line 309, in wait
    status, signo = self._wait_for_exit_or_signal()
  File "/Users/teferi/murano/murano/.tox/venv/lib/python2.7/site-packages/oslo_service/service.py", line 287, in _wait_for_exit_or_signal
    super(ServiceLauncher, self).wait()
  File "/Users/teferi/murano/murano/.tox/venv/lib/python2.7/site-packages/oslo_service/service.py", line 223, in wait
    self.services.wait()
  File "/Users/teferi/murano/murano/.tox/venv/lib/python2.7/site-packages/oslo_service/service.py", line 659, in wait
    service.wait()
  File "/Users/teferi/murano/murano/.tox/venv/lib/python2.7/site-packages/oslo_messaging/server.py", line 263, in wrapper
    log_after, timeout_timer)
  File "/Users/teferi/murano/murano/.tox/venv/lib/python2.7/site-packages/oslo_messaging/server.py", line 162, in wait_for_completion
    msg, log_after, timeout_timer)
  File "/Users/teferi/murano/murano/.tox/venv/lib/python2.7/site-packages/oslo_messaging/server.py", line 127, in _wait
    LOG.debug(''.join(traceback.format_stack()))
 _wait /Users/teferi/murano/murano/.tox/venv/lib/python2.7/site-packages/oslo_messaging/server.py:127

Changed in murano:
status: Fix Released → Confirmed
Revision history for this message
Kirill Zaitsev (kzaitsev) wrote :

Also might be important, but this happens to me, when I launch murano-api service

Changed in murano:
assignee: Stan Lagun (slagun) → nobody
tags: added: api engine
Changed in murano:
assignee: nobody → Varsha Kumari (varshaknit)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to murano (master)

Fix proposed to branch: master
Review: https://review.openstack.org/290628

Changed in murano:
status: Confirmed → In Progress
Changed in murano:
milestone: mitaka-2 → mitaka-rc1
Changed in murano:
assignee: Varsha Kumari (varshaknit) → Kirill Zaitsev (kzaitsev)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to murano (master)

Reviewed: https://review.openstack.org/290628
Committed: https://git.openstack.org/cgit/openstack/murano/commit/?id=940d69394c10c4cd21b8a6febc48d1394b68a423
Submitter: Jenkins
Branch: master

commit 940d69394c10c4cd21b8a6febc48d1394b68a423
Author: varshak05 <email address hidden>
Date: Wed Mar 9 21:28:48 2016 +0530

    Do not wait for MessageHandlingServer

    This commit adds ApiService/NotificationService service
    implementation that has its own wait() and encapsulates
    MessageHandlingServer instance.

    Similar issue was fixed for murano-engine, this commit fixes the same
    for murano-api service.

    Change-Id: Ia1566dc79aa5a05a00851fcf48f3e7318ac85d54
    Closes-Bug: #1521087

Changed in murano:
status: In Progress → Fix Released
tags: removed: liberty-backport-potential
Changed in sahara:
milestone: next → none
Revision history for this message
Kirill Zaitsev (kzaitsev) wrote :

I've noticed you removed liberty-backport-potential. I believe this bug still has that potential for murano. Please file a separate bug for sahara if you need one

tags: added: liberty-backport-potential
no longer affects: sahara
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.