swarm cluster failed to report condition signal

Bug #1504059 reported by Eli Qiao
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Magnum
Fix Released
Undecided
Mark Goddard

Bug Description

bug https://bugs.launchpad.net/magnum/+bug/1502329
proposed a patch https://review.openstack.org/230639
to add additional signal trigger in an on-failure notice
service, but the wait_condition in swarm.yaml/swarmnode.yaml
will ignore the second signal, so that patch is useless.

Eli Qiao (taget-9)
Changed in magnum:
status: New → Confirmed
assignee: nobody → Eli Qiao (taget-9)
Adrian Otto (aotto)
Changed in magnum:
milestone: none → mitaka-1
Revision history for this message
Murali Allada (murali-allada) wrote :

Please update the bug. Close it if it's invalid.

Revision history for this message
Mark Goddard (mgoddard) wrote :

I'm currently seeing swarm clusters fail to correctly report their completion status to heat. I traced this through to the swarm heat templates in magnum using incorrect names for the waitcondition signals.

For example, see https://github.com/openstack/magnum/blob/4ad01cb867734e1cfbd2d681f8aa3c9d3ead0b2a/magnum/drivers/common/templates/swarm/fragments/write-swarm-master-service.sh#L54. The status, reason, data and id fields are capitalised to Status, Reason, Data and Id. Compare this with the k8s wait condition notifier: https://github.com/openstack/magnum/blob/4ad01cb867734e1cfbd2d681f8aa3c9d3ead0b2a/magnum/drivers/common/templates/kubernetes/fragments/wc-notify-master.sh#L14 and various examples of using wait conditions which can be found.

This explains why swarm clusters do not correctly report failure.

Revision history for this message
Spyros Trigazis (strigazi) wrote :

Can you post here sample failure and success outputs?

Changed in magnum:
assignee: Eli Qiao (taget-9) → Mark Goddard (mgoddard)
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to magnum (master)

Reviewed: https://review.openstack.org/467623
Committed: https://git.openstack.org/cgit/openstack/magnum/commit/?id=0dc463e391796dfe4399d848625259c45d93720e
Submitter: Jenkins
Branch: master

commit 0dc463e391796dfe4399d848625259c45d93720e
Author: Mark Goddard <email address hidden>
Date: Wed May 24 14:23:32 2017 +0100

    Use lowercase keys for swarm waitcondition signal

    The heat waitcondition signal API accepts status, reason, data and id
    fields in a JSON object supplied as POST data. Missing fields will be
    filled with defaults. Previously, the swarm script fragments used a
    capitalised form of these keys (Status, Reason, Data, Id) which was
    not being recognised by heat. This caused failures to not be reported.

    This change uses the correct lowercase names for these fields and also
    fixes some quoting and incorrect use of UUIDs provided as the id field.

    Change-Id: I9bfe36e5dd956280eaa42d1c3f1620c4ec27bc0c
    Closes-Bug: #1504059

Changed in magnum:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to magnum (stable/ocata)

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/474972

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to magnum (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/474983

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to magnum (stable/newton)

Reviewed: https://review.openstack.org/474983
Committed: https://git.openstack.org/cgit/openstack/magnum/commit/?id=dad5b6340a08705c6fdc4c8116393b0c0bb4b56d
Submitter: Jenkins
Branch: stable/newton

commit dad5b6340a08705c6fdc4c8116393b0c0bb4b56d
Author: Mark Goddard <email address hidden>
Date: Wed May 24 14:23:32 2017 +0100

    Use lowercase keys for swarm waitcondition signal

    The heat waitcondition signal API accepts status, reason, data and id
    fields in a JSON object supplied as POST data. Missing fields will be
    filled with defaults. Previously, the swarm script fragments used a
    capitalised form of these keys (Status, Reason, Data, Id) which was
    not being recognised by heat. This caused failures to not be reported.

    This change uses the correct lowercase names for these fields and also
    fixes some quoting and incorrect use of UUIDs provided as the id field.

    Change-Id: I9bfe36e5dd956280eaa42d1c3f1620c4ec27bc0c
    Closes-Bug: #1504059

tags: added: in-stable-newton
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to magnum (stable/ocata)

Reviewed: https://review.openstack.org/474972
Committed: https://git.openstack.org/cgit/openstack/magnum/commit/?id=8d0f25f130ccb645c51ee080c49971b7816b8e80
Submitter: Jenkins
Branch: stable/ocata

commit 8d0f25f130ccb645c51ee080c49971b7816b8e80
Author: Mark Goddard <email address hidden>
Date: Wed May 24 14:23:32 2017 +0100

    Use lowercase keys for swarm waitcondition signal

    The heat waitcondition signal API accepts status, reason, data and id
    fields in a JSON object supplied as POST data. Missing fields will be
    filled with defaults. Previously, the swarm script fragments used a
    capitalised form of these keys (Status, Reason, Data, Id) which was
    not being recognised by heat. This caused failures to not be reported.

    This change uses the correct lowercase names for these fields and also
    fixes some quoting and incorrect use of UUIDs provided as the id field.

    Change-Id: I9bfe36e5dd956280eaa42d1c3f1620c4ec27bc0c
    Closes-Bug: #1504059

tags: added: in-stable-ocata
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/magnum 3.3.0

This issue was fixed in the openstack/magnum 3.3.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/magnum 4.1.3

This issue was fixed in the openstack/magnum 4.1.3 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/magnum 5.0.0

This issue was fixed in the openstack/magnum 5.0.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.