[queens promotion] - fs001 fails overcloud deployment with 'MistralOvercloud configuration failed'

Bug #1750270 reported by Ronelle Landy
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Heat
Fix Released
Critical
Thomas Herve
tripleo
Invalid
Critical
Thomas Herve

Bug Description

Featureset001 is failing in the Queens promotion run with:

2018-02-18 12:00:35 | Started MistralOvercloud configuration failed.
2018-02-18 12:00:35 | END return value: 1
2018-02-18 12:00:35 | Workflow tripleo.deployment.v1.config_download_deploy. Execution ID: 9a05eb54-f6bb-4928-a991-37139715f9f1
2018-02-18 12:00:35 | Failed to run action [action_ex_id=cff8347d-0165-4f60-9c28-1636bbb42102, action_cls='<class 'mistral.actions.action_factory.GetOvercloudConfig'>', attributes='{}', params='{}']
2018-02-18 12:00:35 | argument of type 'NoneType' is not iterable

Full log is linked below:
https://logs.rdoproject.org/openstack-periodic/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens/f8fd98f/undercloud/home/jenkins/overcloud_deploy.log.txt.gz

This may be a duplicate of : https://bugs.launchpad.net/tripleo/+bug/1749645?

Logging for completeness.

Ronelle Landy (rlandy)
summary: - [queens promotion] - fe001 fails overcloud deplpoyment with 'NoneType'
- error on mistral.actions.action_factory.GetOvercloudConfig
+ [queens promotion] - fs001 fails overcloud deployment with
+ 'MistralOvercloud configuration failed'
Ronelle Landy (rlandy)
tags: added: ci
Revision history for this message
Ronelle Landy (rlandy) wrote :

Notes:
<mwhahaha> rlandy: i'm not sure that's a dupe
<mwhahaha> That's failing on the config-download buts where as the latter was failing on cep deploy
<mwhahaha> so there's probably a problem in mistal/config-download for 1750270

tags: added: promotion-blocker
Changed in tripleo:
milestone: none → queens-rc1
importance: Undecided → Critical
status: New → Triaged
Revision history for this message
Thomas Herve (therve) wrote :

It's failing on ServerIdData which looks like this:

{"server_ids": {"ObjectStorage": null, "Controller": ["16ed25eb-ecc8-4569-ba2a-76cd202e5248", "6ccbf5b2-ea19-4ecc-a65e-3c2a88427d2b", "5afeb4a5-5d48-4387-8a23-d82a05ba6ca0"], "Compute": ["b8076035-5de1-4754-a080-d9b1550e21df"], "CephStorage": null, "BlockStorage": null}, "bootstrap_server_id": "16ed25eb-ecc8-4569-ba2a-76cd202e5248"}

Presumably some roles don't have servers assigned to it? We should skip the server check.

Revision history for this message
Thomas Herve (therve) wrote :

I wonder if it's not a Heat behavior change, looking at it.

Arx Cruz (arxcruz)
Changed in tripleo:
assignee: nobody → Thomas Herve (therve)
Zane Bitter (zaneb)
Changed in heat:
milestone: none → queens-rc2
assignee: nobody → Thomas Herve (therve)
importance: Undecided → Critical
status: New → In Progress
Revision history for this message
Thomas Herve (therve) wrote :

https://review.openstack.org/#/c/546179/ with the potential fix. Will need to backport.

Changed in heat:
assignee: Thomas Herve (therve) → Zane Bitter (zaneb)
Zane Bitter (zaneb)
Changed in heat:
assignee: Zane Bitter (zaneb) → Thomas Herve (therve)
Revision history for this message
Alan Pevec (apevec) wrote :

Fix will be in Heat only, closing in tripleo.

Changed in tripleo:
status: Triaged → Invalid
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to heat (master)

Reviewed: https://review.openstack.org/546179
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=731502e2bfa29fa6a88b5b7a4ccec9f74eac3606
Submitter: Zuul
Branch: master

commit 731502e2bfa29fa6a88b5b7a4ccec9f74eac3606
Author: Thomas Herve <email address hidden>
Date: Tue Feb 20 16:20:00 2018 +0100

    Handle non-stored stack in resource group

    When creating a size 0 resource group with batch create policy, we could
    get into a situation where the resource was created but the stack
    wasn't. This fixes that, and still handle the problem in attribute
    resolution for backward compatibility.

    Change-Id: I8e139215281dc57fe4dd3ff856eef3ef88e20c6b
    Closes-Bug: #1750270

Changed in heat:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to heat (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/546574

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to heat (stable/queens)

Reviewed: https://review.openstack.org/546574
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=f8f7c32e254916e6255c695611ca894db02c4d5d
Submitter: Zuul
Branch: stable/queens

commit f8f7c32e254916e6255c695611ca894db02c4d5d
Author: Thomas Herve <email address hidden>
Date: Tue Feb 20 16:20:00 2018 +0100

    Handle non-stored stack in resource group

    When creating a size 0 resource group with batch create policy, we could
    get into a situation where the resource was created but the stack
    wasn't. This fixes that, and still handle the problem in attribute
    resolution for backward compatibility.

    Change-Id: I8e139215281dc57fe4dd3ff856eef3ef88e20c6b
    Closes-Bug: #1750270
    (cherry picked from commit 731502e2bfa29fa6a88b5b7a4ccec9f74eac3606)

tags: added: in-stable-queens
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/heat 10.0.0.0rc2

This issue was fixed in the openstack/heat 10.0.0.0rc2 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/heat 11.0.0.0b1

This issue was fixed in the openstack/heat 11.0.0.0b1 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.