scenario007 failing with resources.ServiceChain: resources.BlockStorageServiceChain.Property error: resources[2].properties: Property DockerCinderConfigImage not assigned

[Alex Schultz]

Scenario007 is currently failing because of a missing DockerCinderConfigImage

http://logs.openstack.org/71/584771/18/check/tripleo-ci-centos-7-scenario007-multinode-oooq-container/d4b2f7d/logs/undercloud/var/log/extra/errors.txt.gz#_2018-07-25_16_04_08_991

2018-07-25 16:04:07.506 ERROR /var/log/containers/heat/heat-engine.log: 28 ERROR heat.engine.resource
2018-07-25 16:04:08.991 ERROR /var/log/containers/heat/heat-engine.log: 31 ERROR heat.engine.resource Traceback (most recent call last):
2018-07-25 16:04:08.991 ERROR /var/log/containers/heat/heat-engine.log: 31 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 926, in _action_recorder
2018-07-25 16:04:08.991 ERROR /var/log/containers/heat/heat-engine.log: 31 ERROR heat.engine.resource yield
2018-07-25 16:04:08.991 ERROR /var/log/containers/heat/heat-engine.log: 31 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 1034, in _do_action
2018-07-25 16:04:08.991 ERROR /var/log/containers/heat/heat-engine.log: 31 ERROR heat.engine.resource yield self.action_handler_task(action, args=handler_args)
2018-07-25 16:04:08.991 ERROR /var/log/containers/heat/heat-engine.log: 31 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/scheduler.py", line 346, in wrapper
2018-07-25 16:04:08.991 ERROR /var/log/containers/heat/heat-engine.log: 31 ERROR heat.engine.resource step = next(subtask)
2018-07-25 16:04:08.991 ERROR /var/log/containers/heat/heat-engine.log: 31 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 985, in action_handler_task
2018-07-25 16:04:08.991 ERROR /var/log/containers/heat/heat-engine.log: 31 ERROR heat.engine.resource done = check(handler_data)
2018-07-25 16:04:08.991 ERROR /var/log/containers/heat/heat-engine.log: 31 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resources/stack_resource.py", line 409, in check_create_complete
2018-07-25 16:04:08.991 ERROR /var/log/containers/heat/heat-engine.log: 31 ERROR heat.engine.resource return self._check_status_complete(self.CREATE)
2018-07-25 16:04:08.991 ERROR /var/log/containers/heat/heat-engine.log: 31 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resources/stack_resource.py", line 459, in _check_status_complete
2018-07-25 16:04:08.991 ERROR /var/log/containers/heat/heat-engine.log: 31 ERROR heat.engine.resource action=action)
2018-07-25 16:04:08.991 ERROR /var/log/containers/heat/heat-engine.log: 31 ERROR heat.engine.resource ResourceFailure: resources.ServiceChain: resources.BlockStorageServiceChain.Property error: resources[2].properties: Property DockerCinderConfigImage not assigned

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/585908

[Steve Baker (steve-stevebaker)]

This is strange. container prepare is working correctly by not setting DockerCinderConfigImage, because that job is not deploying BlockStorage.

Sometimes overcloud.BlockStorageServiceChain.ServiceChain is created correctly:
http://logs.openstack.org/71/584771/19/check/tripleo-ci-centos-7-scenario007-multinode-oooq-container/0845ead/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-07-26_01_18_00

And sometimes it fails on mandatory parameter validation even though we're deploying zero of that service:
http://logs.openstack.org/08/585908/1/check/tripleo-ci-centos-7-scenario007-multinode-oooq-container/2917c3d/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-07-26_00_37_05

Its like heat is inconsistently validating OS::Heat::ResourceChain resources

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/585908
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=60124eb653cfab2d3af1f69be7b365398b011c8d
Submitter: Zuul
Branch: master

commit 60124eb653cfab2d3af1f69be7b365398b011c8d
Author: Steve Baker <email address hidden>
Date: Thu Jul 26 11:05:01 2018 +1200

    Add missing service for DockerCinderConfigImage

    OS::TripleO::Services::BlockStorageCinderVolume has a mandatory
    DockerCinderConfigImage parameter, but is missing an entry here, this
    change adds that entry.

    Change-Id: I20f04e8b821058a89a5464c9be1bb14d30f1f223
    Closes-Bug: 1783653

[Alex Schultz (alex-schultz)]

Dropping alert/ci/promotion blocker since the job is now passing.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/586715

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/586715
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=60ae8677c0293e0137ce62ca5774e6d4b03c06cb
Submitter: Zuul
Branch: stable/queens

commit 60ae8677c0293e0137ce62ca5774e6d4b03c06cb
Author: Steve Baker <email address hidden>
Date: Thu Jul 26 11:05:01 2018 +1200

    Add missing service for DockerCinderConfigImage

    OS::TripleO::Services::BlockStorageCinderVolume has a mandatory
    DockerCinderConfigImage parameter, but is missing an entry here, this
    change adds that entry.

    Change-Id: I20f04e8b821058a89a5464c9be1bb14d30f1f223
    Closes-Bug: 1783653
    (cherry picked from commit 60124eb653cfab2d3af1f69be7b365398b011c8d)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-common 8.6.4 release.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/590576

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/590576
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=0f8138ddd657cede7a3be98b8e81462862b796da
Submitter: Zuul
Branch: master

commit 0f8138ddd657cede7a3be98b8e81462862b796da
Author: Steve Baker <email address hidden>
Date: Fri Aug 10 10:45:15 2018 +1200

    Add missing service for DockerCinderVolumeImage

    OS::TripleO::Services::BlockStorageCinderVolume has a mandatory
    DockerCinderVolumeImage parameter, but is missing an entry here, this
    change adds that entry.

    Change-Id: Ib818df1e2bcde3aa1713ad10b9aecc97d1b4eff8
    Related-Bug: #1783653

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/queens
Review: https://review.openstack.org/595357

[Daniel Alvarez (dalvarezs)]

I'm still hitting this after the recheck... maybe I'm missing something?
http://logs.openstack.org/30/587830/4/check/tripleo-ci-centos-7-scenario007-multinode-oooq-container/2ac0340/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-08-23_14_02_09

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-common 9.3.0 release.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Alex Schultz (<email address hidden>) on branch: stable/queens
Review: https://review.openstack.org/595357
Reason: http://lists.openstack.org/pipermail/openstack-dev/2018-September/135224.html

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/595357
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=f51047cd7c6de80e862e5b1700c7e04a65772b9e
Submitter: Zuul
Branch: stable/queens

commit f51047cd7c6de80e862e5b1700c7e04a65772b9e
Author: Steve Baker <email address hidden>
Date: Fri Aug 10 10:45:15 2018 +1200

    Add missing service for DockerCinderVolumeImage

    OS::TripleO::Services::BlockStorageCinderVolume has a mandatory
    DockerCinderVolumeImage parameter, but is missing an entry here, this
    change adds that entry.

    Change-Id: Ib818df1e2bcde3aa1713ad10b9aecc97d1b4eff8
    Related-Bug: #1783653
    (cherry picked from commit 0f8138ddd657cede7a3be98b8e81462862b796da)