Update AutoScale stack with nested template fails when you add or remove OS::Neutron::Pool resource

Bug #1453923 reported by Lisa Armstrong
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Heat
Fix Released
Medium
Rabi Mishra
Kilo
Fix Released
Medium
Rabi Mishra

Bug Description

Whenever I try to do an update to add a OS::Neutron::PoolMember resource to the nested template to enable load balancing on my stack, the update fails with this error:
Resource failed - Unknown status FAILED due to "StackValidationFailed: Property error : ruuxgt6l6hx5.Properties: Unknown Property pool_id"
But then if I do the update again immediately after with the exact same template (and nested template) the update will succeed. The same thing happens when I try to remove the pool member resource also.
The templates I use are named the same same thing whenever I do the update
(i.e heat stack-create -f autoscale.yaml my_asg
then I modify the autoscale.yaml file and the nested template file and run heat stack-update -f autoscale.yaml my_asg)

Here is my autoscale template without load balancing
{
    "description": "A document-based template to configure your Software Defined Environment.\n",
    "heat_template_version": "2013-05-23",
    "outputs": {

    },
    "parameters": {
        "availability_zone": {
            "description": "The Availability Zone to launch the instance.",
            "type": "string",
            "default": "nova"
        },
        "cooldown_seconds": {
            "default": 30,
            "description": "Number of seconds for autoscaling cooldown",
            "type": "number"
        },
        "cpu_high_threshold": {
            "default": 50,
            "description": "CPU usage to trigger scale up",
            "type": "number"
        },
        "cpu_low_threshold": {
            "default": 20,
            "description": "CPU usage to trigger scale down",
            "type": "number"
        },
        "cpu_monitor_seconds": {
            "default": 600,
            "description": "Period of time the cpu has to be at threshold",
            "type": "number"
        },
        "desired_instances": {
            "description": "Desired initial number of instances",
            "type": "number",
            "default": 1
        },
        "image_id": {
            "description": "Image to be used for compute instance",
            "label": "Image ID",
            "type": "string",
            "default": "577482ca-5641-4ed4-8fa3-3c856cb96e66"
        },
        "instance_type": {
            "description": "Type of instance (flavor) to be used",
            "label": "Instance Type",
            "type": "string",
            "default": "m1.tiny"
        },
        "max_instances": {
            "description": "Maximum number of instances",
            "type": "number",
            "default": 3
        },
        "min_instances": {
            "description": "Minimum number of instances",
            "type": "number",
            "default": 1
        },
        "key_name":{
            "description": "Name of the nova keypair.",
            "type": "string",
            "default": "rb_cloud"
        }
    },
    "resources": {
        "cpu_alarm_high": {
            "properties": {
                "alarm_actions": [
                    {
                        "get_attr": [
                            "scaleup_policy",
                            "alarm_url"
                        ]
                    }
                ],
                "comparison_operator": "gt",
                "evaluation_periods": 1,
                "matching_metadata": {
                    "metadata.user_metadata.stack": {
                        "get_param": "OS::stack_id"
                    }
                },
                "meter_name": "cpu_util",
                "period": {
                    "get_param": "cpu_monitor_seconds"
                },
                "statistic": "avg",
                "threshold": {
                    "get_param": "cpu_high_threshold"
                }
            },
            "type": "OS::Ceilometer::Alarm"
        },
        "cpu_alarm_low": {
            "properties": {
                "alarm_actions": [
                    {
                        "get_attr": [
                            "scaledown_policy",
                            "alarm_url"
                        ]
                    }
                ],
                "comparison_operator": "lt",
                "evaluation_periods": 1,
                "matching_metadata": {
                    "metadata.user_metadata.stack": {
                        "get_param": "OS::stack_id"
                    }
                },
                "meter_name": "cpu_util",
                "period": {
                    "get_param": "cpu_monitor_seconds"
                },
                "statistic": "avg",
                "threshold": {
                    "get_param": "cpu_low_threshold"
                }
            },
            "type": "OS::Ceilometer::Alarm"
        },
        "my_asg": {
            "properties": {
                "desired_capacity": {
                    "get_param": "desired_instances"
                },
                "max_size": {
                    "get_param": "max_instances"
                },
                "min_size": {
                    "get_param": "min_instances"
                },
                "resource": {
                    "properties": {
                        "availability_zone": {
                            "get_param": "availability_zone"
                        },
                        "flavor": {
                            "get_param": "instance_type"
                        },
                        "image": {
                            "get_param": "image_id"
                        },
                        "metadata": {
                            "metering.stack": {
                                "get_param": "OS::stack_id"
                            }
                        },
                        "key_name": {
                            "get_param": "key_name"
                        }
                    },
                    "type": "lb-member.yaml"
                }
            },
            "type": "OS::Heat::AutoScalingGroup"
        },
        "scaledown_policy": {
            "properties": {
                "adjustment_type": "change_in_capacity",
                "auto_scaling_group_id": {
                    "get_resource": "my_asg"
                },
                "cooldown": {
                    "get_param": "cooldown_seconds"
                },
                "scaling_adjustment": -1
            },
            "type": "OS::Heat::ScalingPolicy"
        },
        "scaleup_policy": {
            "properties": {
                "adjustment_type": "change_in_capacity",
                "auto_scaling_group_id": {
                    "get_resource": "my_asg"
                },
                "cooldown": {
                    "get_param": "cooldown_seconds"
                },
                "scaling_adjustment": 1
            },
            "type": "OS::Heat::ScalingPolicy"
        }
    }
}

and the nested template
{
    "description": "A document-based template to configure your Software Defined Environment.",
    "heat_template_version": "2013-05-23",
    "parameters": {
        "availability_zone": {
            "description": "AZ for the Docker instances",
            "type": "string"
        },
        "flavor": {
            "description": "Flavor to be used for compute instance",
            "type": "string"
        },
        "image": {
            "description": "Image used for servers",
            "type": "string"
        },
        "metadata": {
            "type": "json"
        },
        "key_name": {
            "description": "SSH key",
            "type": "string"
        }
    },
    "resources": {
        "server": {
            "properties": {
                "availability_zone": {
                    "get_param": "availability_zone"
                },
                "flavor": {
                    "get_param": "flavor"
                },
                "image": {
                    "get_param": "image"
                },
                "key_name": {
                    "get_param": "key_name"
                },
                "metadata": {
                    "get_param": "metadata"
                }
            },
            "type": "OS::Nova::Server"
        }
    }
}

Here is my autoscale template WITH the load balancer:

{
    "description": "A document-based template to configure your Software Defined Environment.\n",
    "heat_template_version": "2013-05-23",
    "outputs": {

    },
    "parameters": {
        "availability_zone": {
            "description": "The Availability Zone to launch the instance.",
            "type": "string",
            "default": "nova"
        },
        "cooldown_seconds": {
            "default": 30,
            "description": "Number of seconds for autoscaling cooldown",
            "type": "number"
        },
        "cpu_high_threshold": {
            "default": 50,
            "description": "CPU usage to trigger scale up",
            "type": "number"
        },
        "cpu_low_threshold": {
            "default": 20,
            "description": "CPU usage to trigger scale down",
            "type": "number"
        },
        "cpu_monitor_seconds": {
            "default": 600,
            "description": "Period of time the cpu has to be at threshold",
            "type": "number"
        },
        "desired_instances": {
            "description": "Desired initial number of instances",
            "type": "number",
            "default": 1
        },
        "image_id": {
            "description": "Image to be used for compute instance",
            "label": "Image ID",
            "type": "string",
            "default": "577482ca-5641-4ed4-8fa3-3c856cb96e66"
        },
        "instance_protocol_port": {
            "description": "Port loadbalancer should use to connect to instances",
            "type": "number",
            "default": 80
        },
        "instance_type": {
            "description": "Type of instance (flavor) to be used",
            "label": "Instance Type",
            "type": "string",
            "default": "m1.tiny"
        },
        "load_balancer_name": {
            "description": "Name of the load balancer that is fronting this ASG for easy reference",
            "type": "string",
            "default": "red-black-test"
        },
        "load_balancer_pool_id": {
            "description": "Pool ID of the load balancer in which the instances should be deployed to",
            "type": "string",
            "default": "a3dc761a-7925-44d6-b9ac-6ac39ff4e932"
        },
        "max_instances": {
            "description": "Maximum number of instances",
            "type": "number",
            "default": 3
        },
        "min_instances": {
            "description": "Minimum number of instances",
            "type": "number",
            "default": 1
        },
        "key_name":{
            "description": "Name of the nova keypair.",
            "type": "string",
            "default": "rb_cloud"
        }
    },
    "resources": {
        "cpu_alarm_high": {
            "properties": {
                "alarm_actions": [
                    {
                        "get_attr": [
                            "scaleup_policy",
                            "alarm_url"
                        ]
                    }
                ],
                "comparison_operator": "gt",
                "evaluation_periods": 1,
                "matching_metadata": {
                    "metadata.user_metadata.stack": {
                        "get_param": "OS::stack_id"
                    }
                },
                "meter_name": "cpu_util",
                "period": {
                    "get_param": "cpu_monitor_seconds"
                },
                "statistic": "avg",
                "threshold": {
                    "get_param": "cpu_high_threshold"
                }
            },
            "type": "OS::Ceilometer::Alarm"
        },
        "cpu_alarm_low": {
            "properties": {
                "alarm_actions": [
                    {
                        "get_attr": [
                            "scaledown_policy",
                            "alarm_url"
                        ]
                    }
                ],
                "comparison_operator": "lt",
                "evaluation_periods": 1,
                "matching_metadata": {
                    "metadata.user_metadata.stack": {
                        "get_param": "OS::stack_id"
                    }
                },
                "meter_name": "cpu_util",
                "period": {
                    "get_param": "cpu_monitor_seconds"
                },
                "statistic": "avg",
                "threshold": {
                    "get_param": "cpu_low_threshold"
                }
            },
            "type": "OS::Ceilometer::Alarm"
        },
        "my_asg": {
            "properties": {
                "desired_capacity": {
                    "get_param": "desired_instances"
                },
                "max_size": {
                    "get_param": "max_instances"
                },
                "min_size": {
                    "get_param": "min_instances"
                },
                "resource": {
                    "properties": {
                        "availability_zone": {
                            "get_param": "availability_zone"
                        },
                        "flavor": {
                            "get_param": "instance_type"
                        },
                        "image": {
                            "get_param": "image_id"
                        },
                        "instance_protocol_port": {
                            "get_param": "instance_protocol_port"
                        },
                        "metadata": {
                            "metering.stack": {
                                "get_param": "OS::stack_id"
                            }
                        },
                        "pool_id": {
                            "get_param": "load_balancer_pool_id"
                        },
                        "key_name": {
                            "get_param": "key_name"
                        }
                    },
                    "type": "lb-member.yaml"
                }
            },
            "type": "OS::Heat::AutoScalingGroup"
        },
        "scaledown_policy": {
            "properties": {
                "adjustment_type": "change_in_capacity",
                "auto_scaling_group_id": {
                    "get_resource": "my_asg"
                },
                "cooldown": {
                    "get_param": "cooldown_seconds"
                },
                "scaling_adjustment": -1
            },
            "type": "OS::Heat::ScalingPolicy"
        },
        "scaleup_policy": {
            "properties": {
                "adjustment_type": "change_in_capacity",
                "auto_scaling_group_id": {
                    "get_resource": "my_asg"
                },
                "cooldown": {
                    "get_param": "cooldown_seconds"
                },
                "scaling_adjustment": 1
            },
            "type": "OS::Heat::ScalingPolicy"
        }
    }
}

The lb-member nested template with LB pool member
{
    "description": "A document-based template to configure your Software Defined Environment.",
    "heat_template_version": "2013-05-23",
    "parameters": {
        "availability_zone": {
            "description": "AZ for the Docker instances",
            "type": "string"
        },
        "flavor": {
            "description": "Flavor to be used for compute instance",
            "type": "string"
        },
        "image": {
            "description": "Image used for servers",
            "type": "string"
        },
        "instance_protocol_port": {
            "description": "Port loadbalancer should use to connect to instances",
            "type": "number"
        },
        "metadata": {
            "type": "json"
        },
        "pool_id": {
            "description": "Generated to reference pool",
            "type": "string"
        },
        "key_name": {
            "description": "SSH key",
            "type": "string"
        }
    },
    "resources": {
        "lbaas_pool_member": {
            "properties": {
                "address": {
                    "get_attr": [
                        "server",
                        "first_address"
                    ]
                },
                "pool_id": {
                    "get_param": "pool_id"
                },
                "protocol_port": {
                    "get_param": "instance_protocol_port"
                }
            },
            "type": "OS::Neutron::PoolMember"
        },
        "server": {
            "properties": {
                "availability_zone": {
                    "get_param": "availability_zone"
                },
                "flavor": {
                    "get_param": "flavor"
                },
                "image": {
                    "get_param": "image"
                },
                "key_name": {
                    "get_param": "key_name"
                },
                "metadata": {
                    "get_param": "metadata"
                }
            },
            "type": "OS::Nova::Server"
        }
    }
}

tags: added: autoscaling
Changed in heat:
status: New → Triaged
importance: Undecided → Medium
Rabi Mishra (rabi)
Changed in heat:
assignee: nobody → Rabi Mishra (rabi)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to heat (master)

Fix proposed to branch: master
Review: https://review.openstack.org/182763

Changed in heat:
status: Triaged → In Progress
tags: added: kilo-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/184850

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on heat (master)

Change abandoned by Rabi Mishra (<email address hidden>) on branch: master
Review: https://review.openstack.org/184850
Reason: should be for kilo

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to heat (stable/kilo)

Fix proposed to branch: stable/kilo
Review: https://review.openstack.org/184862

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to heat (master)

Reviewed: https://review.openstack.org/182763
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=141597f032a38fdd96c97c14c0517db9463dd9c9
Submitter: Jenkins
Branch: master

commit 141597f032a38fdd96c97c14c0517db9463dd9c9
Author: Rabi Mishra <email address hidden>
Date: Wed May 13 16:39:04 2015 +0530

    Fix property validation for TemplateResource during update

    This fix adds schema regeneration before props are validated
    for TemplateResource during stack-update.

    Change-Id: Ia79a25083489f48ed7332fbbd089a0090452cdc4
    Closes-Bug: #1452983
    Closes-Bug: #1453923

Changed in heat:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to heat (stable/kilo)

Reviewed: https://review.openstack.org/184862
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=ed2bed2555c85dc8d830477194065654400c5627
Submitter: Jenkins
Branch: stable/kilo

commit ed2bed2555c85dc8d830477194065654400c5627
Author: Rabi Mishra <email address hidden>
Date: Wed May 13 16:39:04 2015 +0530

    Fix property validation for TemplateResource during update

    This fix adds schema regeneration before props are validated
    for TemplateResource during stack-update.

    Change-Id: Ia79a25083489f48ed7332fbbd089a0090452cdc4
    Closes-Bug: #1452983
    Closes-Bug: #1453923
    (cherry picked from commit 141597f032a38fdd96c97c14c0517db9463dd9c9)

Thierry Carrez (ttx)
Changed in heat:
milestone: none → liberty-1
status: Fix Committed → Fix Released
tags: added: in-stable-kilo
removed: kilo-backport-potential
Thierry Carrez (ttx)
Changed in heat:
milestone: liberty-1 → 5.0.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.