Can not add a new image in glance and use it to spawn VM in same template due to image validation

[Soumik Bhattacharya]

I am trying to create a new glance image, a Nova flavor, a new private Neutron network and then boot a couple of VMs using these resources. But stack creation is failing currently as it is trying to validate the image to check if it's a glance.image already and fails since it has not been created yet. IMO when I am using get_resource image ( vis-a-vis get_param image where an already existing glance.image is passed on by user ), Heat orchestration should try to create the image first. Because of this, I am unable to add a new image in glance and then use that image to boot up a VM in a single template.

The exact error received is as following :
================================

$ heat stack-create test1 -f ex0.yaml -e env0.yaml
ERROR: Property error : server1: image Error validating value u'image': The Image (image) could not be found.

The template used :
===============

parameters:
  disk_format:
    type: string
    description: Disk format of image to be used
  location:
    type: string
    description: http location of the image to be used
  image_name:
    type: string
    description: user-defined name for the new image created
  disk:
    type: number
    description: Root disk size in GB to use for VM flavor
  ram:
    type: number
    description: RAM size in MB to use for VM flavor
  vcpu:
    type: number
    description: # of vcpus to be used in VM flavor
  public_net:
    type: string
    description: ID or name of public network for which floating IP addresses will be allocated
  vm1_name:
    type: string
    description: VM1 name
  vm2_name:
    type: string
    description: VM2 name
  mgmt_net:
    type: string
    description: Name of mgmt network to be created
  mgmt_subnet_id:
    type: string
    description: Mgmt network address (CIDR notation)
  private_net_name:
    type: string
    description: Name of private network to be created
  private_net_cidr:
    type: string
    description: Private network address (CIDR notation)
  private_net_gateway:
    type: string
    description: Private network gateway address
  private_net_pool_start:
    type: string
    description: Start of private network IP address allocation pool
  private_net_pool_end:
    type: string
    description: End of private network IP address allocation pool

resources:
  image:
    type: OS::Glance::Image
    properties:
      disk_format: { get_param: disk_format }
      container_format: bare
      location: { get_param: location }
      name: { get_param: image_name }

  flavor:
    type: OS::Nova::Flavor
    properties:
      disk: { get_param: disk }
      ram: { get_param: ram }
      vcpus: { get_param: vcpu }

  private_net:
    type: OS::Neutron::Net
    properties:
      name: { get_param: private_net_name }

  private_subnet:
    type: OS::Neutron::Subnet
    properties:
      network_id: { get_resource: private_net }
      cidr: { get_param: private_net_cidr }
      gateway_ip: { get_param: private_net_gateway }
      allocation_pools:
        - start: { get_param: private_net_pool_start }
          end: { get_param: private_net_pool_end }

  server1:
    type: OS::Nova::Server
    properties:
      name: { get_param: vm1_name }
      image: { get_resource: image }
      flavor: { get_resource: flavor }
      networks:
        - port: { get_resource: server1_port1 }
        - network: { get_resource: private_net }

  server1_port1:
    type: OS::Neutron::Port
    properties:
      network_id: { get_param: mgmt_net }
      fixed_ips:
        - subnet_id: { get_param: mgmt_subnet_id }
      security_groups: [{ get_resource: server_security_group }]

  server1_floating_ip:
    type: OS::Neutron::FloatingIP
    properties:
      floating_network_id: { get_param: public_net }
      port_id: { get_resource: server1_port1 }

  server2:
    type: OS::Nova::Server
    properties:
      name: { get_param: vm2_name }
      image: { get_resource: image }
      flavor: { get_resource: flavor }
      networks:
        - port: { get_resource: server2_port1 }
        - network: { get_resource: private_net }

  server2_port1:
    type: OS::Neutron::Port
    properties:
      network_id: { get_param: mgmt_net }
      fixed_ips:
        - subnet_id: { get_param: mgmt_subnet_id }
      security_groups: [{ get_resource: server_security_group }]

  server2_floating_ip:
    type: OS::Neutron::FloatingIP
    properties:
      floating_network_id: { get_param: public_net }
      port_id: { get_resource: server2_port1 }

  server_security_group:
    type: OS::Neutron::SecurityGroup
    properties:
      description: Add security group rules for server
      name: security-group
      rules:
        - remote_ip_prefix: 0.0.0.0/0
          protocol: tcp
          port_range_min: 22
          port_range_max: 22
        - remote_ip_prefix: 0.0.0.0/0
          protocol: icmp

outputs:
  server1_private_ip:
    description: IP address of server1 in private network
    value: { get_attr: [ server1, first_address ] }
  server1_public_ip:
    description: Floating IP address of server1 in public network
    value: { get_attr: [ server1_floating_ip, floating_ip_address ] }
  server2_private_ip:
    description: IP address of server2 in private network
    value: { get_attr: [ server2, first_address ] }
  server2_public_ip:
    description: Floating IP address of server2 in public network
    value: { get_attr: [ server2_floating_ip, floating_ip_address ] }

[huangtianhua (huangtianhua)]

I think may be the fllowing patch can solve this problem:

https://review.openstack.org/#/c/93080/

[Soumik Bhattacharya (soumikb-x)]

Thanks for the response - I tried to add the patch as retried after restarting heat engine and api - but the same error result. The heat engine threw error log as :
...
...
...
2014-07-10 05:02:33.406 DEBUG iso8601.iso8601 [req-85f6a5cd-4484-48b6-9f93-50c23cb0151f admin demo] Got u'33' for 'second' with default None from (pid=5571) to_int /usr/lib/python2.7/dist-packages/iso8601/iso8601.py:140
2014-07-10 05:02:33.412 DEBUG glanceclient.common.http [req-85f6a5cd-4484-48b6-9f93-50c23cb0151f admin demo] curl -i -X GET -H 'X-Auth-Token: ***' -H 'Content-Type: application/json' -H 'User-Agent: python-glanceclient' http://10.92.249.81:9292/v1/images/detail?limit=20&name=image from (pid=5571) log_curl_request /opt/stack/python-glanceclient/glanceclient/common/http.py:156
2014-07-10 05:02:34.008 DEBUG glanceclient.common.http [req-85f6a5cd-4484-48b6-9f93-50c23cb0151f admin demo]
HTTP/1.1 200 OK
date: Thu, 10 Jul 2014 09:02:34 GMT
content-length: 14
content-type: application/json; charset=UTF-8
x-openstack-request-id: req-290ff40f-00a6-44bb-8966-1d35fadf788f

{"images": []}
 from (pid=5571) log_http_response /opt/stack/python-glanceclient/glanceclient/common/http.py:170
2014-07-10 05:02:34.009 INFO heat.engine.resources.glance_utils [req-85f6a5cd-4484-48b6-9f93-50c23cb0151f admin demo] Image image was not found in glance
2014-07-10 05:02:34.010 ERROR heat.engine.parser [req-85f6a5cd-4484-48b6-9f93-50c23cb0151f admin demo] Property error : server1: image Error validating value u'image': The Image (image) could not be found.
2014-07-10 05:02:34.010 TRACE heat.engine.parser Traceback (most recent call last):
2014-07-10 05:02:34.010 TRACE heat.engine.parser File "/opt/stack/heat/heat/engine/parser.py", line 418, in validate
2014-07-10 05:02:34.010 TRACE heat.engine.parser result = res.validate()
2014-07-10 05:02:34.010 TRACE heat.engine.parser File "/opt/stack/heat/heat/engine/resources/server.py", line 863, in validate
2014-07-10 05:02:34.010 TRACE heat.engine.parser super(Server, self).validate()
2014-07-10 05:02:34.010 TRACE heat.engine.parser File "/opt/stack/heat/heat/engine/resource.py", line 640, in validate
2014-07-10 05:02:34.010 TRACE heat.engine.parser return self.properties.validate()
2014-07-10 05:02:34.010 TRACE heat.engine.parser File "/opt/stack/heat/heat/engine/properties.py", line 324, in validate
2014-07-10 05:02:34.010 TRACE heat.engine.parser raise exception.StackValidationFailed(message=msg)
2014-07-10 05:02:34.010 TRACE heat.engine.parser StackValidationFailed: Property error : server1: image Error validating value u'image': The Image (image) could not be found.
2014-07-10 05:02:34.010 TRACE heat.engine.parser
2014-07-10 05:02:34.013 DEBUG oslo.messaging.rpc.dispatcher [req-85f6a5cd-4484-48b6-9f93-50c23cb0151f admin demo] Expected exception during message handling (Property error : server1: image Error validating value u'image': The Image (image) could not be found.) from (pid=5571) _dispatch_and_reply /usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py:137
...
...
...

[Qiming Teng (tengqim)]

how about this?

  server1:
    depends_on: image
    properties:
      image: {get_param: image_name}

  server2:
    depends_on: image
    properties:
      image: {get_param: image_name}

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/108951
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=e1e852856047313159699f6a814135b6a6fde5e5
Submitter: Jenkins
Branch: master

commit e1e852856047313159699f6a814135b6a6fde5e5
Author: Sergey Kraynev <email address hidden>
Date: Wed Jul 23 06:10:02 2014 -0400

    Skip validation if depends on not created resource

    Using references on resources defined in template may be cause
    of error during validation. This fix checks reference on another
    resource and skips validation if resource is in INIT state.

    Change-Id: I95531a1603de1e8c4b9f0f4b05b62eebc48beb3c
    Co-Authored-By: Zane Bitter <email address hidden>
    Closes-Bug: #1347571
    Closes-Bug: #1339942