Updating Nova::Server with Neutron::Port resource fails

[Dan Sneddon]

A Nova::Server resource that was created with an implicit port cannot be updated.

If I first create the following resource:
# template1.yaml
resources:
  my_ironic_instance:
    type: OS::Nova::Server
    properties:
      key_name: default
      image: overcloud-full
      flavor: baremetal
      networks:
        - network: ctlplane
          ip_address: "192.168.24.10"

And then try to run a stack update with a different ip_address:
# template2.yaml
resources:
  my_ironic_instance:
    type: OS::Nova::Server
    properties:
      key_name: default
      image: overcloud-full
      flavor: baremetal
      networks:
        - network: ctlplane
          ip_address: "192.168.24.20"

This fails with the following error:
RetryError: resources.my_ironic_instance: RetryError[<Future at 0x72fbad0 state=finished returned bool>]

I also tried assigning an external IP to the Nova::Server created in the template1.yaml, but that gave me the same error.
# template3.yaml
resources:
  instance_port:
    type: OS::Neutron::Port
    properties:
      network: ctlplane
      fixed_ips:
        - subnet: "ctlplane-subnet"
          ip_address: "192.168.24.20"

  my_ironic_instance:
    type: OS::Nova::Server
    properties:
      key_name: default
      image: overcloud-full
      flavor: baremetal
      networks:
        - network: ctlplane
          port: {get_resource: instance_port}

However, if I first create the Nova::Server resource with an external port specified (as in template3.yaml above), then I can update the port to a different IP address and Ironic/Neutron does the right thing (at least since the recent attach/detach VIF in Ironic code has merged). So it appears that you can update a port if the port was created externally, but not if the port was created as part of the Nova::Server resource.

[huangtianhua (huangtianhua)]

What version of heat? And I can't reproduce this problem as following steps in master:
1. create a stack with the template:
  th_server:
    type: OS::Nova::Server
    properties:
      name: test_server
      flavor: {get_param: flavor}
      image: {get_param: image}
      networks:
        - network: {get_param: net}
          fixed_ip: 10.0.0.24
2. update the stack to change the fixed_ip to '10.0.0.25', stack update success and the server ip is changed to the new one.

[Rabi Mishra (rabi)]

I think the issue with with baremetal (ironic behind nova). Our implict port creation stuff in OS::Nova::Server is not working with baremetal.

[Rabi Mishra (rabi)]

From the error it looks like it's failing with same vif attach/detach. Would you be able to share the heat engine nova logs?

[Dan Sneddon (dsneddon)]

Here are the applicable log snippets, let me know if you want any more before/after:

/var/log/heat/heat-engine.log:
#############
2017-05-22 23:14:52.991 10755 DEBUG novaclient.v2.client [req-bf42524f-28e2-4bee-93a9-ea9af0f9455a - admin - - -] REQ: curl -g -i -X GET http://192.168.24.3:8774/v2.1/servers/ed5c1500-a701-42a0-be71-da1
27c61742f -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-Auth-Token: {SHA1}53db7a95abe854503eb534fde8c81a0605197d83" _http_log_request /usr
/lib/python2.7/site-packages/keystoneauth1/session.py:347
2017-05-22 23:14:53.171 10755 DEBUG novaclient.v2.client [req-bf42524f-28e2-4bee-93a9-ea9af0f9455a - admin - - -] RESP: [200] Content-Length: 1694 Content-Type: application/json Openstack-Api-Version: c
ompute 2.1 X-Openstack-Nova-Api-Version: 2.1 Vary: OpenStack-API-Version, X-OpenStack-Nova-API-Version X-Compute-Request-Id: req-c1ded3c0-b482-4df9-b8bf-e71bb95f2eb3 Date: Mon, 22 May 2017 23:14:53 GMT
RESP BODY: {"server": {"OS-EXT-STS:task_state": null, "addresses": {"ctlplane": [{"OS-EXT-IPS-MAC:mac_addr": "00:08:15:24:63:75", "version": 4, "addr": "192.168.24.18", "OS-EXT-IPS:type": "fixed"}]}, "l
inks": [{"href": "http://192.168.24.3:8774/v2.1/servers/ed5c1500-a701-42a0-be71-da127c61742f", "rel": "self"}, {"href": "http://192.168.24.3:8774/servers/ed5c1500-a701-42a0-be71-da127c61742f", "rel": "b
ookmark"}], "image": {"id": "d1d7f682-e19a-4f38-995a-5d79d13c538f", "links": [{"href": "http://192.168.24.3:8774/images/d1d7f682-e19a-4f38-995a-5d79d13c538f", "rel": "bookmark"}]}, "OS-EXT-STS:vm_state"
: "active", "OS-EXT-SRV-ATTR:instance_name": "instance-00000005", "OS-SRV-USG:launched_at": "2017-05-22T22:55:51.000000", "flavor": {"id": "7e7cddba-05ec-41f1-b013-2cd042b5b3c7", "links": [{"href": "htt
p://192.168.24.3:8774/flavors/7e7cddba-05ec-41f1-b013-2cd042b5b3c7", "rel": "bookmark"}]}, "id": "ed5c1500-a701-42a0-be71-da127c61742f", "security_groups": [{"name": "default"}], "user_id": "5e8b5568e5f
b4c3598b829e45cad0ae6", "OS-DCF:diskConfig": "MANUAL", "accessIPv4": "", "accessIPv6": "", "progress": 0, "OS-EXT-STS:power_state": 1, "OS-EXT-AZ:availability_zone": "nova", "config_drive": "True", "sta
tus": "ACTIVE", "updated": "2017-05-22T22:55:51Z", "hostId": "7162f724b85a38f9e8206234bac96271a23b2d87aa68d7ee11121750", "OS-EXT-SRV-ATTR:host": "undercloud", "OS-SRV-USG:terminated_at": null, "key_name": "default", "OS-EXT-SRV-ATTR:hypervisor_hostname": "38b3dd32-71a5-4f95-9fa7-6f6670458a83", "name": "test_stack-my_ironic_instance-y2nqalnjzdl4", "created": "2017-05-22T22:53:19Z", "tenant_id": "fb04e494916f41ea8bb9976c53b71c92", "os-extended-volumes:volumes_attached": [], "metadata": {}}}
 _http_log_response /usr/lib/python2.7/site-packages/keystoneauth1/session.py:395
2017-05-22 23:14:53.171 10755 DEBUG novaclient.v2.client [req-bf42524f-28e2-4bee-93a9-ea9af0f9455a - admin - - -] GET call to compute for http://192.168.24.3:8774/v2.1/servers/ed5c1500-a701-42a0-be71-da127c61742f used request id req-c1ded3c0-b482-4df9-b8bf-e71bb95f2eb3 request /usr/lib/python2.7/site-packages/keystoneauth1/session.py:640
2017-05-22 23:14:53.172 10755 DEBUG novaclient.v2.client [req-bf4...

[Rabi Mishra (rabi)]

Looks to me like a neutron issue. neutron is looking for floatingips for the port/fixed_ip combination[1]. Not sure why? And probably floatingip extension is not available as this message suggests "No controller found for: floatingips - returning response code 404"

[1] 2017-05-22 23:15:41.822 12962 INFO neutron.wsgi [req-e54946bc-b617-4505-b963-d65ff8b65483 b429b29882af4f5b95787f646e860ebc 0fbac869a6d144f7abc7ae92788c9eb5 - default default] 192.168.24.1 "GET /v2.0/floatingips.json?fixed_ip_address=192.168.24.18&port_id=69e79425-38a8-4782-ab29-ed6f7c5ec981 HTTP/1.1" status: 404 len: 285 time: 0.0071950

[Hirofumi Ichihara (ichihara-hirofumi)]

In my opinion, even if nova gets the error from neutron, nova doesn't output Error[1]. It's not neutron issue.

[1]: https://github.com/openstack/nova/blob/master/nova/network/neutronv2/api.py#L1990-L2005

[Sean Dague (sdague)]

I don't believe this is implemented for baremetal back end. This is more of a spec to get that work in than a bug.