keystone_endpoint report undefined method `[]' for nil:NilClass when executing instances method

Bug #1559013 reported by Liao Penghui
16
This bug affects 3 people
Affects Status Importance Assigned to Milestone
puppet-keystone
Fix Released
Critical
Sofer Athlan-Guyot

Bug Description

This is a bug in the lib/puppet/provider/keystone_endpoint/openstack.rb. And this bug will trigger prefetch method fail when puppet runs, this will lead to adding duplicate endpoints that already exist cause prefetch method won't return actual results.

Now keystone_endpoint is managing admin/internal/public url in only one resource. The code below is triggering this bug:

```
  def self.instances
    names = []
    list = []
    endpoints.each do |current|
      name = transform_name(current[:region], current[:service_name], current[:service_type])
      unless names.include?(name)
        names << name
        endpoint = { :name => name, current[:interface].to_sym => current }
        endpoints.each do |ep_osc|
          if (ep_osc[:id] != current[:id]) &&
            (ep_osc[:service_name] == current[:service_name]) &&
            (ep_osc[:service_type] == current[:service_type]) &&
            (ep_osc[:region] == current[:region])
            endpoint.merge!(ep_osc[:interface].to_sym => ep_osc)
          end
        end
        list << endpoint
      end
    end
    list.collect do |endpoint|
      new(
        :name => endpoint[:name],
        :ensure => :present,
        :id => "#{endpoint[:admin][:id]},#{endpoint[:internal][:id]},#{endpoint[:public][:id]}",
        :region => endpoint[:admin][:region],
        :admin_url => endpoint[:admin][:url],
        :internal_url => endpoint[:internal][:url],
        :public_url => endpoint[:public][:url]
      )
    end
  end
```

Say I have only admin and internal url of neutron, the endpoint[:public] is nil, and endpoint[:public][:id] will throw an error of undefined method `[]' for nil:NilClass. Same of endpoint[:public][:id], endpoint[:admin][:url] and endpoint[:admin][:region] is problematic. We need add some nil? check in the code above.

Secondly, even if I can fix this bug by adding some nil check. Say I do not have admin url of neutron. What I expect to happen after a puppet run is that adding a missing admin url of neutron. But the actual behavior is that puppet can only change exist admin url to a right one but not adding a new one. See code below

```
  def flush
    if @property_flush && @property_hash[:id]
      ids = @property_hash[:id].split(',')
      if @property_flush[:admin_url]
        self.class.request('endpoint', 'set', [ids[0], "--url=#{resource[:admin_url]}"])
      end
      if @property_flush[:internal_url]
        self.class.request('endpoint', 'set', [ids[1], "--url=#{resource[:internal_url]}"])
      end
      if @property_flush[:public_url]
        self.class.request('endpoint', 'set', [ids[2], "--url=#{resource[:public_url]}"])
      end
    end
    @property_hash = resource.to_hash
  end
```

The flush method can change a wrong endpoint to a right one, but dose not support adding a missing endpoint. So we need to add some logic here to handle that when some of admin/internal/public endpoint is missing and add the missing one.

I'm not a ruby guru but I'm trying my best to fix these bugs. I've already commit a patch and tested it. Welcome for advices.

description: updated
Changed in puppet-keystone:
assignee: nobody → Liao Penghui (liaoishere)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to puppet-keystone (master)

Fix proposed to branch: master
Review: https://review.openstack.org/294506

Changed in puppet-keystone:
status: New → In Progress
Xingchao Yu (yuxcer)
Changed in puppet-keystone:
importance: Undecided → Critical
Revision history for this message
Sofer Athlan-Guyot (sofer-athlan-guyot) wrote :

Hi,

I could reproduce the problem by
 1. bootstraping a keystone server using this manifest https://github.com/openstack/puppet-keystone/blob/master/spec/acceptance/keystone_wsgi_apache_spec.rb#L9-L68

    +----------------------------------+-----------+--------------+--------------+---------+-----------+-----------------------------+
    | ID | Region | Service Name | Service Type | Enabled | Interface | URL |
    +----------------------------------+-----------+--------------+--------------+---------+-----------+-----------------------------+
    | 0937b98eca5944fbb8fc1c1f55f76e67 | RegionOne | beakerv3 | beakerv3 | True | admin | http://127.0.0.1:1234/v3 |
    | 2d91a27c2d124420b0c4a1fb45589e23 | RegionOne | keystone | identity | True | admin | http://127.0.0.1:35357/v2.0 |
    | 3e6f0a6c562a4d4fa13393336f95b6fa | RegionOne | keystone | identity | True | internal | http://127.0.0.1:5000/v2.0 |
    | 45f9f23448454dc79879a10f3fc2a182 | RegionOne | beakerv3 | beakerv3 | True | public | http://127.0.0.1:1234/v3 |
    | 4caf7eb54da84b3fa1942cbef67a9147 | RegionOne | beaker | beaker | True | internal | http://127.0.0.1:1234 |
    | 898bc682514542139258d3c73d1d55a8 | RegionOne | beaker | beaker | True | admin | http://127.0.0.1:1234 |
    | 9ad98d975e624f90a5c590f9b03b0886 | RegionOne | beaker | beaker | True | public | http://127.0.0.1:1234 |
    | a7aaad39f0fb43cba3dd407118895cc0 | RegionOne | beakerv3 | beakerv3 | True | internal | http://127.0.0.1:1234/v3 |
    | be02f96703f94820879a5dab68db3a9c | RegionOne | keystone | identity | True | public | http://127.0.0.1:5000/v2.0 |
    +----------------------------------+-----------+--------------+--------------+---------+-----------+-----------------------------+

 2. delete one endpoint (beaker/internal): openstack endpoint delete 4caf7eb54da84b3fa1942cbef67a9147
 3. rerun the manifest:

    Debug: Executing '/bin/openstack endpoint list --quiet --format csv'
    Error: Could not prefetch keystone_endpoint provider 'openstack': undefined method `[]' for nil:NilClass

And all endpoints were recreated.

description: updated
Revision history for this message
Matt Fischer (mfisch) wrote :

Repro case is easy and mine matches Sofer's. I'm not sure about the fix, I left a comment there. Was anyone able to test it?

Revision history for this message
Sofer Athlan-Guyot (sofer-athlan-guyot) wrote :

Liao, would you mind if I take over the fix ?

Revision history for this message
Liao Penghui (liaoishere) wrote :

Sofer, You can take over this fix cause I'm busy on other things. Thanks for your effort.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/331623

Changed in puppet-keystone:
assignee: Liao Penghui (liaoishere) → Sofer Athlan-Guyot (sofer-athlan-guyot)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to puppet-keystone (master)

Reviewed: https://review.openstack.org/331623
Committed: https://git.openstack.org/cgit/openstack/puppet-keystone/commit/?id=fe0edef97dc0b62828bb0420550dd082b99510a6
Submitter: Jenkins
Branch: master

commit fe0edef97dc0b62828bb0420550dd082b99510a6
Author: Sofer Athlan-Guyot <email address hidden>
Date: Mon Jun 20 13:14:58 2016 +0200

    Fix endpoint update when one endpoint is missing.

    Endpoint are created for admin, internal and public network by this
    provider. If only one of the endpoint is missing then all the endpoints
    are recreated as puppet fails to match the resource with the remaining
    endpoint.

    This fix enable one to update the resource where update means "recreate
    the missing endpoint".

    Change-Id: Ic605725d1923680c6518ebadda36cb5d596c08fe
    Closes-bug: 1559013

Changed in puppet-keystone:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on puppet-keystone (master)

Change abandoned by Liao Penghui (<email address hidden>) on branch: master
Review: https://review.openstack.org/294506

Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/puppet-keystone 9.1.0

This issue was fixed in the openstack/puppet-keystone 9.1.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to puppet-keystone (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/371776

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to puppet-keystone (stable/mitaka)

Reviewed: https://review.openstack.org/371776
Committed: https://git.openstack.org/cgit/openstack/puppet-keystone/commit/?id=6d76a2650c5552ea8c792fc59c3fb3444d9afd6c
Submitter: Jenkins
Branch: stable/mitaka

commit 6d76a2650c5552ea8c792fc59c3fb3444d9afd6c
Author: Sofer Athlan-Guyot <email address hidden>
Date: Mon Jun 20 13:14:58 2016 +0200

    Fix endpoint update when one endpoint is missing.

    Endpoint are created for admin, internal and public network by this
    provider. If only one of the endpoint is missing then all the endpoints
    are recreated as puppet fails to match the resource with the remaining
    endpoint.

    This fix enable one to update the resource where update means "recreate
    the missing endpoint".

    Change-Id: Ic605725d1923680c6518ebadda36cb5d596c08fe
    Closes-bug: 1559013
    (cherry picked from commit fe0edef97dc0b62828bb0420550dd082b99510a6)

tags: added: in-stable-mitaka
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.