[network, common]: usage of common/libraries/cli functions presuppose keystone client being installed

[Stephan Renatus]

When using the cookbooks to just deploy parts of the infrastructure in a node, it can happen that you only want to use parts of openstack-network. When doing so, however, the ruby_block in openstack-network::common,

https://github.com/stackforge/cookbook-openstack-network/blob/master/recipes/common.rb#L159-L179

fails because identity_uuid, from openstack-common/libraries/cli.rb,

https://github.com/stackforge/cookbook-openstack-common/blob/master/libraries/cli.rb#L83

will try to shell out `keystone tenant-list` without having installed any python-keystoneclient packages.

I suppose including openstack-identity::client from openstack-network::common is a way to fix that. (Patch follows.)

Traceback for the curious:

Errno::ENOENT
-------------
No such file or directory - keystone

Cookbook Trace:
---------------
/var/chef/cache/cookbooks/openstack-common/libraries/cli.rb:61:in `openstack_command'
/var/chef/cache/cookbooks/openstack-common/libraries/cli.rb:81:in `identity_uuid'
/var/chef/cache/cookbooks/openstack-network/recipes/common.rb:168:in `block (2 levels) in from_file'

Resource Declaration:
---------------------
# In /var/chef/cache/cookbooks/openstack-network/recipes/common.rb

161: ruby_block 'query service tenant uuid' do
162: # query keystone for the service tenant uuid
163: block do
164: begin
165: admin_user = node['openstack']['identity']['admin_user']
166: admin_tenant = node['openstack']['identity']['admin_tenant_name']
167: env = openstack_command_env admin_user, admin_tenant
168: tenant_id = identity_uuid 'tenant', 'name', 'service', env
169: Chef::Log.error('service tenant UUID for nova_admin_tenant_id not found.') if tenant_id.nil?
170: node.set['openstack']['network']['nova']['admin_tenant_id'] = tenant_id
171: rescue RuntimeError => e
172: Chef::Log.error("Could not query service tenant UUID for nova_admin_tenant_id. Error was #{e.message}")
173: end
174: end
175: action :run

Compiled Resource:
------------------
# Declared in /var/chef/cache/cookbooks/openstack-network/recipes/common.rb:161:in `from_file'

ruby_block("query service tenant uuid") do
  action [:run]
  retries 0
  retry_delay 2
  guard_interpreter :default
  block_name "query service tenant uuid"
  cookbook_name "openstack-network"
  recipe_name "common"
  block #<Proc:0x00000006b70358@/var/chef/cache/cookbooks/openstack-network/recipes/common.rb:163>
  only_if { #code block }
end

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/100252

[Stephan Renatus (s-renatus)]

In the change's review, Mark Vanderwiel commented:

> Do you think there's a way to add this into the library call support in Common to allow any needed client to be temporarily installed to allow a cleaner footprint after the converge?

> It would be nice to allow folks to have control over which nodes have client packages installed. Doing it this way, we take away a bit of that control. But I don't know of a good way to have "temp" resources, not sure if Chef has that capability.

I'd rather see it in the libraries/cli.rb of openstack-common, too. However, that library seems like it's supposed to be as neutral wrt. the services it talks to as possible, only providing a generic `openstack_command`,

https://github.com/stackforge/cookbook-openstack-common/blob/master/libraries/cli.rb#L55

that wraps whatever you want to get, be it tenants,

https://github.com/stackforge/cookbook-openstack-common/blob/master/libraries/cli.rb#L85

or images,

https://github.com/stackforge/cookbook-openstack-common/blob/master/libraries/cli.rb#L102

Hence to have that library take care of things (or at least warn with a descriptive error message pointing to `openstack-#{service}::client`), we'd have to add a mapping of common names to service names...

Besides a warning, I doubt that the libraries/cli.rb code could do much (e.g. include_recipe), judging from this https://github.com/opscode-cookbooks/chef-vault/blob/master/libraries/chef_vault_item.rb and the missing replies to this http://lists.opscode.com/sympa/arc/chef/2014-02/msg00089.html ...

[Mark Vanderwiel (vanderwl)]

Great details. My thought was to maybe go after a enhancement in Chef to allow for resources that are only available during the converge (created up front and removed at the end). Kinda like the delay and immediate notify settings. Some obvious resources that would be useful in that mode would be: templates, files, directories or packages for setup only work (like the openstack clients). One issue will be overlap with something like the client recipe which wants to laydown the client packages.
Not sure if this is possible but would like to explore it a bit.

I think your current patch is fine and addresses the issue, but maybe a resource/provider CLI wrapper might also be something to explore. It would include that package resource and then make the library call.

[Stephan Renatus (s-renatus)]

The problem with a resource/provider CLI wrapper would be passing data back, I suppose. Looking at the hops openstack_identity_register 'user' { action :create_ec2_credentials} has to go through to pass data "back",

https://github.com/stackforge/cookbook-openstack-identity/blob/master/providers/register.rb#L308-L312

I wonder if this would be a nicer approach.

Would you really want to go with "install python-keystoneclient && retrieve tenant_uuid && remove python-keystoneclient"? use_inline_resources within a LWRP might handle that nicely, though (untested):

package "python-keystoneclient" do
  notify "package[python-keystoneclient]", :delayed
end

# do stuff...

[Stephan Renatus (s-renatus)]

The last bit should be

package "python-keystoneclient" do
  notify :remove, "package[python-keystoneclient]", :delayed
end

[Stephan Renatus (s-renatus)]

The last bit should be

package "python-keystoneclient" do
  notifies :remove, "package[python-keystoneclient]", :delayed
end

(Sorry for the noise)

[Mark Vanderwiel (vanderwl)]

Yeah, that's kinda what I had in mind. I think one obvious issue is that the python-xxxclient could already be installed via some other recipe, so we might need some type of checks to prevent upgrading/removing it if it's already there.

[Mark Vanderwiel (vanderwl)]

Or maybe we should switch to use something like curl and the REST interfaces for this type of work and avoid the whole client package issue?

[Mark Vanderwiel (vanderwl)]

For now, I think the patch for network is fine, maybe we should copy this over to a new blueprint and we can hash out a better solution there.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/100252
Committed: https://git.openstack.org/cgit/stackforge/cookbook-openstack-network/commit/?id=c129d7d41a7c8e6f55bfed515cf407ceedf19615
Submitter: Jenkins
Branch: master

commit c129d7d41a7c8e6f55bfed515cf407ceedf19615
Author: Stephan Renatus <email address hidden>
Date: Mon Jun 16 15:31:02 2014 +0200

    Include the keystone client in the common recipe

    The client is needed in the call to openstack-common's cli library,
    identity_uuid, in the ruby_block.

    Change-Id: I0d5dbb18eb57ea848baf91bfce0eaa23a3c860fa
    Closes-Bug: #1330484