Required credentials are missing when heat-engine calls heatclient operations

[Michael Elder]

While adopting the latest from the software configurations in Icehouse, we discovered an issue with the new software configuration type and its assumptions about using the heat client to perform behavior.

The change was introduced in:

commit 21f60b155e4b65396ebf77e05a0ef300e7c3c1cf
Author: Steve Baker <email address hidden>
Change: https://review.openstack.org/#/c/67621/

The net is that the software config type in software_config.py lines 147-152 relies on the heat client to create/clone software configuration resources in the heat database:

    def handle_create(self):
        props = dict(self.properties)
        props[self.NAME] = self.physical_resource_name()

        sc = self.heat().software_configs.create(**props) ## HERE THE HEAT CLIENT IS CREATING A NEW SOFTWARE_CONFIG TO MAKE EACH ONE IMMUTABLE
        self.resource_id_set(sc.id)

My concerns with this approach:

When used in standalone mode, the Heat engine receives headers which are used to drive authentication (X-Auth-Url, X-Auth-User, X-Auth-Key, ..):

curl -i -X POST -H 'X-Auth-Key: password' -H 'Accept: application/json' -H 'Content-Type: application/json' -H 'X-Auth-Url: http://[host]:5000/v2.0' -H 'X-Auth-User: admin' -H 'User-Agent: python-heatclient' -d '{...}' http://10.0.2.15:8004/v1/{tenant_id}

In this mode, the heat config file indicates standalone mode and can also indicate multicloud support:

# /etc/heat/heat.conf
[paste_deploy]
flavor = standalone

[auth_password]
allowed_auth_uris = http://[host1]:5000/v2.0,http://[host2]:5000/v2.0
multi_cloud = true

Any keystone URL which is referenced is unaware of the orchestration engine which is interacting with it. Herein lies the design flaw.

Further, at this point, the username and password are null, and when the auth_password standza is applied in the config file, Heat will deny any attempts at authorization which only provide a token. As I understand it today, that's because it doesn't have individual keystone admin users for all remote keystone services in the list of allowed_auth_urls. Hence, if only provided with a token, I don't think the heat engine can validate the token against the remote keystone.

One workaround that I've implemented locally is to change the logic to check for standalone mode and send the username and password.

       flavor = 'default'
        try:
            logger.info("Configuration is %s" % str(cfg.CONF))
            flavor = cfg.CONF.paste_deploy.flavor
        except cfg.NoSuchOptError as nsoe:
            flavor = 'default'
        logger.info("Flavor is %s" % flavor)

        # We really should examine the pipeline to determine whether we're using authtoken or authpassword.
        if flavor == 'standalone':

            context_map = self.context.to_dict()

            if 'username' in context_map.keys():
                username = context_map['username']
            else:
                username = None

            if 'password' in context_map.keys():
                password = context_map['password']
            else:
                password = None

            logger.info("Configuring username='%s' and password='%s'" % (username, password))
            args = {
                'auth_url': con.auth_url,
                'token': None,
                'username': username,
                'password': password,
                'ca_file': self._get_client_option('heat', 'ca_file'),
                'cert_file': self._get_client_option('heat', 'cert_file'),
                'key_file': self._get_client_option('heat', 'key_file'),
                'insecure': self._get_client_option('heat', 'insecure')
            }
        else:
            if self.auth_token is None:
                logger.error(_("Heat connection failed, no auth_token!"))
                return None
...

Responses from Steve Baker from the mailing list:

If you look at self._get_heat_url() you'll see that the heat.conf [clients_heat] url will be used for the heat endpoint if it is set. I would recommend setting that for standalone mode. A devstack change for HEAT_STANDALONE would be helpful here.

This is a great summary of the problem, but it really belongs in a launchpad bug. Lets discuss potential solutions there.

[Steve Baker (steve-stevebaker)]

I think the solution here would be for KeystonePasswordAuthProtocol to set a header like HTTP_X_AUTH_METHOD='authpassword', and RequestContext to store this in an auth_method attribute. Then the heat() method can include the credentials if auth_method=='authpassword'

Feel free to assign this bug to yourself if you intend to submit a patch to fix this.

[Steven Hardy (shardy)]

No fix proposed so bumping to J3

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/123291

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/123292

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/123292
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=b61a45c13ffa82356f896f0914d2f28dabea7a7f
Submitter: Jenkins
Branch: master

commit b61a45c13ffa82356f896f0914d2f28dabea7a7f
Author: Steve Baker <email address hidden>
Date: Tue Sep 23 11:59:12 2014 +1200

    Include credentials for heat calling self

    When in standalone mode, heat needs to include heat credentials
    when calling its own REST API.

    This change assumes that heat is running in standalone mode if
    heat.conf [clients_heat] url has been set.

    Change-Id: I34e662c554ea66c6df6ebee6ce3bfb0dc3f8a948
    Closes-Bug: #1302578