EC2 v4 signature calculation is wrong, in case of request is POST

Bug #1360892 reported by Yukinori Sagara on 2014-08-24
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
python-keystoneclient
High
Yukinori Sagara

Bug Description

In Heat, instance run cfn-push-stats, which uses boto library, and boto creates EC2 v4 signature.
heat-api-cfn service receives that request, and delegates signature check to Keystone.
Keystone recreates EC2 v4 signature with keystoneclient for checking.
But keystoneclient has its own EC2 v4 signature implementation, and so there is a logic mismatch between boto and keystoneclient.

The following is what has been previously transmitted to the openstack-dev mailing list.

> I am trying Heat instance HA, using RDO Icehouse.
>
> After instance boot, instance push own stats to heat alarm with cfn-push-stats command.
> But cfn-push-stats always failed with error '403 SignatureDoesNotMatch', this message is
> output to /var/log/cfn-push-stats.log.
>
> I debugged client and server side code. (i.e. cfn-push-stats, boto, heat, keystone,
> keystoneclient) And I found curious code mismatch between boto and keystoneclient about
> signature calculation.
>
> Here is a result of debugging, and code examination.
>
> * Client side
>
> cfn-push-stats uses heat-cfntools library, and heat-cfntools do 'POST' request with boto.
> boto perfomes signature calculation. [1]
> for signature calculation, firstly it construct 'CanonicalRequest', some strings are joined.
> And create a digest hash of the CanonicalRequest for signature calculation.
> CanonicalRequest contains CanonicalQueryString, which is transfomed URL query strings.
>
> CanonicalRequest =
> HTTPRequestMethod + '\n' +
> CanonicalURI + '\n' +
> CanonicalQueryString + '\n' +
> CanonicalHeaders + '\n' +
> SignedHeaders + '\n' +
> HexEncode(Hash(RequestPayload))
>
> **AWS original tool (aws-cfn-bootstrap-1.4) and boto uses empty string as
> CanonicalQueryString, when request is POST.**
>
> ----
>
> AWS original tool's code is following.
>
> cfnbootstrap/aws_client.py
>
> 110 class V4Signer(Signer):
>
> 144 (canonical_headers, signed_headers) = self._canonicalize_headers(new_headers)
> 145 canonical_request += canonical_headers + '\n' + signed_headers + '\n'
> 146 canonical_request += hashlib.sha256(self._construct_query(params).encode('utf-8') if verb == 'POST' else '').hexdigest()
>
> ----
>
> boto's code is below.
>
> boto/auth.py
>
> 283 class HmacAuthV4Handler(AuthHandler, HmacKeys):
>
> 393 def canonical_request(self, http_request):
> 394 cr = [http_request.method.upper()]
> 395 cr.append(self.canonical_uri(http_request))
> 396 cr.append(self.canonical_query_string(http_request))
> 397 headers_to_sign = self.headers_to_sign(http_request)
> 398 cr.append(self.canonical_headers(headers_to_sign) + '\n')
> 399 cr.append(self.signed_headers(headers_to_sign))
> 400 cr.append(self.payload(http_request))
> 401 return '\n'.join(cr)
>
> 337 def canonical_query_string(self, http_request):
> 338 # POST requests pass parameters in through the
> 339 # http_request.body field.
> 340 if http_request.method == 'POST':
> 341 return ""
> 342 l = []
> 343 for param in sorted(http_request.params):
> 344 value = boto.utils.get_utf8_value(http_request.params[param])
> 345 l.append('%s=%s' % (urllib.quote(param, safe='-_.~'),
> 346 urllib.quote(value, safe='-_.~')))
> 347 return '&'.join(l)
>
> ----
>
> * Server side
>
> heat-api-cfn queries to keystone in order to check request authorization,
> and keystone uses keystoneclient to check EC2 format request signature.
>
> **keystoneclient uses (non-empty) query string as CanonicalQueryString, even
> though request is POST.**
> And create a digest hash of the CanonicalRequest for signature calculation.
>
> ----
>
> keystoneclient's code is below.
>
> keystoneclient/contrib/ec2/utils.py
>
> 28 class Ec2Signer(object):
>
> 154 def _calc_signature_4(self, params, verb, server_string, path, headers,
> 155 body_hash):
> 156 """Generate AWS signature version 4 string."""
>
> 235 # Create canonical request:
> 236 # http://docs.aws.amazon.com/general/latest/gr/
> 237 # sigv4-create-canonical-request.html
> 238 # Get parameters and headers in expected string format
> 239 cr = "\n".join((verb.upper(), path,
> 240 self._canonical_qs(params),
> 241 canonical_header_str(),
> 242 auth_param('SignedHeaders'),
> 243 body_hash))
>
> 125 @staticmethod
> 126 def _canonical_qs(params):
> 127 """Construct a sorted, correctly encoded query string as required for
> 128 _calc_signature_2 and _calc_signature_4.
> 129 """
> 130 keys = list(params)
> 131 keys.sort()
> 132 pairs = []
> 133 for key in keys:
> 134 val = Ec2Signer._get_utf8_value(params[key])
> 135 val = urllib.parse.quote(val, safe='-_~')
> 136 pairs.append(urllib.parse.quote(key, safe='') + '=' + val)
> 137 qs = '&'.join(pairs)
> 138 return qs
>
> ----
>
> So it should be different from boto(client side) to keystoneclient(server side).
>
> I wrote a patch to resolve this how to treat CanonicalQueryString mismatch,
> My patch honored AWS original tool and boto, so if request is POST,
> if request is POST, 'CanonicalQueryString' is regarded as a empty string.
>
> With my patch, Heat instance HA works fine.
>
> This bug affects Heat and Keystone, but patch is only needed in python-keystoneclient.
> So I will report to python-keystoneclient launchpad and submit a patch to Gerrit.
> Please confirm it.
>
> ----
>
> My environment is RDO Icehouse/CentOS6.5, and package versions is following.
>
> * Client side
>
> cloud-init-0.7.4-2.el6.noarch
> heat-cfntools-1.2.6-2.el6.noarch
> python-boto-2.27.0-1.el6.noarch
>
> * Server side
>
> python-keystoneclient-0.9.0-1.el6.noarch
> python-keystone-2014.1.1-1.el6.noarch
> openstack-keystone-2014.1.1-1.el6.noarch
>
> ----
>
> References
>
> [1] http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html

Yukinori Sagara (sagara) on 2014-08-24
description: updated

Fix proposed to branch: master
Review: https://review.openstack.org/116523

Changed in python-keystoneclient:
assignee: nobody → Yukinori Sagara (sagara177)
status: New → In Progress
Dolph Mathews (dolph) on 2014-08-28
Changed in python-keystoneclient:
importance: Undecided → High
milestone: none → 0.11.0

Reviewed: https://review.openstack.org/116523
Committed: https://git.openstack.org/cgit/openstack/python-keystoneclient/commit/?id=cf5e45dd5b1ae9b98698a05e7d39989b6bfd4747
Submitter: Jenkins
Branch: master

commit cf5e45dd5b1ae9b98698a05e7d39989b6bfd4747
Author: Yukinori Sagara <email address hidden>
Date: Mon Aug 25 10:53:30 2014 +0900

    fix EC2 Signature Version 4 calculation, in the case of POST

    When calculating the AWS Signature Version 4, in the case of POST,
    We need to set the CanonicalQueryString to an empty string. this
    follows the implementation of the AWS and boto clients.

    Change-Id: Iad4e392119067e246c7b77009da3fef48d251382
    Closes-Bug: 1360892

Changed in python-keystoneclient:
status: In Progress → Fix Committed
Dolph Mathews (dolph) on 2014-09-21
Changed in python-keystoneclient:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers