| 2015-11-12 05:27:39 |
Miguel Grinberg |
bug |
|
|
added bug |
| 2015-11-12 05:28:20 |
Miguel Grinberg |
summary |
Heat CFN signals do not pass authentication |
Heat CFN signals do not pass authorization |
|
| 2015-11-12 05:29:23 |
Miguel Grinberg |
description |
Note that this bug applies to the Kilo release. Master does not appear to have this problem. I did not test liberty yet.
Heat templates that rely on CFN signals timeout because the API calls that execute these signals return 403 errors. Heat signals, on the other side, do work.
The problem was reported to me by Alex Cantu. I have verified it on his multinode lab and have also reproduced on my own single-node system hosted on a public cloud server. I suspect liberty/master avoided the problem after Jesse and I reworked the Heat configuration to use Keystone v3 the last day before the L release.
Example template, which can be executed in an AIO after running the tempest playbook:
heat_template_version: 2013-05-23
resources:
wait_condition:
type: AWS::CloudFormation::WaitCondition
properties:
Handle: { get_resource: wait_handle }
Count: 1
Timeout: 600
wait_handle:
type: AWS::CloudFormation::WaitConditionHandle
my_instance:
type: OS::Nova::Server
properties:
image: cirros
flavor: m1.tiny
networks:
- network: "private"
user_data_format: RAW
user_data:
str_replace:
template: |
#!/bin/sh
echo "wc_notify"
wc_notify --data-binary '{"status": "SUCCESS"}'
params:
wc_notify: { get_resource: wait_handle }
This template should end very quickly, as it starts a cirros instance that just sends a signal back to heat. But instead, it timeouts. The user data script dumps the signal URL to the console log, if you then try to send the signal manually you will get a 403. The original 403 can also be seen in the heat-api-cfn.log file.
For reference, the curl command to trigger the signal is: curl -H "Content-Type:" -X PUT "<cfn-signal-url". |
Note that this bug applies to the Kilo release. Master does not appear to have this problem. I did not test liberty yet.
Heat templates that rely on CFN signals timeout because the API calls that execute these signals return 403 errors. Heat signals, on the other side, do work.
The problem was reported to me by Alex Cantu. I have verified it on his multinode lab and have also reproduced on my own single-node system hosted on a public cloud server. I suspect liberty/master avoided the problem after Jesse and I reworked the Heat configuration to use Keystone v3 the last day before the L release.
Example template, which can be executed in an AIO after running the tempest playbook:
heat_template_version: 2013-05-23
resources:
wait_condition:
type: AWS::CloudFormation::WaitCondition
properties:
Handle: { get_resource: wait_handle }
Count: 1
Timeout: 600
wait_handle:
type: AWS::CloudFormation::WaitConditionHandle
my_instance:
type: OS::Nova::Server
properties:
image: cirros
flavor: m1.tiny
networks:
- network: "private"
user_data_format: RAW
user_data:
str_replace:
template: |
#!/bin/sh
echo "wc_notify"
wc_notify --data-binary '{"status": "SUCCESS"}'
params:
wc_notify: { get_resource: wait_handle }
This template should end very quickly, as it starts a cirros instance that just sends a signal back to heat. But instead, it timeouts. The user data script dumps the signal URL to the console log, if you then try to send the signal manually you will get a 403. The original 403 can also be seen in the heat-api-cfn.log file. Here is the log snippet:
2015-11-12 05:13:34.491 1862 INFO heat.api.aws.ec2token [-] Checking AWS credentials..
2015-11-12 05:13:34.492 1862 INFO heat.api.aws.ec2token [-] AWS credentials found, checking against keystone.
2015-11-12 05:13:34.493 1862 INFO heat.api.aws.ec2token [-] Authenticating with http://172.29.236.100:5000/v3/ec2tokens
2015-11-12 05:13:34.533 1862 INFO heat.api.aws.ec2token [-] AWS authentication failure.
2015-11-12 05:13:34.534 1862 INFO eventlet.wsgi.server [-] 10.0.3.181,172.29.236.100 - - [12/Nov/2015 05:13:34] "PUT /v1/waitcondition/arn%3Aopenstack%3Aheat%3A%3A683acadf4d04489f8e991b44014e6fc1%3Astacks%2Fwc1%2Faa4083b6-ce6c-411f-9df9-d059abacf40c%2Fresources%2Fwait_handle?Timestamp=2015-11-12T05%3A12%3A27Z&SignatureMethod=HmacSHA256&AWSAccessKeyId=65657d1021e24e49ba4fb6f217ca4a22&SignatureVersion=2&Signature=aCG%2FO04MNLzSlf5gIBGw1hMcC7bQzB3pZXVKzXLLNSo%3D HTTP/1.1" 403 301 0.043961
For reference, the curl command to trigger the signal is: curl -H "Content-Type:" -X PUT "<cfn-signal-url". |
|
| 2015-11-12 16:50:15 |
Miguel Grinberg |
description |
Note that this bug applies to the Kilo release. Master does not appear to have this problem. I did not test liberty yet.
Heat templates that rely on CFN signals timeout because the API calls that execute these signals return 403 errors. Heat signals, on the other side, do work.
The problem was reported to me by Alex Cantu. I have verified it on his multinode lab and have also reproduced on my own single-node system hosted on a public cloud server. I suspect liberty/master avoided the problem after Jesse and I reworked the Heat configuration to use Keystone v3 the last day before the L release.
Example template, which can be executed in an AIO after running the tempest playbook:
heat_template_version: 2013-05-23
resources:
wait_condition:
type: AWS::CloudFormation::WaitCondition
properties:
Handle: { get_resource: wait_handle }
Count: 1
Timeout: 600
wait_handle:
type: AWS::CloudFormation::WaitConditionHandle
my_instance:
type: OS::Nova::Server
properties:
image: cirros
flavor: m1.tiny
networks:
- network: "private"
user_data_format: RAW
user_data:
str_replace:
template: |
#!/bin/sh
echo "wc_notify"
wc_notify --data-binary '{"status": "SUCCESS"}'
params:
wc_notify: { get_resource: wait_handle }
This template should end very quickly, as it starts a cirros instance that just sends a signal back to heat. But instead, it timeouts. The user data script dumps the signal URL to the console log, if you then try to send the signal manually you will get a 403. The original 403 can also be seen in the heat-api-cfn.log file. Here is the log snippet:
2015-11-12 05:13:34.491 1862 INFO heat.api.aws.ec2token [-] Checking AWS credentials..
2015-11-12 05:13:34.492 1862 INFO heat.api.aws.ec2token [-] AWS credentials found, checking against keystone.
2015-11-12 05:13:34.493 1862 INFO heat.api.aws.ec2token [-] Authenticating with http://172.29.236.100:5000/v3/ec2tokens
2015-11-12 05:13:34.533 1862 INFO heat.api.aws.ec2token [-] AWS authentication failure.
2015-11-12 05:13:34.534 1862 INFO eventlet.wsgi.server [-] 10.0.3.181,172.29.236.100 - - [12/Nov/2015 05:13:34] "PUT /v1/waitcondition/arn%3Aopenstack%3Aheat%3A%3A683acadf4d04489f8e991b44014e6fc1%3Astacks%2Fwc1%2Faa4083b6-ce6c-411f-9df9-d059abacf40c%2Fresources%2Fwait_handle?Timestamp=2015-11-12T05%3A12%3A27Z&SignatureMethod=HmacSHA256&AWSAccessKeyId=65657d1021e24e49ba4fb6f217ca4a22&SignatureVersion=2&Signature=aCG%2FO04MNLzSlf5gIBGw1hMcC7bQzB3pZXVKzXLLNSo%3D HTTP/1.1" 403 301 0.043961
For reference, the curl command to trigger the signal is: curl -H "Content-Type:" -X PUT "<cfn-signal-url". |
Note that this bug applies to the Kilo release. Master does not appear to have this problem. I did not test liberty yet.
Heat templates that rely on CFN signals timeout because the API calls that execute these signals return 403 errors. Heat signals, on the other side, do work.
The problem was reported to me by Alex Cantu. I have verified it on his multinode lab and have also reproduced on my own single-node system hosted on a public cloud server. I suspect liberty/master avoided the problem after Jesse and I reworked the Heat configuration to use Keystone v3 the last day before the L release.
Example template, which can be executed in an AIO after running the tempest playbook:
heat_template_version: 2013-05-23
resources:
wait_condition:
type: AWS::CloudFormation::WaitCondition
properties:
Handle: { get_resource: wait_handle }
Count: 1
Timeout: 600
wait_handle:
type: AWS::CloudFormation::WaitConditionHandle
my_instance:
type: OS::Nova::Server
properties:
image: cirros
flavor: m1.tiny
networks:
- network: "private"
user_data_format: RAW
user_data:
str_replace:
template: |
#!/bin/sh
echo "wc_notify"
curl -H "Content-Type:" -X PUT wc_notify --data-binary '{"status": "SUCCESS"}'
params:
wc_notify: { get_resource: wait_handle }
This template should end very quickly, as it starts a cirros instance that just sends a signal back to heat. But instead, it timeouts. The user data script dumps the signal URL to the console log, if you then try to send the signal manually you will get a 403. The original 403 can also be seen in the heat-api-cfn.log file. Here is the log snippet:
2015-11-12 05:13:34.491 1862 INFO heat.api.aws.ec2token [-] Checking AWS credentials..
2015-11-12 05:13:34.492 1862 INFO heat.api.aws.ec2token [-] AWS credentials found, checking against keystone.
2015-11-12 05:13:34.493 1862 INFO heat.api.aws.ec2token [-] Authenticating with http://172.29.236.100:5000/v3/ec2tokens
2015-11-12 05:13:34.533 1862 INFO heat.api.aws.ec2token [-] AWS authentication failure.
2015-11-12 05:13:34.534 1862 INFO eventlet.wsgi.server [-] 10.0.3.181,172.29.236.100 - - [12/Nov/2015 05:13:34] "PUT /v1/waitcondition/arn%3Aopenstack%3Aheat%3A%3A683acadf4d04489f8e991b44014e6fc1%3Astacks%2Fwc1%2Faa4083b6-ce6c-411f-9df9-d059abacf40c%2Fresources%2Fwait_handle?Timestamp=2015-11-12T05%3A12%3A27Z&SignatureMethod=HmacSHA256&AWSAccessKeyId=65657d1021e24e49ba4fb6f217ca4a22&SignatureVersion=2&Signature=aCG%2FO04MNLzSlf5gIBGw1hMcC7bQzB3pZXVKzXLLNSo%3D HTTP/1.1" 403 301 0.043961
For reference, the curl command to trigger the signal is: curl -H "Content-Type:" -X PUT "<cfn-signal-url". |
|
| 2015-11-12 19:36:44 |
Dolph Mathews |
bug task added |
|
keystone |
|
| 2015-11-12 19:36:58 |
Dolph Mathews |
nominated for series |
|
keystone/kilo |
|
| 2015-11-12 19:36:58 |
Dolph Mathews |
bug task added |
|
keystone/kilo |
|
| 2015-11-12 19:37:05 |
Dolph Mathews |
keystone: status |
New |
Invalid |
|
| 2015-11-12 19:37:08 |
Dolph Mathews |
keystone/kilo: status |
New |
Incomplete |
|
| 2015-11-17 16:33:05 |
Kevin Carter |
openstack-ansible: status |
New |
Triaged |
|
| 2015-11-17 16:33:16 |
Kevin Carter |
openstack-ansible: importance |
Undecided |
High |
|
| 2015-11-17 16:33:23 |
Kevin Carter |
openstack-ansible: milestone |
|
11.2.5 |
|
| 2015-11-17 16:33:44 |
Kevin Carter |
openstack-ansible: importance |
High |
Medium |
|
| 2015-11-20 14:12:06 |
Jesse Pretorius |
nominated for series |
|
openstack-ansible/trunk |
|
| 2015-11-20 14:12:06 |
Jesse Pretorius |
bug task added |
|
openstack-ansible/trunk |
|
| 2015-11-20 14:12:06 |
Jesse Pretorius |
nominated for series |
|
openstack-ansible/kilo |
|
| 2015-11-20 14:12:06 |
Jesse Pretorius |
bug task added |
|
openstack-ansible/kilo |
|
| 2015-11-20 14:12:06 |
Jesse Pretorius |
nominated for series |
|
openstack-ansible/liberty |
|
| 2015-11-20 14:12:06 |
Jesse Pretorius |
bug task added |
|
openstack-ansible/liberty |
|
| 2015-11-20 14:12:18 |
Jesse Pretorius |
openstack-ansible/trunk: status |
Triaged |
Invalid |
|
| 2015-11-20 14:12:20 |
Jesse Pretorius |
openstack-ansible/liberty: status |
New |
Invalid |
|
| 2015-11-20 14:12:23 |
Jesse Pretorius |
openstack-ansible/kilo: status |
New |
In Progress |
|
| 2015-11-20 14:12:28 |
Jesse Pretorius |
openstack-ansible/kilo: status |
In Progress |
Triaged |
|
| 2015-11-20 14:12:31 |
Jesse Pretorius |
openstack-ansible/kilo: milestone |
|
11.2.5 |
|
| 2015-11-20 14:12:34 |
Jesse Pretorius |
openstack-ansible/kilo: assignee |
|
Jesse Pretorius (jesse-pretorius) |
|
| 2015-11-20 14:12:36 |
Jesse Pretorius |
openstack-ansible/trunk: milestone |
11.2.5 |
|
|
| 2015-11-20 14:12:44 |
Jesse Pretorius |
openstack-ansible/kilo: milestone |
11.2.5 |
11.2.6 |
|
| 2015-11-24 00:15:24 |
OpenStack Infra |
keystone/kilo: status |
Incomplete |
Fix Committed |
|
| 2015-12-01 21:22:40 |
Steve Martinelli |
keystone/kilo: importance |
Undecided |
Medium |
|
| 2015-12-02 09:31:24 |
Jesse Pretorius |
openstack-ansible/kilo: status |
Triaged |
Fix Committed |
|
| 2015-12-02 09:31:29 |
Jesse Pretorius |
openstack-ansible/kilo: importance |
Undecided |
Medium |
|
| 2015-12-03 17:59:47 |
Jesse Pretorius |
openstack-ansible/kilo: status |
Fix Committed |
Fix Released |
|
| 2016-01-21 20:26:16 |
Dave Walker |
keystone/kilo: milestone |
|
2015.1.3 |
|
| 2016-01-21 23:16:31 |
Dave Walker |
keystone/kilo: status |
Fix Committed |
Fix Released |
|
