Incorrect SQL statement in inserting to table EVENT

Bug #1224434 reported by Jukka Syrjanen
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Heat
Fix Released
High
Angus Salkeld

Bug Description

This problem occurs with Havana version of Ceilometer/Heat. I ran this with devstack pulled today.

The following patch has been taken into use so that the alarm info can be successfully inserted to ceilometer DB during heat stack create: https://review.openstack.org/#/c/45076/

Heat fills an incorrect parameter to be inserted to database as a result of Ceilometer calling Heat's webhook when an alarm has been triggered. See the attached template file. Adjust properties KeyName and ImageId in the template before running it.

Value of parameter resource_status_reason in log the entry is invalid.

Heat engine's log:
=====================================

2013-09-12 14:20:21.344 ERROR heat.engine.resource [-] DB error
(ProgrammingError) (1064, 'You have an error in your SQL syntax; check the
manual that corresponds to your MySQL server version for the right syntax to
use near \': "\'insufficient data\'", u\'alarm_id\':
"\'212341eb-f99a-4171-b45d-b698c6d301e3\'", \' at line 1') 'INSERT INTO event
(created_at, updated_at, stack_id, resource_action, resource_status,
resource_name, physical_resource_id, resource_status_reason, resource_type,
resource_properties) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s)'
(datetime.datetime(2013, 9, 12, 11, 20, 21, 256164), None,
'408c1ad7-3d0b-4504-a52a-18f683e3f509', 'signal', 'COMPLETE',
'VmRestartPolicy', 'e106677a51714e2395952026cbf974b3', {u'current':
u'insufficient data', u'alarm_id': u'212341eb-f99a-4171-b45d-b698c6d301e3',
u'reason': [u'1', u' ', u'd', u'a', u't', u'a', u'p', u'o', u'i', u'n', u't',
u's', u' ', u'a', u'r', u'e', u' ', u'u', u'n', u'k', u'n', u'o', u'w', u'n'],
u'previous': u'alarm'}, 'OS::Heat::HARestarter',
'\x80\x02}q\x01U\nInstanceIdq\x02X$\x00\x00\x0081f9fc73-6a64-47f8-99e9-017e074ce661q\x03s.')

2013-09-12 14:20:21.344 INFO heat.engine.resources.instance [-] VmRestartPolicy Alarm, new state alarm
2013-09-12 14:20:21.344 INFO heat.engine.resources.instance [-] VmRestartPolicy Alarm, restarting resource: MyInstance
2013-09-12 14:20:21.345 DEBUG heat.engine.scheduler [-] Task destroy from CeilometerAlarm "VmDownAlarm" starting from (pid=15250) start /opt/stack/heat/heat/engine/scheduler.py:131
2013-09-12 14:20:21.345 DEBUG heat.engine.scheduler [-] Task destroy from CeilometerAlarm "VmDownAlarm" running from (pid=15250) step /opt/stack/heat/heat/engine/scheduler.py:164
2013-09-12 14:20:21.345 INFO heat.engine.resource [-] deleting CeilometerAlarm "VmDownAlarm"
2013-09-12 14:20:21.462 INFO requests.packages.urllib3.connectionpool [-] Starting new HTTP connection (1): 192.168.56.110
2013-09-12 14:20:21.806 DEBUG requests.packages.urllib3.connectionpool [-] "POST /v2.0/tokens HTTP/1.1" 200 10815 from (pid=15250) _make_request /usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/connectionpool.py:296
2013-09-12 14:20:21.808 INFO requests.packages.urllib3.connectionpool [-] Starting new HTTP connection (1): 192.168.56.110

=================================

Steps to reproduce the problem:

The steps to reproduce the problem:
1) add the following to /etc/ceilometer/pipeline.yaml:
        - name: "accumulator"
          parameters:
              target:
                  name: "vm_down"
                  unit: "count"
                  type: "count"

2) restart ceilometer-acompute

3) Create the stack: heat stack-create ha -f <templateFileName>

4) when the stack is up, locate the alarm id of metric "vm_down" with
ceilometer alarm-list

5) Change the alarm status:
ceilometer alarm-update -a <alarmID> --state alarm

6) The log entry above appears in heat engine's log within a minute

PS. Note that there's another problem in Ceilometer due to which you are forced to supply insufficient_data_actions too in the template file. You cannot reproduce the problem if you remove it, because in that case Ceilometer fails to call Heat's webhook entirely.

Revision history for this message
Jukka Syrjanen (jukka-syrjanen) wrote :
Changed in heat:
milestone: none → havana-rc2
assignee: nobody → Angus Salkeld (asalkeld)
importance: Undecided → High
status: New → Triaged
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to heat (master)

Fix proposed to branch: master
Review: https://review.openstack.org/50511

Changed in heat:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to heat (master)

Reviewed: https://review.openstack.org/50511
Committed: http://github.com/openstack/heat/commit/4202a2dbe5b949bdd7615b2f778300c0658ad8ee
Submitter: Jenkins
Branch: master

commit 4202a2dbe5b949bdd7615b2f778300c0658ad8ee
Author: Angus Salkeld <email address hidden>
Date: Wed Oct 9 11:02:41 2013 +1100

    Fix the signal details string

    This could come from multiple places so try make it useful in the
    common case (Ceilometer and watchrule) and safe in the other cases.

    Change-Id: Iee88968b52ec15addf4721e004d2994b2f8c2134
    Closes-bug: #1224434

Changed in heat:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to heat (milestone-proposed)

Fix proposed to branch: milestone-proposed
Review: https://review.openstack.org/50886

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to heat (milestone-proposed)

Reviewed: https://review.openstack.org/50886
Committed: http://github.com/openstack/heat/commit/29211b99f318d4555e02cabaab89fd410df31419
Submitter: Jenkins
Branch: milestone-proposed

commit 29211b99f318d4555e02cabaab89fd410df31419
Author: Angus Salkeld <email address hidden>
Date: Wed Oct 9 11:02:41 2013 +1100

    Fix the signal details string

    This could come from multiple places so try make it useful in the
    common case (Ceilometer and watchrule) and safe in the other cases.

    Change-Id: Iee88968b52ec15addf4721e004d2994b2f8c2134
    Closes-bug: #1224434

Changed in heat:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in heat:
milestone: havana-rc2 → 2013.2
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.