event/meter-time-to-live values written to config file incorrect, crashing api

Bug #1651645 reported by Benjamin Kaehne
24
This bug affects 4 people
Affects Status Importance Assigned to Milestone
Charm Helpers
New
Undecided
Unassigned
OpenStack Ceilometer Charm
Fix Released
High
Billy Olsen
ceilometer (Juju Charms Collection)
Invalid
Undecided
Unassigned

Bug Description

When you input:
juju set ceilometer event-time-to-live=7776000
juju set ceilometer metering-time-to-live=7776000

On the latest charm checkout you receive in the api logs:
2016-12-21 05:19:18.737 382839 ERROR ceilometer.api.hooks [-] Failed to connect to db, purpose event retry later: Value for option event_time_to
_live is not valid: invalid literal for int() with base 10: '7776000.0'

summary: - large even/meter-time0to-live get converted to int, crashing api
+ event/meter-time-to-live values written to config file incorrect,
+ crashing api
James Page (james-page)
Changed in ceilometer (Juju Charms Collection):
status: New → Invalid
Xav Paice (xavpaice)
Changed in charm-ceilometer:
assignee: nobody → Xav Paice (xavpaice)
status: New → Confirmed
Revision history for this message
Xav Paice (xavpaice) wrote :

Tested with 17.02 - fixed. Closing this one :)

Changed in charm-ceilometer:
status: Confirmed → Fix Released
assignee: Xav Paice (xavpaice) → nobody
Revision history for this message
Billy Olsen (billy-olsen) wrote :

I can confirm that this is indeed a problem and it appears that it is due to the numeric representation of large numbers. Examining the value returned by juju's config-get construct from the charm perspective, we see the following values set:

> juju run --unit ceilometer/0 'config-get'
action-managed-upgrade: false
api-workers: 1
debug: false
dns-ha: false
event-time-to-live: 7.776e+06
ha-bindiface: eth0
ha-mcastport: 5403
metering-time-to-live: 7.776e+06
nagios_context: juju
nagios_servicegroups: ""
openstack-origin: distro
rabbit-user: ceilometer
rabbit-vhost: openstack
region: RegionOne
use-syslog: false
verbose: false

Note, the values for the time-to-live values are stored in scientific notation, 7.776e+06. When this is converted in python, its treated as a floating point value (due to these numbers being a float multiplied by 10^x):

>>> test = 7.776e+06
>>> test
7776000.0

This value is rendered directly into the charm itself. The appropriate place to fix this would be the charm-helpers library.

Changed in charm-ceilometer:
status: Fix Released → Triaged
importance: Undecided → High
assignee: nobody → Billy Olsen (billy-olsen)
milestone: none → 17.05
Revision history for this message
Billy Olsen (billy-olsen) wrote :

I think fixing the charm-helpers library should be possibly be done. This can be accomplished by loading the charm's config schema (in config.yaml) and ensure that the proper type is returned. That's a bit more involved than what is needed to address the issue in the ceilometer charm in which a simple cast to an int will suffice.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to charm-ceilometer (master)

Fix proposed to branch: master
Review: https://review.openstack.org/456792

Changed in charm-ceilometer:
status: Triaged → In Progress
Revision history for this message
Billy Olsen (billy-olsen) wrote :

Note: I went with the cheaper fix for now - ensuring that the ttl values were specified as int in the ceilometer charm itself.

James Page (james-page)
Changed in charm-ceilometer:
milestone: 17.05 → 17.08
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-ceilometer (master)

Reviewed: https://review.openstack.org/456792
Committed: https://git.openstack.org/cgit/openstack/charm-ceilometer/commit/?id=09bb38e488026f2755d12a3e471364bd490df240
Submitter: Jenkins
Branch: master

commit 09bb38e488026f2755d12a3e471364bd490df240
Author: Billy Olsen <email address hidden>
Date: Thu Apr 13 17:37:21 2017 -0700

    Cast time-to-live values to int

    For large integer values, Juju can choose to return the number
    in scientific notation. This causes the python code loading the
    config data to return a floating point number instead of an
    integer. Floating point values in ceilometer's configuration file
    will prevent ceilometer from starting.

    This change forces the type to an integer, allowing users to
    specify large ttl values.

    Change-Id: I8e29e6868b4c4a4ae20ad87ab585508fd321261c
    Closes-Bug: #1651645

Changed in charm-ceilometer:
status: In Progress → Fix Committed
tags: added: stable-backport sts
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to charm-ceilometer (stable/17.02)

Fix proposed to branch: stable/17.02
Review: https://review.openstack.org/464728

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-ceilometer (stable/17.02)

Reviewed: https://review.openstack.org/464728
Committed: https://git.openstack.org/cgit/openstack/charm-ceilometer/commit/?id=75d3f03ee00639b606b6f6f8f8c51f107fd8de4d
Submitter: Jenkins
Branch: stable/17.02

commit 75d3f03ee00639b606b6f6f8f8c51f107fd8de4d
Author: Billy Olsen <email address hidden>
Date: Thu Apr 13 17:37:21 2017 -0700

    Cast time-to-live values to int

    For large integer values, Juju can choose to return the number
    in scientific notation. This causes the python code loading the
    config data to return a floating point number instead of an
    integer. Floating point values in ceilometer's configuration file
    will prevent ceilometer from starting.

    This change forces the type to an integer, allowing users to
    specify large ttl values.

    Note: also contains charmhelpers sync for amulet test updates, and
    an update to the amulet test.

    Change-Id: I8e29e6868b4c4a4ae20ad87ab585508fd321261c
    Closes-Bug: #1651645
    (cherry picked from commit 09bb38e488026f2755d12a3e471364bd490df240)

Changed in charm-ceilometer:
status: Fix Committed → Fix Released
milestone: 17.08 → 17.05
milestone: 17.05 → 17.02
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.