Juniper Openstack

Ceilometer:R3.2 : ceilometer sample list not working

Bug #1644490 reported by sundarkh
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R3.1
Invalid
High
Nitish Krishna Kaveri
R3.2
Fix Committed
High
Nitish Krishna Kaveri
Juniper Openstack r3.2.1.0
Trunk
Fix Committed
High
Nitish Krishna Kaveri
Juniper Openstack r4.0.0.0-fcs "r4.0.0.0"

Bug Description

SM:R3.2 : ceilometer sample list not working

1) Install SM R3.2 (since the later builds are having installation issues)
2) Provision the target with R3.2 build 5 , Mitaka contrail package; Provision gets completed succesfully
3) Observe that the ceilometer sample list is empty

ceilometer sample-list
+----+-------------+------+------+--------+------+-----------+
| ID | Resource ID | Name | Type | Volume | Unit | Timestamp |
+----+-------------+------+------+--------+------+-----------+
+----+-------------+------+------+--------+------+-----------+
root@nodeh8:~# ceilometer meter-list
+------+------+------+-------------+---------+------------+
| Name | Type | Unit | Resource ID | User ID | Project ID |
+------+------+------+-------------+---------+------------+
+------+------+------+-------------+---------+------------+

ceilometer-api.log
-------------------

2016-11-23 06:19:39.031 1903 WARNING keystonemiddleware.auth_token [-] Using the in-process token cache is deprecated as of the 4.2.0 release and may be removed in the 5.0.0 release or the 'O' development cycle. The in-process cache causes inconsistent results and high memory usage. When the feature is removed the auth_token middleware will not cache tokens by default which may result in performance issues. It is recommended to use memcache for the auth_token token cache by setting the memcached_servers option.
2016-11-23 06:19:39.648 1903 WARNING ceilometer.api.controllers.v2.root [-] Can't connect to keystone, assuming aodh is disabled and retry later.
2016-11-23 06:19:39.652 1903 DEBUG oslo_policy._cache_handler [-] Reloading cached file /etc/ceilometer/policy.json read_cached_file /usr/lib/python2.7/dist-packages/oslo_policy/_cache_handler.py:38
2016-11-23 06:19:39.687 1903 DEBUG oslo_policy.policy [-] Reloaded policy file: /etc/ceilometer/policy.json _load_policy_file /usr/lib/python2.7/dist-packages/oslo_policy/policy.py:493
2016-11-23 06:19:39.688 1903 INFO ceilometer.api.controllers.v2.utils [-] No limit value provided, result set will be limited to 100.
2016-11-23 06:19:39.782 1903 INFO werkzeug [-] 172.17.0.2 - - [23/Nov/2016 06:19:39] "GET /v2/meters/ip.floating.transmit.packets HTTP/1.1" 200 -
2016-11-24 01:30:43.244 1903 INFO ceilometer.api.controllers.v2.utils [-] No limit value provided, result set will be limited to 100.
2016-11-24 01:30:43.247 1903 INFO werkzeug [-] 172.17.0.2 - - [24/Nov/2016 01:30:43] "GET /v2/meters/ip.floating.transmit.packets HTTP/1.1" 200 -
2016-11-24 01:31:43.898 1903 INFO ceilometer.api.controllers.v2.utils [-] No limit value provided, result set will be limited to 100.
2016-11-24 01:31:43.917 1903 INFO werkzeug [-] 172.17.0.2 - - [24/Nov/2016 01:31:43] "GET /v2/meters HTTP/1.1" 200 -
2016-11-24 01:31:48.991 1903 INFO ceilometer.api.controllers.v2.utils [-] No limit value provided, result set will be limited to 100.
2016-11-24 01:31:48.993 1903 INFO werkzeug [-] 172.17.0.2 - - [24/Nov/2016 01:31:48] "GET /v2/meters HTTP/1.1" 200 -
2016-11-24 01:41:56.405 1903 INFO ceilometer.api.controllers.v2.utils [-] No limit value provided, result set will be limited to 100.
2016-11-24 01:41:56.419 1903 INFO werkzeug [-] 172.17.0.2 - - [24/Nov/2016 01:41:56] "GET /v2/samples HTTP/1.1" 200 -
2016-11-24 01:50:25.665 1903 INFO ceilometer.api.controllers.v2.utils [-] No limit value provided, result set will be limited to 100.
2016-11-24 01:50:25.668 1903 INFO werkzeug [-] 10.204.217.112 - - [24/Nov/2016 01:50:25] "GET /v2/samples HTTP/1.1" 200 -
2016-11-24 01:50:32.056 1903 INFO ceilometer.api.controllers.v2.utils [-] No limit value provided, result set will be limited to 100.
2016-11-24 01:50:32.058 1903 INFO werkzeug [-] 10.204.217.112 - - [24/Nov/2016 01:50:32] "GET /v2/meters HTTP/1.1" 200 -

ceilometer-agent-central.log
-----------------------------

2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client [req-553abf1d-4923-417f-a7fa-b0332988b3cb admin - - - -] Cannot use v2 authentication with domain scope
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client Traceback (most recent call last):
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client File "/usr/lib/python2.7/dist-packages/ceilometer/nova_client.py", line 52, in with_logging
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client return func(*args, **kwargs)
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client File "/usr/lib/python2.7/dist-packages/ceilometer/nova_client.py", line 171, in instance_get_all
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client search_opts=search_opts)
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client File "/usr/lib/python2.7/dist-packages/novaclient/v2/servers.py", line 749, in list
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client "servers")
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client File "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 242, in _list
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client resp, body = self.api.client.get(url)
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client File "/usr/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 173, in get
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client return self.request(url, 'GET', **kwargs)
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 89, in request
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client **kwargs)
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client File "/usr/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 331, in request
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client resp = super(LegacyJsonAdapter, self).request(*args, **kwargs)
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client File "/usr/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 98, in request
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client return self.session.request(url, method, **kwargs)
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client File "/usr/lib/python2.7/dist-packages/positional/__init__.py", line 94, in inner
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client return func(*args, **kwargs)
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line 370, in request
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client auth_headers = self.get_auth_headers(auth)
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line 624, in get_auth_headers
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client return auth.get_headers(self, **kwargs)
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client File "/usr/lib/python2.7/dist-packages/keystoneauth1/plugin.py", line 84, in get_headers
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client token = self.get_token(session)
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client File "/usr/lib/python2.7/dist-packages/keystoneauth1/identity/base.py", line 90, in get_token
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client return self.get_access(session).auth_token
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client File "/usr/lib/python2.7/dist-packages/keystoneauth1/identity/base.py", line 136, in get_access
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client self.auth_ref = self.get_auth_ref(session)
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client File "/usr/lib/python2.7/dist-packages/keystoneauth1/identity/generic/base.py", line 179, in get_auth_ref
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client self._plugin = self._do_create_plugin(session)
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client File "/usr/lib/python2.7/dist-packages/keystoneauth1/identity/generic/base.py", line 168, in _do_create_plugin
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client 'Cannot use v2 authentication with domain scope')
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client DiscoveryFailure: Cannot use v2 authentication with domain scope
2016-11-24 01:48:54.920 1876 ERROR ceilometer.nova_client
2016-11-24 01:48:54.923 1876 DEBUG kazoo.clien

Jeba Paulaiyan (jebap)
Changed in juniperopenstack:
importance: Undecided → High
assignee: nobody → Nitish Krishna Kaveri (nitishk)
milestone: none → r4.0
Abhay Joshi (abhayj)
Changed in juniperopenstack:
assignee: Nitish Krishna Kaveri (nitishk) → Siva Gurumurthy (sgurumurthy)
Siva Gurumurthy (sgurumurthy)
Changed in juniperopenstack:
assignee: Siva Gurumurthy (sgurumurthy) → Megh Bhatt (meghb)
summary: - SM:R3.2 : ceilometer sample list not working
+ Ceilometer:R3.2 : ceilometer sample list not working
tags: added: openstack
removed: server-manager
Revision history for this message
Siva Gurumurthy (sgurumurthy) wrote :

This issue is not specific to SM and is seen in fab too.
Assigning this to Megh to determine if this is a valid bug and take appropriate action

Revision history for this message
Siva Gurumurthy (sgurumurthy) wrote :

root@nodec21:/etc/contrail# ceilometer sample-list
+----+-------------+------+------+--------+------+-----------+
| ID | Resource ID | Name | Type | Volume | Unit | Timestamp |
 +----+-------------+------+------+--------+------+-----------+
+----+-------------+------+------+--------+------+-----------+
root@nodec21:/etc/contrail# ceilometer meter-list
+------+------+------+-------------+---------+------------+
| Name | Type | Unit | Resource ID | User ID | Project ID |
 +------+------+------+-------------+---------+------------+
+------+------+------+-------------+---------+------------+

Ashish Ranjan (aranjan-n)
tags: added: analytics
Revision history for this message
Megh Bhatt (meghb) wrote :
Download full text (21.1 KiB)

Can you please confirm if any instances were launched and then ceilometer meter-list was run?

I tried on a box - 10.84.23.58 with the same build, provisioned using fab and after launching the instance and using floating IP, the meter-list shows up fine:

root@b2s35:~# (source /etc/contrail/openstackrc; ceilometer meter-list)
+---------------------------------+------------+-----------+-----------------------------------------------------------------------+----------------------------------+----------------------------------+
| Name | Type | Unit | Resource ID | User ID | Project ID |
 +---------------------------------+------------+-----------+-----------------------------------------------------------------------+----------------------------------+----------------------------------+
| cpu | cumulative | ns | 7c39ffe2-e927-4b76-916e-76874fe50010 | 243535eec55243219484aa8d2607b15e | 20edad3cd54b4770836ebe9dc90129cc |
| cpu | cumulative | ns | b3be1104-9c35-46b6-a288-e19dff2139bc | 243535eec55243219484aa8d2607b15e | 20edad3cd54b4770836ebe9dc90129cc |
| cpu.delta | delta | ns | 7c39ffe2-e927-4b76-916e-76874fe50010 | 243535eec55243219484aa8d2607b15e | 20edad3cd54b4770836ebe9dc90129cc |
| cpu.delta | delta | ns | b3be1104-9c35-46b6-a288-e19dff2139bc | 243535eec55243219484aa8d2607b15e | 20edad3cd54b4770836ebe9dc90129cc |
| cpu_util | gauge | % | 7c39ffe2-e927-4b76-916e-76874fe50010 | 243535eec55243219484aa8d2607b15e | 20edad3cd54b4770836ebe9dc90129cc |
| cpu_util | gauge | % | b3be1104-9c35-46b6-a288-e19dff2139bc | 243535eec55243219484aa8d2607b15e | 20edad3cd54b4770836ebe9dc90129cc |
| disk.allocation | gauge | B | 7c39ffe2-e927-4b76-916e-76874fe50010 | 243535eec55243219484aa8d2607b15e | 20edad3cd54b4770836ebe9dc90129cc |
| disk.allocation | gauge | B | b3be1104-9c35-46b6-a288-e19dff2139bc | 243535eec55243219484aa8d2607b15e | 20edad3cd54b4770836ebe9dc90129cc |
| disk.capacity | gauge | B | 7c39ffe2-e927-4b76-916e-76874fe50010 | 243535eec55243219484aa8d2607b15e | 20edad3cd54b4770836ebe9dc90129cc |
| disk.capacity | gauge | B | b3be1104-9c35-46b6-a288-e19dff2139bc | 243535eec55243219484aa8d2607b15e | 20edad3cd54b4770836ebe9dc90129cc |
| disk.device.allocation | gauge | B | 7c39ffe2-e927-4b76-916e-76874fe50010-vda | 243535eec55243219484aa8d2607b15e | 20edad3cd54b4770836ebe9dc90129cc |
| disk.device.allocation ...

Changed in juniperopenstack:
assignee: Megh Bhatt (meghb) → sundarkh (sundar-kh)
status: New → Incomplete
Revision history for this message
sundarkh (sundar-kh) wrote :

Issue is seen only with SM provisioned node

Jeba Paulaiyan (jebap)
tags: added: sanity
Revision history for this message
Nitish Krishna Kaveri (nitishk) wrote :

Additional issues found:

Provisioning ceilometer fails in Keystone v2 due to v3 parameters coming into ceilometer.conf by default: project_domain_name, user_domain_name

ceilometer-agent-notification and ceilometer-plugin-contrail packages missing in SM. The service ceilometer-agent-notification is also not running due to this.

We are not calling ceilometer::agent::notification in puppet

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/27544
Submitter: Nitish Krishna Kaveri (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.2

Review in progress for https://review.opencontrail.org/27545
Submitter: Nitish Krishna Kaveri (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/27544
Committed: http://github.org/Juniper/contrail-puppet/commit/6ed0d4c0eedf282630b7d7a5f494f6085b9d1670
Submitter: Zuul (<email address hidden>)
Branch: master

commit 6ed0d4c0eedf282630b7d7a5f494f6085b9d1670
Author: nitishkrishna <email address hidden>
Date: Thu Dec 22 11:40:23 2016 -0800

Closes-Bug: #1644490 - Ceilometer meter-list issue fixed

The meter-list and sample-list were not showing up.
The ceilometer.conf was changed to match fab based provision.
service_credentials section has new params os_auth_url and os_tenant_name.
Also new resource ceilometer::agent::notification was brought in.
This change was verified for Keystone V2 but doesnt seem to work with V3.0

Change-Id: I086d125153a20ac234d9d25d72047472f58d173e

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/27545
Committed: http://github.org/Juniper/contrail-puppet/commit/f063c7d47ecf5b13af096c3f04d2c2c32236d0ff
Submitter: Zuul (<email address hidden>)
Branch: R3.2

commit f063c7d47ecf5b13af096c3f04d2c2c32236d0ff
Author: nitishkrishna <email address hidden>
Date: Thu Dec 22 11:40:23 2016 -0800

Closes-Bug: #1644490 - Ceilometer meter-list issue fixed

The meter-list and sample-list were not showing up.
The ceilometer.conf was changed to match fab based provision.
service_credentials section has new params os_auth_url and os_tenant_name.
Also new resource ceilometer::agent::notification was brought in.
This change was verified for Keystone V2 but doesnt seem to work with V3.0

Change-Id: I086d125153a20ac234d9d25d72047472f58d173e

Revision history for this message
Nitish Krishna Kaveri (nitishk) wrote :

3.2 era bug opened on R3.1
Dheeraj confirmed 3.1 has ceilometer meter list working, probably closed as a separate bug

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.