Ceilometer swift pollsters aren't working

[Vadim Rovachev]

{"build_id": "2014-08-03_02-01-14", "ostf_sha": "a3fa823ea0e4e03beb637ae07a91adea82c33182", "build_number": "389", "auth_required": true, "api": "1.0", "nailgun_sha": "bd0127be0061029f9f910547db5e633c82244942", "production": "docker", "fuelmain_sha": "e99879292cf6e96b8991300d947df76b69134bb1", "astute_sha": "ce86172e77661026c91fdf1ff8066d7df1f7d89d", "feature_groups": ["mirantis"], "release": "5.1", "fuellib_sha": "4e3fdd75f8dabde8e5d07067545d8043a70a176b"}

Env:

virt_type=kvm
config_mode=multinode
release_name=Icehouse on CentOS 6.5
net_provider=neutron
net_segment_type=gre
debug=true
auto_assign_floating_ip=true
nova_quota=true

settings:
   "volumes_lvm": false
   "volumes_ceph": true
   "images_ceph": true
   "ephemeral_ceph": false
   "objects_ceph": true
   "osd_pool_size": 1
   "sahara": true
   "murano": false
   "ceilometer": true

nodes:
   "node_1_controller":{"manufacturer":"KVM", "roles": ["controller", "mongo"]},
   "node_2_compute":{"manufacturer":"Supermicro", "roles": ["compute"]},
   "node_3_ceph":{"manufacturer":"KVM", "roles": ["ceph-osd"]}}

Swift pollsters(http://docs.openstack.org/developer/ceilometer/measurements.html#object-storage-swift) does not working.

log file in controller /var/log/ceilometer/central.log have erorrs:
http://paste.openstack.org/show/90449/

[Denis Egorenko (degorenko)]

Verified on ISO 425, bug is reproduced.

The reason is wrong URL for polling pollsters:
https://github.com/openstack/ceilometer/blob/stable/icehouse/ceilometer/objectstore/swift.py#L83

So, if we try to use curl:
# curl -I -H X-Auth-Token:2c6179171af243f3b88681c8dcde6ee6 http://172.16.0.4:6780/v1/AUTH_56c73c5a792844b797d06aa9a812b24f
HTTP/1.1 400 Bad Request
Date: Tue, 12 Aug 2014 14:52:27 GMT
Server: Apache/2.2.15 (CentOS)
Accept-Ranges: bytes
Content-Length: 83
Connection: close
Content-Type: application/xml

The same problem will be when we use URL from 'master':
https://github.com/openstack/ceilometer/blob/master/ceilometer/objectstore/swift.py#L81

# curl -I -H X-Auth-Token:2c6179171af243f3b88681c8dcde6ee6 http://172.16.0.4:6780/swift/v1/AUTH_56c73c5a792844b797d06aa9a812b24f
HTTP/1.1 404 Not Found
Date: Tue, 12 Aug 2014 14:51:44 GMT
Server: Apache/2.2.15 (CentOS)
Accept-Ranges: bytes
Content-Length: 12
Connection: close
Content-Type: text/plain; charset=utf-8

But, i found URL which works:
# curl -I -H X-Auth-Token:2c6179171af243f3b88681c8dcde6ee6 http://172.16.0.4:6780/swift/v1/
HTTP/1.1 204 No Content
Date: Tue, 12 Aug 2014 14:51:48 GMT
Server: Apache/2.2.15 (CentOS)
X-Account-Container-Count: 2
X-Account-Object-Count: 4
X-Account-Bytes-Used: 253258037
X-Account-Bytes-Used-Actual: 253259776
Connection: close
Content-Type: text/plain; charset=utf-8

Then, i replaced source code (with my URL) and result:
# ceilometer meter-list
+----------------------------+-------+-----------+--------------------------------------+----------------------------------+----------------------------------+
| Name | Type | Unit | Resource ID | User ID | Project ID |
+----------------------------+-------+-----------+--------------------------------------+----------------------------------+----------------------------------+
| storage.objects | gauge | object | 56c73c5a792844b797d06aa9a812b24f | None | 56c73c5a792844b797d06aa9a812b24f |
| storage.objects | gauge | object | 82bfef509df14dbfb72ce1f2df4b017f | None | 82bfef509df14dbfb72ce1f2df4b017f |
| storage.objects.containers | gauge | container | 56c73c5a792844b797d06aa9a812b24f | None | 56c73c5a792844b797d06aa9a812b24f |
| storage.objects.containers | gauge | container | 82bfef509df14dbfb72ce1f2df4b017f | None | 82bfef509df14dbfb72ce1f2df4b017f |
| storage.objects.size | gauge | B | 56c73c5a792844b797d06aa9a812b24f | None | 56c73c5a792844b797d06aa9a812b24f |
| storage.objects.size | gauge | B | 82bfef509df14dbfb72ce1f2df4b017f | None | 82bfef509df14dbfb72ce1f2df4b017f |

So, there are two possible ways:
1) Bug in source code for ceilometer stable/icehouse (i will test it on multi-node HA additionally);
2) This URL is particularity for RadosGW. As a result, we need different settings for this mode.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/114482

[OSCI Robot (oscirobot)]

Package ceilometer has been built from changeset: http://gerrit.mirantis.com/20600
RPM Repository URL: http://osci-obs.vm.mirantis.net:82/centos-fuel-master-20600/centos

[OSCI Robot (oscirobot)]

Package ceilometer has been built from changeset: http://gerrit.mirantis.com/20600
DEB Repository URL: http://osci-obs.vm.mirantis.net:82/ubuntu-fuel-master-20600/ubuntu

[Dmitry Mescheryakov (dmitrymex)]

The bug does not require any change in Ceilometer code, so removing MOS as an affected project.

[Dina Belova (dbelova)]

Dmitry, that's not true. Option used by Denis is not implemented in the Ceilometer. That's my task to impl it. Here is the change https://gerrit.mirantis.com/#/c/20600/

[Dmitry Mescheryakov (dmitrymex)]

Dina, thank you for correction

[Dina Belova (dbelova)]

Known issue
==========

If we're using Ceph and Rados, Ceilometer won't poll Swift due to the endpoints incompatibility for plain Swift installation and installation with Ceph. This will be solved in 6.0.

Workaround
==========

None

[Dina Belova (dbelova)]

#release-notes-new-variant

Known issue
==========

If we're using Ceph and Rados Gateway, Ceilometer won't poll Ceph due to the endpoints incompatibility between plain Swift installation and installation with Ceph.

Workaround
==========

None

[OSCI Robot (oscirobot)]

Package ceilometer has been built from changeset: http://gerrit.mirantis.com/20600
DEB Repository URL: http://osci-obs.vm.mirantis.net:82/ubuntu-fuel-master-20600/ubuntu

[OSCI Robot (oscirobot)]

Package ceilometer has been built from changeset: http://gerrit.mirantis.com/20600
RPM Repository URL: http://osci-obs.vm.mirantis.net:82/centos-fuel-master-20600/centos

[OSCI Robot (oscirobot)]

Package ceilometer has been built from changeset: http://gerrit.mirantis.com/28872
RPM Repository URL: http://osci-obs.vm.mirantis.net:82/centos-fuel-6.0-stable-28872/centos

[OSCI Robot (oscirobot)]

Package ceilometer has been built from changeset: http://gerrit.mirantis.com/28872
DEB Repository URL: http://osci-obs.vm.mirantis.net:82/ubuntu-fuel-6.0-stable-28872/ubuntu

[OSCI Robot (oscirobot)]

Package ceilometer has been built from changeset: http://gerrit.mirantis.com/29860
RPM Repository URL: http://osci-obs.vm.mirantis.net:82/centos-fuel-6.0-stable-29860/centos

[OSCI Robot (oscirobot)]

Package ceilometer has been built from changeset: http://gerrit.mirantis.com/20600
RPM Repository URL: http://osci-obs.vm.mirantis.net:82/centos-fuel-master-20600/centos

[OSCI Robot (oscirobot)]

Package ceilometer has been built from changeset: http://gerrit.mirantis.com/29860
DEB Repository URL: http://osci-obs.vm.mirantis.net:82/ubuntu-fuel-6.0-stable-29860/ubuntu

[OSCI Robot (oscirobot)]

Package ceilometer has been built from changeset: http://gerrit.mirantis.com/20600
DEB Repository URL: http://osci-obs.vm.mirantis.net:82/ubuntu-fuel-master-20600/ubuntu

[OSCI Robot (oscirobot)]

Package ceilometer has been built from changeset: http://gerrit.mirantis.com/29860
RPM Repository URL: http://osci-obs.vm.mirantis.net:82/centos-fuel-6.0-stable/centos

[OSCI Robot (oscirobot)]

Package ceilometer has been built from changeset: http://gerrit.mirantis.com/29860
DEB Repository URL: http://osci-obs.vm.mirantis.net:82/ubuntu-fuel-6.0-stable/ubuntu

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/114482
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=9b35a41896cdd55aca517e198aa3063e58c362ac
Submitter: Jenkins
Branch: master

commit 9b35a41896cdd55aca517e198aa3063e58c362ac
Author: Denis Egorenko <email address hidden>
Date: Mon Oct 6 14:59:12 2014 +0400

    Fix problem with Ceilometer Swift pollsters for RadosGW

    When RadosGW is used as Swift backend, Swift pollsters
    didn't work, because RadosGW has different endpoint
    from native Swift. In this case, Ceilometer should check
    parameter 'swift_rados_backend' and choose correct endpoint

    Change-Id: If27a62da8f849868848a4e7ab62a275b0a0d323d
    Closes-Bug: #1352861

[Bogdan Dobrelya (bogdando)]

Please check if https://bugs.launchpad.net/fuel/+bug/1380800 is a dup or related to this one

[Anastasia Palkina (apalkina)]

Verified on ISO #79

"build_id": "2014-11-05_21-28-15", "ostf_sha": "9c6fadca272427bb933bc459e14bb1bad7f614aa", "build_number": "79", "auth_required": true, "api": "1.0", "nailgun_sha": "01579a53c2be841dc4cafb0be00213243dde7667", "production": "docker", "fuelmain_sha": "35eed79fd6dcfc86a9206ffcd90d385ba5dcca56", "astute_sha": "c72dac7b31646fbedbfc56a2a87676c6d5713fcf", "feature_groups": ["mirantis"], "release": "6.0", "release_versions": {"2014.2-6.0": {"VERSION": {"build_id": "2014-11-05_21-28-15", "ostf_sha": "9c6fadca272427bb933bc459e14bb1bad7f614aa", "build_number": "79", "api": "1.0", "nailgun_sha": "01579a53c2be841dc4cafb0be00213243dde7667", "production": "docker", "fuelmain_sha": "35eed79fd6dcfc86a9206ffcd90d385ba5dcca56", "astute_sha": "c72dac7b31646fbedbfc56a2a87676c6d5713fcf", "feature_groups": ["mirantis"], "release": "6.0", "fuellib_sha": "03c63d4233f6e1c182cc58213cd4542aba5b706d"}}}, "fuellib_sha": "03c63d4233f6e1c182cc58213cd4542aba5b706d"

[Dmitry Borodaenko (angdraug)]

Marked as Won't Fix, priority is too low for a backport.