CeilometerApiTests.test_list_meters failed with 404 error when tryed to get alarm list

Bug #1553189 reported by Artem Hrechanychenko
18
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Released
High
Timur Nurlygayanov

Bug Description

Reproduced on CI:
https://product-ci.infra.mirantis.net/job/9.0.system_test.ubuntu.partition_preservation/33/console

Steps to reproduce:
            1. Create a cluster
            2. Add 3 nodes with controller and mongo roles
            3. Add a node with compute and cinder roles
            4. Deploy the cluster
            5. Verify that the deployment is completed successfully

Actual result:
2016-03-03 17:03:05,397 - ERROR __init__.py:67 -- assert_ostf_run raised: AssertionError('Failed 1 OSTF tests; should fail 0 tests. Names of failed tests: \n - Ceilometer test to list meters, alarms, resources and events (failure) Failed to get list of alarms. Please refer to OpenStack logs for more details.',)
Traceback: Traceback (most recent call last):
  File "/home/jenkins/workspace/9.0.system_test.ubuntu.partition_preservation/fuelweb_test/__init__.py", line 59, in wrapped
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/9.0.system_test.ubuntu.partition_preservation/fuelweb_test/models/fuel_web_client.py", line 273, in assert_ostf_run
    indent=1)))
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/asserts.py", line 163, in assert_true
    raise ASSERTION_ERROR(message)
AssertionError: Failed 1 OSTF tests; should fail 0 tests. Names of failed tests:
  - Ceilometer test to list meters, alarms, resources and events (failure) Failed to get list of alarms. Please refer to OpenStack logs for more details.

From OSTF log:

fuel_health.common.test_mixins: INFO: STEP:2, verify action: 'getting list of alarms' ceilometerclient.openstack.common.apiclient.client: DEBUG: Request returned failure status: 404 fuel_health.common.test_mixins: DEBUG: Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/fuel_health/common/test_mixins.py", line 177, in verify result = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/ceilometerclient/v2/alarms.py", line 83, in list return self._list(options.build_url(self._path(), q)) File "/usr/lib/python2.7/site-packages/ceilometerclient/common/base.py", line 72, in _list raise exc.HTTPNotFound

Manually from controller-mongo role:
oot@node-5:~# ceilometer meter-list
+--------------------------------+-------+------+--------------------------------------+----------------------------------+----------------------------------+
| Name | Type | Unit | Resource ID | User ID | Project ID |
+--------------------------------+-------+------+--------------------------------------+----------------------------------+----------------------------------+
| disk.ephemeral.size | gauge | GB | 3f5553a7-43fb-4f53-8ce9-aa25eb172fa3 | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| disk.ephemeral.size | gauge | GB | 69dce2e9-e9ef-496f-a343-7e3033161f17 | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| disk.ephemeral.size | gauge | GB | 961128af-7473-4333-b278-689cd17f7387 | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| disk.ephemeral.size | gauge | GB | abf55017-2c82-47b7-a514-af7edd38bb73 | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| disk.ephemeral.size | gauge | GB | cece4dc2-8134-44b0-8d8a-903f72aefdb2 | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| disk.ephemeral.size | gauge | GB | d1e8728e-2e6c-4d4e-b345-50ac74d4aa0e | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| disk.ephemeral.size | gauge | GB | d531ec3a-7584-424a-99c1-73a13781828f | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| disk.root.size | gauge | GB | 3f5553a7-43fb-4f53-8ce9-aa25eb172fa3 | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| disk.root.size | gauge | GB | 69dce2e9-e9ef-496f-a343-7e3033161f17 | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| disk.root.size | gauge | GB | 961128af-7473-4333-b278-689cd17f7387 | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| disk.root.size | gauge | GB | abf55017-2c82-47b7-a514-af7edd38bb73 | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| disk.root.size | gauge | GB | cece4dc2-8134-44b0-8d8a-903f72aefdb2 | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| disk.root.size | gauge | GB | d1e8728e-2e6c-4d4e-b345-50ac74d4aa0e | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| disk.root.size | gauge | GB | d531ec3a-7584-424a-99c1-73a13781828f | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| image.download | delta | B | a805ef68-ac50-4ad2-912f-b4a55f397039 | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| image.download | delta | B | e579fd20-0ed6-4552-9544-9c83ad84582b | None | None |
| image.serve | delta | B | a805ef68-ac50-4ad2-912f-b4a55f397039 | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| image.serve | delta | B | e579fd20-0ed6-4552-9544-9c83ad84582b | None | None |
| image.size | gauge | B | a805ef68-ac50-4ad2-912f-b4a55f397039 | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| image.size | gauge | B | c869c9df-76a7-4315-99db-ecb8da7b76a9 | None | d348405e3eb94632839e021a170be313 |
| image.size | gauge | B | e579fd20-0ed6-4552-9544-9c83ad84582b | None | None |
| memory | gauge | MB | 3f5553a7-43fb-4f53-8ce9-aa25eb172fa3 | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| memory | gauge | MB | 69dce2e9-e9ef-496f-a343-7e3033161f17 | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| memory | gauge | MB | 961128af-7473-4333-b278-689cd17f7387 | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| memory | gauge | MB | abf55017-2c82-47b7-a514-af7edd38bb73 | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| memory | gauge | MB | cece4dc2-8134-44b0-8d8a-903f72aefdb2 | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| memory | gauge | MB | d1e8728e-2e6c-4d4e-b345-50ac74d4aa0e | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| memory | gauge | MB | d531ec3a-7584-424a-99c1-73a13781828f | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| storage.objects.incoming.bytes | delta | B | 6b3151b7cb1249cfa16f622c9e6778c0 | 9556315800e04d4ca895610b64cb209e | 6b3151b7cb1249cfa16f622c9e6778c0 |
| storage.objects.outgoing.bytes | delta | B | 6b3151b7cb1249cfa16f622c9e6778c0 | 9556315800e04d4ca895610b64cb209e | 6b3151b7cb1249cfa16f622c9e6778c0 |
| vcpus | gauge | vcpu | 3f5553a7-43fb-4f53-8ce9-aa25eb172fa3 | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| vcpus | gauge | vcpu | 69dce2e9-e9ef-496f-a343-7e3033161f17 | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| vcpus | gauge | vcpu | 961128af-7473-4333-b278-689cd17f7387 | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| vcpus | gauge | vcpu | abf55017-2c82-47b7-a514-af7edd38bb73 | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| vcpus | gauge | vcpu | cece4dc2-8134-44b0-8d8a-903f72aefdb2 | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| vcpus | gauge | vcpu | d1e8728e-2e6c-4d4e-b345-50ac74d4aa0e | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| vcpus | gauge | vcpu | d531ec3a-7584-424a-99c1-73a13781828f | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| volume.size | gauge | GB | a3beca46-fb36-4e53-a19e-4f49c7d1d7df | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
| volume.size | gauge | GB | f5a3e896-bfd4-4fd4-91e6-507651272283 | 6ba37f9e47544cc2a29fb34de8081a67 | d348405e3eb94632839e021a170be313 |
+--------------------------------+-------+------+--------------------------------------+----------------------------------+----------------------------------+

Revision history for this message
Artem Hrechanychenko (agrechanichenko) wrote :
summary: - OSTF test
+ [OSTF]
fuel_health.tests.sanity.test_sanity_ceilometer.CeilometerApiTests.test_list_meters
failed with 404 error when tryed to get metric list
summary: - [OSTF]
- fuel_health.tests.sanity.test_sanity_ceilometer.CeilometerApiTests.test_list_meters
+ [OSTF]fuel_health.tests.sanity.test_sanity_ceilometer.CeilometerApiTests.test_list_meters
failed with 404 error when tryed to get metric list
tags: added: swarm-blocker
Revision history for this message
Artem Hrechanychenko (agrechanichenko) wrote : Re: [OSTF]fuel_health.tests.sanity.test_sanity_ceilometer.CeilometerApiTests.test_list_meters failed with 404 error when tryed to get metric list

There a lot of swarm test failed due this bug

Changed in fuel:
assignee: nobody → MOS QA Team (mos-qa)
Revision history for this message
Vitaly Gusev (vgusev) wrote :

In Mitaka release all alarm-related code was moved from Ceilometer to the separate project - aodh. We need to rewrite all tests for Ceilometer alarms using aodh client.

Changed in fuel:
status: New → Confirmed
summary: [OSTF]fuel_health.tests.sanity.test_sanity_ceilometer.CeilometerApiTests.test_list_meters
- failed with 404 error when tryed to get metric list
+ failed with 404 error when tryed to get alarm list
Changed in fuel:
assignee: MOS QA Team (mos-qa) → Artem Minasyan (aminasyan)
tags: added: area-ostf
removed: area-qa
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-ostf (master)

Fix proposed to branch: master
Review: https://review.openstack.org/291156

Changed in fuel:
assignee: Artem Minasyan (aminasyan) → Vitaly Gusev (vgusev)
status: Confirmed → In Progress
Changed in fuel:
assignee: Vitaly Gusev (vgusev) → Artem Minasyan (aminasyan)
summary: - [OSTF]fuel_health.tests.sanity.test_sanity_ceilometer.CeilometerApiTests.test_list_meters
- failed with 404 error when tryed to get alarm list
+ CeilometerApiTests.test_list_meters failed with 404 error when tryed to
+ get alarm list
Changed in fuel:
assignee: Artem Minasyan (aminasyan) → Timur Nurlygayanov (tnurlygayanov)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-ostf (master)

Reviewed: https://review.openstack.org/291156
Committed: https://git.openstack.org/cgit/openstack/fuel-ostf/commit/?id=d6ec2aa719cb693d8d23f80130960f6d852768f2
Submitter: Jenkins
Branch: master

commit d6ec2aa719cb693d8d23f80130960f6d852768f2
Author: vgusev <email address hidden>
Date: Thu Mar 10 15:59:17 2016 +0300

    [Ceilometer] Fix for sanity test

    Add aodhclient to fuel-ostf and rewrite sanity check using it

    Change-Id: I90cdbe08c67d0a79e7c10f2e4bee7d9e11d8165a
    Closes-bug: #1553189

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
Tatyanka (tatyana-leontovich) wrote :

test stilll fails on fuel-ostf-9.0.0-1.mos920.noarch.rpm
* Thu Mar 31 2016 vgusev <email address hidden> - 9.0.0-1.mos920
- d6ec2aa [Ceilometer] Fix for sanity test
- 24db798 Get cinder-block-device role
- 6e5c57c Merge "Add RAM filters and increased the timers in heat tests"
- 09a8a87 Merge "Fix ironic check_service_availability"
- 2bc719d Revert "Workaround for UCA known bug in nova service-list"
- 894e3e8 Fix ironic check_service_availability
- aa8375a Add RAM filters and increased the timers in heat tests
- 1bccfe6 Workaround for UCA known bug in nova service-list
- 003272d Update tox.ini to skip dtabase creation steps
- 155450a Change link to fuel documentation in OSTF

Revision history for this message
Tatyanka (tatyana-leontovich) wrote :
Changed in fuel:
status: Fix Committed → Confirmed
Revision history for this message
Tatyanka (tatyana-leontovich) wrote :

Error Message

Failed 1 OSTF tests; should fail 0 tests. Names of failed tests:
  - Ceilometer test to list meters, alarms, resources and events (failure) Failed to get list of alarms. Please refer to OpenStack logs for more details.

https://172.18.160.103/job/9.0.system_test.ubuntu.services_ha/64/testReport/junit/%28root%29/deploy_heat_ha/deploy_heat_ha/

Revision history for this message
Bug Checker Bot (bug-checker) wrote : Autochecker

(This check performed automatically)
Please, make sure that bug description contains the following sections filled in with the appropriate data related to the bug you are describing:

expected result

For more detailed information on the contents of each of the listed sections see https://wiki.openstack.org/wiki/Fuel/How_to_contribute#Here_is_how_you_file_a_bug

tags: added: need-info
Revision history for this message
Timur Nurlygayanov (tnurlygayanov) wrote :

This issue didn't reproduce on my environment with MOS 9.0 #143 build, but we have another issue which breaks OSTF on my environment:
https://bugs.launchpad.net/mos/+bug/1524682

This issue is already fixed too and we expect that all Ceilometer OSTF tests will pass for 153+ builds.

Let's wait for the next SWARM results and mark the status to Fix Released if all tests will pass.

Revision history for this message
Alexandr Kostrikov (akostrikov-mirantis) wrote :
Revision history for this message
Timur Nurlygayanov (tnurlygayanov) wrote :

Yes, these tests failed because of different issue:
https://bugs.launchpad.net/fuel/+bug/1566371

When this fix for #1566371 will be merged this issue will be fixed as well.

Le't wait for 1-2 days while it will be merged.

Changed in fuel:
status: Confirmed → Invalid
status: Invalid → Confirmed
Revision history for this message
Timur Nurlygayanov (tnurlygayanov) wrote :

All required fixes were merged, one fix is still on review https://review.openstack.org/#/c/302668/ but this fix is related to another bug.

Status changed to Fix Committed, let's wait for green BVT and move it to Fix Released after green SWARM OS tests.

Changed in fuel:
status: Confirmed → Fix Committed
Revision history for this message
Artem Minasyan (aminasyan) wrote :
Changed in fuel:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.