[ceilometer]ceilometer-collector doesn't stop child processes after deploying

Bug #1566202 reported by Artem Minasyan
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mirantis OpenStack
Fix Released
Undecided
Igor Degtiarov
8.0.x
Fix Released
High
Igor Degtiarov
9.x
Fix Released
High
Igor Degtiarov

Bug Description

Currently ceilometer-collector services have many error entries from the oslo.messaging._drivers.impl_rabbit in the /var/log/ceilometer/ceilometer-collector.log.
This log entries caused by issue that happens when ceilometer-collector stop a collector process which can't connect to the any dispatcher. This situation happens when ceilometer-collector service starts after package installation without creation a right config file. It uses a default configs which haven't right credentials or urls for the messaging, storages and etc.
Log entry: AttributeError: 'CollectorService' object has no attribute 'sample_listener'.
This error is raised from the stop method of the Collector class and interrupt stopping of the all collector services. So, after this we have a several child processes which trying to connect to the RabbitMQ with default credentials (which invalid for the real environment) and they can't.
This issue doesn't affect real work of the Ceilometer collector, because after defining right configs and restarting ceilometer-collector services all work fine. But we have a many error log entries which bulk a ceilometer-collector.log.

Steps to reproduce:
1. Deploy an environment with 3 controllers+mongo, 1 compute, Ceilometer enabled
2. Execute vim /var/log/ceilometer/ceilometer-collector.log

Expected result: Ceilometer-collector works correctly.

Actual result:
2016-04-05 08:43:05.663 12036 ERROR oslo.messaging._drivers.impl_rabbit [-] AMQP server 192.168.0.5:5673 closed the connection. Check login credentials: Socket closed

Information about ISO:

VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "8.0"
  api: "1.0"
  build_number: "570"
  build_id: "570"
  fuel-nailgun_sha: "558ca91a854cf29e395940c232911ffb851899c1"
  python-fuelclient_sha: "4f234669cfe88a9406f4e438b1e1f74f1ef484a5"
  fuel-agent_sha: "658be72c4b42d3e1436b86ac4567ab914bfb451b"
  fuel-nailgun-agent_sha: "b2bb466fd5bd92da614cdbd819d6999c510ebfb1"
  astute_sha: "b81577a5b7857c4be8748492bae1dec2fa89b446"
  fuel-library_sha: "c2a335b5b725f1b994f78d4c78723d29fa44685a"
  fuel-ostf_sha: "3bc76a63a9e7d195ff34eadc29552f4235fa6c52"
  fuel-mirror_sha: "fb45b80d7bee5899d931f926e5c9512e2b442749"
  fuelmenu_sha: "78ffc73065a9674b707c081d128cb7eea611474f"
  shotgun_sha: "63645dea384a37dde5c01d4f8905566978e5d906"
  network-checker_sha: "a43cf96cd9532f10794dce736350bf5bed350e9d"
  fuel-upgrade_sha: "616a7490ec7199f69759e97e42f9b97dfc87e85b"
  fuelmain_sha: "d605bcbabf315382d56d0ce8143458be67c53434"

Changed in mos:
assignee: nobody → MOS Ceilometer (mos-ceilometer)
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix proposed to openstack/ceilometer (openstack-ci/fuel-8.0/liberty)

Fix proposed to branch: openstack-ci/fuel-8.0/liberty
Change author: Igor Degtiarov <email address hidden>
Review: https://review.fuel-infra.org/19446

Changed in mos:
status: New → In Progress
Changed in mos:
milestone: none → 8.0-updates
importance: Undecided → High
Ilya Tyaptin (ityaptin)
summary: - [ceilometer]Ceilometer-collector doesn't work
+ [ceilometer]Ceilometer-collector doesn't stop child processes
summary: - [ceilometer]Ceilometer-collector doesn't stop child processes
+ [ceilometer]ceilometer-collector doesn't stop child processes after
+ deploying
Ilya Tyaptin (ityaptin)
description: updated
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix proposed to openstack/ceilometer (9.0/mitaka)

Fix proposed to branch: 9.0/mitaka
Change author: Igor Degtiarov <email address hidden>
Review: https://review.fuel-infra.org/20692

Changed in mos:
milestone: 8.0-updates → 9.0
milestone: 9.0 → 8.0-updates
no longer affects: mos
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix merged to openstack/ceilometer (9.0/mitaka)

Reviewed: https://review.fuel-infra.org/20692
Submitter: Pkgs Jenkins <email address hidden>
Branch: 9.0/mitaka

Commit: 9e8597138ef85c119c89c62c0dcd6570c3a8a0d3
Author: Igor Degtiarov <email address hidden>
Date: Mon May 16 12:37:15 2016

Fix bug 1566202

Here is improved order of initialisation parameters for collector,
this fix issue with stopping child processes after deploying.
And change stop method in collector module.

Change-Id: I22612d877074f0ef6cde88d6abb013ac6e2d0333
Closes-bug: #1566202

Revision history for this message
Artem Minasyan (aminasyan) wrote :
tags: added: ct-patched-8.0
tags: added: customer-found support
Revision history for this message
Ekaterina Shutova (eshutova) wrote :

Verified on MOS 8.0 + mu4 updates.

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix proposed to openstack/ceilometer (mcp/newton)

Fix proposed to branch: mcp/newton
Change author: Igor Degtiarov <email address hidden>
Review: https://review.fuel-infra.org/33372

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix proposed to openstack/ceilometer (11.0/ocata)

Fix proposed to branch: 11.0/ocata
Change author: Igor Degtiarov <email address hidden>
Review: https://review.fuel-infra.org/33746

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix proposed to openstack/ceilometer (mcp/ocata)

Fix proposed to branch: mcp/ocata
Change author: Igor Degtiarov <email address hidden>
Review: https://review.fuel-infra.org/34477

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Change abandoned on openstack/ceilometer (mcp/newton)

Change abandoned by Ilya Tyaptin <email address hidden> on branch: mcp/newton
Review: https://review.fuel-infra.org/33372

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Change abandoned on openstack/ceilometer (mcp/ocata)

Change abandoned by Ilya Tyaptin <email address hidden> on branch: mcp/ocata
Review: https://review.fuel-infra.org/34477

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Change abandoned on openstack/ceilometer (11.0/ocata)

Change abandoned by Roman Podoliaka <email address hidden> on branch: 11.0/ocata
Review: https://review.fuel-infra.org/33746
Reason: we don't use 11.0/ocata anymore - mcp/ocata is the correct branch name

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.