config-changed IOError: [Errno 2] No such file or directory: '/var/lib/nagios/service-check-haproxy.txt' after recent update to check_status_file.py

Bug #1670223 reported by Ryan Beisner
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Charm Helpers
Fix Released
High
Felipe Reyes
OpenStack Glance Charm
Fix Released
High
Felipe Reyes
OpenStack Keystone Charm
Fix Released
High
Unassigned

Bug Description

When resyncing charm-helpers into the Glance charm, the following hook failure exists (for at least the Trusty + Icehouse combo):

config-changed IOError: [Errno 2] No such file or directory: '/var/lib/nagios/service-check-haproxy.txt' after recent update to check_status_file.py

2017-03-03 22:22:19 INFO config-changed Setting up python-gi (3.12.0-1ubuntu1) ...
2017-03-03 22:22:19 INFO worker.uniter.jujuc server.go:172 running hook tool "juju-log" ["Check command not found: check_status_file.py"]
2017-03-03 22:22:19 DEBUG worker.uniter.jujuc server.go:173 hook context id "glance/0-config-changed-2954441461654224842"; dir "/var/lib/juju/agents/unit-glance-0/charm"
2017-03-03 22:22:19 INFO juju-log Check command not found: check_status_file.py
2017-03-03 22:22:19 INFO config-changed Traceback (most recent call last):
2017-03-03 22:22:19 INFO config-changed File "/var/lib/juju/agents/unit-glance-0/charm/hooks/config-changed", line 632, in <module>
2017-03-03 22:22:19 INFO config-changed hooks.execute(sys.argv)
2017-03-03 22:22:19 INFO config-changed File "/var/lib/juju/agents/unit-glance-0/charm/hooks/charmhelpers/core/hookenv.py", line 731, in execute
2017-03-03 22:22:19 INFO config-changed self._hooks[hook_name]()
2017-03-03 22:22:19 INFO config-changed File "/var/lib/juju/agents/unit-glance-0/charm/hooks/charmhelpers/contrib/openstack/utils.py", line 1864, in wrapped_f
2017-03-03 22:22:19 INFO config-changed restart_functions)
2017-03-03 22:22:19 INFO config-changed File "/var/lib/juju/agents/unit-glance-0/charm/hooks/charmhelpers/core/host.py", line 655, in restart_on_change_helper
2017-03-03 22:22:19 INFO config-changed r = lambda_f()
2017-03-03 22:22:19 INFO config-changed File "/var/lib/juju/agents/unit-glance-0/charm/hooks/charmhelpers/contrib/openstack/utils.py", line 1863, in <lambda>
2017-03-03 22:22:19 INFO config-changed (lambda: f(*args, **kwargs)), restart_map, stopstart,
2017-03-03 22:22:19 INFO config-changed File "/var/lib/juju/agents/unit-glance-0/charm/hooks/charmhelpers/contrib/hardening/harden.py", line 79, in _harden_inner2
2017-03-03 22:22:19 INFO config-changed return f(*args, **kwargs)
2017-03-03 22:22:19 INFO config-changed File "/var/lib/juju/agents/unit-glance-0/charm/hooks/config-changed", line 405, in config_changed
2017-03-03 22:22:19 INFO config-changed update_nrpe_config()
2017-03-03 22:22:19 INFO config-changed File "/var/lib/juju/agents/unit-glance-0/charm/hooks/config-changed", line 593, in update_nrpe_config
2017-03-03 22:22:19 INFO config-changed nrpe.add_init_service_checks(nrpe_setup, services(), current_unit)
2017-03-03 22:22:19 INFO config-changed File "/var/lib/juju/agents/unit-glance-0/charm/hooks/charmhelpers/contrib/charmsupport/nrpe.py", line 387, in add_init_service_checks
2017-03-03 22:22:19 INFO config-changed f = open(checkpath, 'w')
2017-03-03 22:22:19 INFO config-changed IOError: [Errno 2] No such file or directory: '/var/lib/nagios/service-check-haproxy.txt'
2017-03-03 22:22:19 INFO juju.worker.uniter.context context.go:579 handling reboot

Logs and artifacts available for viewing at:

https://openstack-ci-reports.ubuntu.com/artifacts/test_charm_pipeline_amulet_full/openstack/charm-glance/441237/2/725/index.html

Tags: sts uosci

Related branches

Revision history for this message
Ryan Beisner (1chb1n) wrote :
Revision history for this message
Ryan Beisner (1chb1n) wrote :

It seems that C-H commit changed the default behavior with immediate_check=True.

description: updated
Felipe Reyes (freyes)
Changed in charm-helpers:
assignee: nobody → Felipe Reyes (freyes)
Changed in charm-glance:
assignee: nobody → Felipe Reyes (freyes)
Felipe Reyes (freyes)
Changed in charm-helpers:
status: New → In Progress
Revision history for this message
Felipe Reyes (freyes) wrote :

I think here charm-helpers here is not the one at fault, because /var/lib/nagios will be created once nagios-nrpe-server is installed and here glance is calling update_nrpe_config() withouth checking if it's related to nrpe or if nagios-nrpe-server is installed.

glance should be verify it before it attempts to write the configuration.

tags: added: sts
Revision history for this message
Paul Gear (paulgear) wrote :

@freyes If you find that this is the fault of my recent charm-helpers change, or need any assistance in tracking it down, feel free to drag me in...

Revision history for this message
Felipe Reyes (freyes) wrote :

@paulgear, thanks, for the offer. I came up with a simple patch, if you could take a look would be great.

The other I was thinking is to make sure charms (in this case glance) don't call to add_init_service_checks() before nagios-nrpe-server is installed, but I'm not sure if it's a good approach considering that we may have to end up patching multiple charms due to the change in the behavior.

Revision history for this message
Paul Gear (paulgear) wrote :

@freyes Added a question to your MP; but +1 to your point that patching multiple charms is going to be harder than making add_init_service_checks() handle it. I wonder why these charms were doing things like that in the first place, though, without having the relation to the nrpe charm already established.

Revision history for this message
Ryan Beisner (1chb1n) wrote :
James Page (james-page)
Changed in charm-keystone:
status: New → Triaged
importance: Undecided → High
Changed in charm-glance:
status: New → Triaged
importance: Undecided → High
Changed in charm-helpers:
importance: Undecided → High
Felipe Reyes (freyes)
Changed in charm-helpers:
status: In Progress → Fix Released
Changed in charm-glance:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-glance (master)

Reviewed: https://review.openstack.org/441237
Committed: https://git.openstack.org/cgit/openstack/charm-glance/commit/?id=896ea18db03aef92f0d4bfaf9885c6fdcde38c5a
Submitter: Jenkins
Branch: master

commit 896ea18db03aef92f0d4bfaf9885c6fdcde38c5a
Author: Felipe Reyes <email address hidden>
Date: Fri Mar 3 14:15:34 2017 -0300

    charm-helpers sync to pickup changes to CephContext

    CephContext now is capable of dealing with a space separated list of IPs
    in the ceph-public-address field of the relation with ceph. This allows
    ceph-proxy to pass a list of monitor hosts instead of a single one.

    Change-Id: I83abc2c151636aea0b5dffc32f2e3c91acb7558b
    Partial-Bug: 1642430
    Closes-Bug: 1670223

Changed in charm-glance:
status: In Progress → Fix Committed
James Page (james-page)
Changed in charm-glance:
milestone: none → 17.08
James Page (james-page)
Changed in charm-glance:
status: Fix Committed → Fix Released
Revision history for this message
Liam Young (gnuoy) wrote :

Marking as fix released for keystone as the charmhelper fix was pulled into the keystone charm for the last charm release.

Changed in charm-keystone:
status: Triaged → Fix Released
milestone: none → 17.08
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.