statsenderd sometimes fails to read nailgun settings.yaml

[Matthew Mosesohn]

Source: https://fuel-jenkins.mirantis.com/job/fuellib_review_master_node/1549/console

Symptoms in /var/log/docker-logs/nailgun/statsenderd.log
2015-03-11 08:45:04.547 DEBUG [7fb2f50c0700] (settings) Looking for settings.yaml package config using old style __file__
2015-03-11 08:45:04.549 DEBUG [7fb2f50c0700] (settings) Trying to read config file /usr/lib/python2.6/site-packages/nailgun/settings.yaml
2015-03-11 08:45:04.683 DEBUG [7fb2f50c0700] (settings) Trying to read config file /etc/nailgun/settings.yaml
2015-03-11 08:45:04.683 ERROR [7fb2f50c0700] (settings) Error while reading config file /etc/nailgun/settings.yaml: 'NoneType' object is not iterable
2015-03-11 08:45:04.684 DEBUG [7fb2f50c0700] (settings) Trying to read config file /etc/fuel/version.yaml
2015-03-11 08:45:04.687 INFO [7fb2f50c0700] (settings) DEVELOPMENT MODE ON:
2015-03-11 08:45:04.687 INFO [7fb2f50c0700] (settings) Static dir is /usr/lib/python2.6/site-packages/static
2015-03-11 08:45:04.687 INFO [7fb2f50c0700] (settings) Template dir is /usr/lib/python2.6/site-packages/static
2015-03-11 08:45:05.112 INFO [7fb2f50c0700] (statsenderd) Starting standalone stats sender...
2015-03-11 08:45:05.223 ERROR [7fb2f50c0700] (statsenderd) Stats sender exception: (OperationalError) could not connect to server: Connection refused
 Is the server running on host "localhost" (::1) and accepting
 TCP/IP connections on port 5432?
FATAL: password authentication failed for user "nailgun"

The other daemons seem to run okay except statsenderd. This bug also seems to be quite a rare floating bug.

[Artem Roma (aroma-x)]

The issue is not connected with functionality of the code of statsenderd but with error while reading configurations files ("/etc/nailgun/settings.yaml" to be precisely) in nailgun.settings module. And such error (after analyzing of traceback message) can occur in case the file is present but is empty. So I suppose we should switch focus to finding causes of that happening.

[Matthew Mosesohn (raytrac3r)]

Something seems to be emptying the content of /etc/nailgun/settings.yaml. I will continue researching this issue.

[Matthew Mosesohn (raytrac3r)]

Fixed in https://review.openstack.org/#/c/163921/