Comment 2 for bug 1630290

Revision history for this message
Nemanja Miletic (nmiletic) wrote :

Hi Stanislaw,

- Snapshots are not working:
[root@fuel cone]# fuel snapshot
Can't connect to Nailgun server!
Please check connection settings in your configuration file.

- Installation fails and services are not availabled:
[root@fuel cone]# dockerctl list -l
Name Image Status Full container name
nginx fuel/nginx_8.0 Not created fuel-core-8.0-nginx
rabbitmq fuel/rabbitmq_8.0 Running fuel-core-8.0-rabbitmq
astute fuel/astute_8.0 Not created fuel-core-8.0-astute
rsync fuel/rsync_8.0 Not created fuel-core-8.0-rsync
keystone fuel/keystone_8.0 Not created fuel-core-8.0-keystone
postgres fuel/postgres_8.0 Running fuel-core-8.0-postgres
rsyslog fuel/rsyslog_8.0 Not created fuel-core-8.0-rsyslog
nailgun fuel/nailgun_8.0 Not created fuel-core-8.0-nailgun
cobbler fuel/cobbler_8.0 Not created fuel-core-8.0-cobbler
ostf fuel/ostf_8.0 Not created fuel-core-8.0-ostf
mcollective fuel/mcollective_8.0 Not created fuel-core-8.0-mcollective

- Please find /var/log in the attachment

- Please note "execution expired" in /var/log/docker-logs/puppet/rabbitmq.log

2016-10-06 07:59:54 +0000 Puppet (debug): Executing '/usr/sbin/rabbitmqctl add_vhost /'
2016-10-06 08:00:04 +0000 Puppet (err): Could not set 'present' on ensure: execution expired at 37:/etc/puppet/modules/nailgun/manifests/rabbitmq.pp
2016-10-06 08:00:04 +0000 Puppet (err): Could not set 'present' on ensure: execution expired at 37:/etc/puppet/modules/nailgun/manifests/rabbitmq.pp
Wrapped exception:
execution expired

- Please note that around that time we also have beam.smp crash

Oct 6 07:59:22 fuel kdumpctl: Starting kdump: [OK]
Oct 6 07:59:22 fuel systemd[1]: Started Crash recovery kernel arming.
Oct 6 07:59:31 fuel kernel: 2_scheduler[21218]: segfault at 8 ip 00000000005b8fb0 sp 00007f9db837db30 error 4 in beam.smp[400000+2a7000]
Oct 6 07:59:37 fuel kernel: async_5[21484]: segfault at 0 ip 00000000005b8c81 sp 00007ff06e897e00 error 4 in beam.smp[400000+2a7000]

- Bug that prevents the rabbitmqctl to be run again when it times out is in: /etc/puppet/liberty-8.0/modules/rabbitmq/lib/puppet/provider/rabbitmqctl.rb. Rescue catches wrong exception. Timeout needs to be changed to Timeout::Error.

- I did not try to fix the above issue as that would require rebuilding rabbitmq docker image which I am not sure how to do properly.

Thanks,
Nemanja