Astute failed to create /tmp/dump_config for shotgun

[Tatyanka]

VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "5.1.1"
  api: "1.0"
  build_number: "28"
  build_id: "2014-10-14_00-01-06"
  astute_sha: "f5fbd89d1e0e1f22ef9ab2af26da5ffbfbf24b13"
  fuellib_sha: "46ad455514614ec2600314ac80191e0539ddfc04"
  ostf_sha: "64cb59c681658a7a55cc2c09d079072a41beb346"
  nailgun_sha: "b9792cb5bbecddfa9c5c3afb4d0f961a2a2776a7"
  fuelmain_sha: "431350ba204146f815f0e51dd47bf44569ae1f6d"

http://jenkins-product.srt.mirantis.net:8080/view/5.1_swarm/job/5.1_fuelmain.system_test.centos.thread_3/23/consoleFull

In test ceph ha restart we have 2 node offline and failed ostf tests, so that snapshot creation was invoked
But it hangs while trying to get logs from offline node
 http://pastebin.com/DyLU6z6C

id | status | name | cluster | progress | uuid
---|---------|----------------|---------|----------|-------------------------------------
1 | ready | check_networks | 1 | 100 | 037449bc-b284-4455-bd64-5d79fe76f8af
6 | ready | deployment | 1 | 100 | 715106e9-3a9e-4cc4-b62c-9b0f4a58ff07
2 | ready | deploy | 1 | 100 | 50874599-95f9-4b75-b407-f1d94a3e6478
8 | running | dump | None | 0 | 0c1a8cab-785c-4de3-ae3d-056bc7aefe9e
5 | ready | provision | 1 | 100 | 52d712e9-f865-420c-b4b7-59f901be78ea
[root@nailgun log]# fuel node
id | status | name | cluster | ip | mac | roles | pending_roles | online
---|--------|------------------------------|---------|-------------|-------------------|----------------------|---------------|-------
3 | ready | slave-06_ceph-osd | 1 | 10.108.42.5 | 64:36:16:80:d5:ea | ceph-osd | | False
6 | ready | slave-05_compute_ceph-osd | 1 | 10.108.42.8 | 64:fe:6e:60:c5:11 | ceph-osd, compute | | True
5 | ready | slave-04_compute_ceph-osd | 1 | 10.108.42.7 | 64:64:51:1d:53:0d | ceph-osd, compute | | False
2 | ready | slave-03_controller_ceph-osd | 1 | 10.108.42.4 | 64:74:fd:85:30:66 | ceph-osd, controller | | True
1 | ready | slave-02_controller_ceph-osd | 1 | 10.108.42.3 | 64:b8:58:80:ce:70 | ceph-osd, controller | | True
4 | ready | slave-01_controller_ceph-osd | 1 | 10.108.42.6 | 64:b0:70:96:0b:96 | ceph-osd, controller | | True

[Dima Shulyak (dshulyak)]

There is 2 options:

1. filter offline nodes from shotgun configuration in nailgun
2. gracefully handle connectivity issues in shotgun itself

2 should be added for sure

[Dima Shulyak (dshulyak)]

I tried to reproduce this issue with offline nodes, but unfortunetely :) everything works fine. Some dump in commands file is empty,
but snapshot works well.

In your paste, there is an error:

IOError: [Errno 2] No such file or directory: '/tmp/dump_config'

It basically means that config for shotgun wasnt created, and we need to research what happened in astute, and why if failed
config creation, but started snapshoting

[Dmitry Borodaenko (angdraug)]

Won't Fix for 5.1.1, the priority is not High or Critical.

[Oleksiy Molchanov (omolchanov)]

This bug was incomplete for more than 4 weeks. We cannot investigate it further so we are setting the status to Invalid. If you think it is not correct, please feel free to provide requested information and reopen the bug, and we will look into it further.