Generating diagnostic snapshot failed - Error Message "Next logs aren't present in snapshot logs"

[Alexey. Kalashnikov]

One of the swarm tests failed:
https://product-ci.infra.mirantis.net/job/9.x.system_test.ubuntu.multirole/46/testReport/(root)/deploy_multirole_controller_cinder/deploy_multirole_controller_cinder/

with error message:
http://paste.openstack.org/show/566126/

[root@nailgun ~]# ll /var/dump/fuel-snapshot-2016-09-01_23-00-53/
total 37300
-rw-r--r--. 1 root root 935105 Sep 1 23:01 config.tar.gz
-rw-r--r--. 1 root root 21423955 Sep 1 23:01 logs-node-0.tar.gz
-rw-r--r--. 1 root root 12565438 Sep 1 23:01 logs-node-1.tar.gz
-rw-r--r--. 1 root root 165804 Sep 1 23:01 logs-node-2.tar.gz
-rw-r--r--. 1 root root 3096409 Sep 1 23:01 logs-node-3.tar.gz

seems like folder with mounted remotes from other nodes became not accessible in the moment of generating dump. There is no errors in dmesg or messages logs on fuel node

Or might be connection with nodes had broken at this time.

The shotgun.log is absent on fuel node
[root@nailgun ~]# find / -name shotgun.log -type f
[root@nailgun ~]#

manual run generating dump is successfully done:
[root@nailgun ~]# fuel snapshot
Generating dump...
Downloading: http://10.109.1.2:8000/api/dump/fuel-snapshot-2016-09-02_16-18-12.tar Bytes: 71352320
[==============================================================================]()

Defected snapshot available here:
https://drive.google.com/open?id=0B0EB6QSDWt2vaDZfWkpsSFBKa0k

[Stanislaw Bogatkin (sbogatkin)]

I can't understand why this is an incomplete. Sustaining team, take a look, please.

[Georgy Kibardin (gkibardin)]

# tar xvf logs-node-0.tar.gz
# ls var/log/remote/
10.109.1.5 127.0.0.1 node-1.test.domain.local node-3.test.domain.local
# ls var/log/remote/10.109.1.5/bootstrap/
agent.log dmesg.log mcollective.log syslog.log
auth.log kern.log.log messages.log udev.log

all logs are present. They are just packed differently. The test which relies on files in the snapshot needs to be fixed.

[Dmitry Belyaninov (dbelyaninov)]

Not reproduced. Please, reopen if needed