Astute log do not update after error

[Vladimir Sharshov]

Iso 6.0 #112

Step to reproduce:

1. create simple cluster with 2 nodes,
2. run provision,
3. after both nodes are rebooting, reboot one of them to bootstrap mode.
4. wait when second node will be provisioned and you got error ' Run command: 'apt-get clean; apt-get update' in nodes: ["1", "2"] fail. Check debug output for more information. Inspect Astute logs for the details'
5. Inspect logs. You should saw old logs without errors.

If you can watch astute logs updates with real time, you will see new logs, when after error old logs in top of logs and no error.

[Vladimir Sharshov (vsharshov)]

Actual view in web at moment when in logs last records was:

2014-11-18T13:58:16 debug: [769] Main worker task id is 14455468-074d-4955-95d5-97aa047a9105
2014-11-18T13:58:16 info: [769] Processing RPC call 'dump_environment'
2014-11-18T13:58:17 debug: [769] 14455468-074d-4955-95d5-97aa047a9105: MC agent 'uploadfile', method 'upload', results: {:sender=>"master", :statuscode=>0, :statusmsg=>"OK", :data=>{:msg=>"File was uploaded!"}}
2014-11-18T13:58:17 debug: [769] Try to execute command: shotgun -c /tmp/dump_config >> /var/log/dump.log 2>&1 && cat /var/www/nailgun/dump/last
2014-11-18T13:58:48 debug: [769] 14455468-074d-4955-95d5-97aa047a9105: MC agent 'execute_shell_command', method 'execute', results: {:sender=>"master", :statuscode=>0, :statusmsg=>"OK", :data=>{:stdout=>"/var/www/nailgun/dump/fuel-snapshot-2014-11-18_13-58-17.tgz", :stderr=>"", :exit_code=>0}}
2014-11-18T13:58:48 debug: [769] 14455468-074d-4955-95d5-97aa047a9105: stdout: /var/www/nailgun/dump/fuel-snapshot-2014-11-18_13-58-17.tgz stderr: exit code: 0
2014-11-18T13:58:48 info: [769] 14455468-074d-4955-95d5-97aa047a9105: Snapshot is done. Result: /var/www/nailgun/dump/fuel-snapshot-2014-11-18_13-58-17.tgz
2014-11-18T13:58:48 info: [769] Casting message to Nailgun: {"method"=>"dump_environment_resp", "args"=>{"task_uuid"=>"14455468-074d-4955-95d5-97aa047a9105", "status"=>"ready", "progress"=>100, "msg"=>"/var/www/nailgun/dump/fuel-snapshot-2014-11-18_13-58-17.tgz"}}

[Egor Kotko (ykotko)]

Got the this on:
{"build_id": "2014-11-14_11-06-51", "ostf_sha": "82465a94eed4eff1fc8d8e1f2fb7e9993c22f068", "build_number": "107", "auth_required": true, "api": "1.0", "nailgun_sha": "37a8122f323aa21a1d5858f2e4f0318c64fdda3f", "production": "docker", "fuelmain_sha": "caf72f3e776814371adcec70ab46e37a867bedeb", "astute_sha": "0085021fe327f6f910901b3ca55051b1df33a96e", "feature_groups": ["mirantis"], "release": "6.0", "release_versions": {"2014.2-6.0": {"VERSION": {"build_id": "2014-11-14_11-06-51", "ostf_sha": "82465a94eed4eff1fc8d8e1f2fb7e9993c22f068", "build_number": "107", "api": "1.0", "nailgun_sha": "37a8122f323aa21a1d5858f2e4f0318c64fdda3f", "production": "docker", "fuelmain_sha": "caf72f3e776814371adcec70ab46e37a867bedeb", "astute_sha": "0085021fe327f6f910901b3ca55051b1df33a96e", "feature_groups": ["mirantis"], "release": "6.0", "fuellib_sha": "37abac00bc2e5d3d2cac7d512c2cabde05428797"}}}, "fuellib_sha": "37abac00bc2e5d3d2cac7d512c2cabde05428797"}

[Vladimir Sharshov (vsharshov)]

Start new deploy and got old logs mixed with new logs.

[Przemyslaw Kaminski (pkaminski)]

This seems to be a duplicate of https://bugs.launchpad.net/fuel/+bug/1391874.

[Evgeniy L (rustyrobot)]

Yes, it might be a duplicate, but it doesn't have low priority for sure, because in this particular case API doesn't return some of the messages at all. And we show the message like "See logs for more details" and there is nothing error related in logs.

[Przemyslaw Kaminski (pkaminski)]

The logs display properly with this patch https://review.openstack.org/#/c/134946/

[Dima Shulyak (dshulyak)]

May be you will add this Bug to that patch?

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/134946
Committed: https://git.openstack.org/cgit/stackforge/fuel-web/commit/?id=9136664d8fe6f6fdea82fb08e3dc36fa0370de88
Submitter: Jenkins
Branch: master

commit 9136664d8fe6f6fdea82fb08e3dc36fa0370de88
Author: Przemyslaw Kaminski <email address hidden>
Date: Mon Nov 17 15:02:35 2014 +0100

    Fix backward file reading function

    It contained some special-case errors not caught by the current tests.
    Issue fixed, new tests added.

    Change-Id: I7e4f2a5547b64ceba2870b4b6eb87c1419d2fc61
    Closes-Bug: #1391874
    Closes-Bug: #1393795