atop logrotate number of kept files is too small

[Dmitry Sutyagin]

Currently Fuel sets up logrotate for atop in a way that only 7 days of data are kept. This has proven over time to be insufficient for any long-term issue analysis and for RCAs. Multiple support cases required atop logs from earlier periods, which were not available.

We need to increase atop storage to keep 6 months of data, and maybe make this configurable in UI before deployment. This will also require increasing minimal storage requirements for /var partition.

[Roman Rufanov (rrufanov)]

Customer found on 6.0 - most likely exists in other versions. Need to keep atop logs for at least 90 days so we can do RCAs for past issues.

[Roman Rufanov (rrufanov)]

related - https://blueprints.launchpad.net/fuel/+spec/sar-logging and https://blueprints.launchpad.net/fuel/+spec/netatop-and-atop2

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/336725

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/336726

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/336725
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=dee15ae89ea65320d45326301de299d7526e6d9d
Submitter: Jenkins
Branch: master

commit dee15ae89ea65320d45326301de299d7526e6d9d
Author: Maksim Malchuk <email address hidden>
Date: Sat Jul 2 00:27:30 2016 +0300

    Add the atop service UI settings section

    This change adds the appropriate section to UI settings page.
    It contains the default settings from the fuel-library and the
    note message for an operator.

    DocImpact
    Change-Id: Ie0e69f58183193455d5d036b45668a3476f1a693
    Partial-Bug: #1597250
    Signed-off-by: Maksim Malchuk <email address hidden>

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/336726
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=a7f03d7c55446dd035b499046b6ffc86ec112bed
Submitter: Jenkins
Branch: master

commit a7f03d7c55446dd035b499046b6ffc86ec112bed
Author: Maksim Malchuk <email address hidden>
Date: Sat Jul 2 00:55:38 2016 +0300

    Add atop configuration settings

    Now the atop setting are configurable from UI and can be overriden in
    hiera. Also, this change adds idempotency and fix noop tests.

    DocImpact
    Depends-On: Ie0e69f58183193455d5d036b45668a3476f1a693
    Change-Id: I59cea9ea5ca31c20db2cf3eaaebe09bad0abdcf8
    Closes-Bug: #1597250
    Signed-off-by: Maksim Malchuk <email address hidden>

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/8.0
Review: https://review.openstack.org/338157

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/8.0
Review: https://review.openstack.org/338194

[Rodion Tikunov (rtikunov)]

Closing this bug as Won't Fix for MOS 6 and MOS 6.1 as we have finished active support for those releases and don't merge non-critical and non-security fixes there anymore.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/7.0
Review: https://review.openstack.org/367873

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/7.0
Review: https://review.openstack.org/368031

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/367873
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=2077e661e390dc1981bd3d3537e54cff108aae58
Submitter: Jenkins
Branch: stable/7.0

commit 2077e661e390dc1981bd3d3537e54cff108aae58
Author: Maksim Malchuk <email address hidden>
Date: Sat Jul 2 00:27:30 2016 +0300

    Add the atop service UI settings section

    This change adds the appropriate section to UI settings page.
    It contains the default settings from the fuel-library and the
    note message for an operator.

    DocImpact
    Change-Id: Ie0e69f58183193455d5d036b45668a3476f1a693
    Partial-Bug: #1597250
    Signed-off-by: Maksim Malchuk <email address hidden>
    (cherry picked from commit dee15ae89ea65320d45326301de299d7526e6d9d)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/338157
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=602cd95e65392fb92d1cae14905fca59944451b9
Submitter: Jenkins
Branch: stable/8.0

commit 602cd95e65392fb92d1cae14905fca59944451b9
Author: Maksim Malchuk <email address hidden>
Date: Sat Jul 2 00:27:30 2016 +0300

    Add the atop service UI settings section

    This change adds the appropriate section to UI settings page.
    It contains the default settings from the fuel-library and the
    note message for an operator.

    DocImpact
    Change-Id: Ie0e69f58183193455d5d036b45668a3476f1a693
    Partial-Bug: #1597250
    Signed-off-by: Maksim Malchuk <email address hidden>
    (cherry picked from commit dee15ae89ea65320d45326301de299d7526e6d9d)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/368031
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=3d6aa6140b03072f79210b3cb6d982d4b8e749db
Submitter: Jenkins
Branch: stable/7.0

commit 3d6aa6140b03072f79210b3cb6d982d4b8e749db
Author: Maksim Malchuk <email address hidden>
Date: Sat Jul 2 00:55:38 2016 +0300

    Add atop configuration settings

    Now the atop setting are configurable from UI and can be overriden in
    hiera. Also, this change adds idempotency and fix noop tests.

    DocImpact
    Depends-On: Ie0e69f58183193455d5d036b45668a3476f1a693
    Change-Id: I59cea9ea5ca31c20db2cf3eaaebe09bad0abdcf8
    Closes-Bug: #1597250
    Signed-off-by: Maksim Malchuk <email address hidden>
    (cherry picked from commit a7f03d7c55446dd035b499046b6ffc86ec112bed)
    (cherry picked from commit a55cbdb8c812a3d1c4fd5be3a42d11b0e26809b9)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/338194
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=a55cbdb8c812a3d1c4fd5be3a42d11b0e26809b9
Submitter: Jenkins
Branch: stable/8.0

commit a55cbdb8c812a3d1c4fd5be3a42d11b0e26809b9
Author: Maksim Malchuk <email address hidden>
Date: Sat Jul 2 00:55:38 2016 +0300

    Add atop configuration settings

    Now the atop setting are configurable from UI and can be overriden in
    hiera. Also, this change adds idempotency and fix noop tests.

    Conflicts:
     deployment/puppet/osnailyfacter/manifests/tools/tools.pp has not
            presented in 8.0 yet so changes has been moved to
            deployment/puppet/osnailyfacter/modular/tools/tools.pp
     tests/noop/spec/hosts/tools/tools_spec.rb

    DocImpact
    Depends-On: Ie0e69f58183193455d5d036b45668a3476f1a693
    Change-Id: I59cea9ea5ca31c20db2cf3eaaebe09bad0abdcf8
    Closes-Bug: #1597250
    Signed-off-by: Maksim Malchuk <email address hidden>
    (cherry picked from commit a7f03d7c55446dd035b499046b6ffc86ec112bed)

[TatyanaGladysheva (tgladysheva)]

Verified on MOS 7.0 + MU6 updates.

[TatyanaGladysheva (tgladysheva)]

Verified on 8.0 + MU4 updates.

Steps to verify:
1. Update nailgun database to see setting 'Advanced System & Process Monitor (atop)'
2. Create cluster, set 'Interval between the snapshots' and 'Rotate days' values in Fuel UI
3. Deploy cluster
4. Check files '/etc/cron.daily/atop_retention' and '/etc/default/atop' on controller

Actual results:
Appropriate values from step 2 are present in files '/etc/cron.daily/atop_retention' and '/etc/default/atop'.