Change compression algoritm for diagnostic snapshot

Bug #1394487 reported by Sergey Galkin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Released
High
Dima Shulyak

Bug Description

We have https://bugs.launchpad.net/mos/+bug/1382511

I tried to use xz -9 for snapshot with /var/log/ = 2.9 G and have this results.

305M ноя 20 10:28 fuel-snapshot-2014-11-20_06-35-34.tar.gz
132M ноя 20 10:05 fuel-snapshot-2014-11-20_06-35-34.tar.xz

Can we use xz for snapshots in the Fuel ?

Tags: scale
Sergey Galkin (sgalkin)
tags: added: scale
Dima Shulyak (dshulyak)
Changed in fuel:
assignee: nobody → Fuel Python Team (fuel-python)
milestone: none → 6.0
importance: Undecided → High
status: New → Triaged
Changed in fuel:
assignee: Fuel Python Team (fuel-python) → Vladimir Kozhukalov (kozhukalov)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-web (master)

Fix proposed to branch: master
Review: https://review.openstack.org/135912

Changed in fuel:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-web (stable/5.1)

Fix proposed to branch: stable/5.1
Review: https://review.openstack.org/135913

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on fuel-web (stable/5.1)

Change abandoned by Vladimir Kozhukalov (<email address hidden>) on branch: stable/5.1
Review: https://review.openstack.org/135913
Reason: This change is to be back ported from master.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-main (master)

Fix proposed to branch: master
Review: https://review.openstack.org/136361

Revision history for this message
Dmitry Pyzhov (dpyzhov) wrote : Re: xz for snapshot

We need to compare possible solutions and choose the best one. See this e-mail thread: http://lists.openstack.org/pipermail/openstack-dev/2014-November/051268.html

Changed in fuel:
status: In Progress → Confirmed
assignee: Vladimir Kozhukalov (kozhukalov) → Fuel Python Team (fuel-python)
milestone: 6.0 → 6.1
Dmitry Pyzhov (dpyzhov)
summary: - xz for snapshot
+ Change compression algoritm for diagnostic snapshot
Dima Shulyak (dshulyak)
Changed in fuel:
assignee: Fuel Python Team (fuel-python) → Dima Shulyak (dshulyak)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to fuel-main (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/157331

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-web (master)

Fix proposed to branch: master
Review: https://review.openstack.org/157344

Changed in fuel:
status: Confirmed → In Progress
Revision history for this message
Dima Shulyak (dshulyak) wrote :

Assigning back to fuel-python, will get back to them ater ff

Changed in fuel:
assignee: Dima Shulyak (dshulyak) → Fuel Python Team (fuel-python)
status: In Progress → Triaged
Changed in fuel:
assignee: Fuel Python Team (fuel-python) → Dima Shulyak (dshulyak)
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to fuel-main (master)

Reviewed: https://review.openstack.org/157331
Committed: https://git.openstack.org/cgit/stackforge/fuel-main/commit/?id=404e7a7eec0f43976ecd1c1e6dc096de6641d631
Submitter: Jenkins
Branch: master

commit 404e7a7eec0f43976ecd1c1e6dc096de6641d631
Author: Dmitry Shulyak <email address hidden>
Date: Thu Feb 19 13:13:25 2015 +0200

    Add xz to shotgun spec

    xz utils already installed on master host, but we need them in
    mcollective container

    Change-Id: I2d8cd154cdb258d590227565e34de748e9c71e5d
    Related-Bug: 1394487

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-web (master)

Reviewed: https://review.openstack.org/157344
Committed: https://git.openstack.org/cgit/stackforge/fuel-web/commit/?id=a397cfe51887a163334ab12dc227090ce53125e7
Submitter: Jenkins
Branch: master

commit a397cfe51887a163334ab12dc227090ce53125e7
Author: Dmitry Shulyak <email address hidden>
Date: Thu Feb 19 14:04:03 2015 +0200

    Use xz as compression tool for snapshot

    Default compression level will: -3

    Played with it on logs with ~6Gb of data

    Snapshot compressed with:
    - gzip takes: 479m Mb
    - xz 3 level - 272 and takes 4 minutes
    - xz 9 level - 239 and takes ~ 30 min

    It is possible to use pixz utility that will run compressing
    on multiple cores, but i dont think it is wise to depend on it,
    because when master host will be installed on vm - it may take
    very long time to generate snapshot

    DocImpact
    Change-Id: I84e66e92ebc20ceba2db98ff41f08b1fde323c0b
    Closes-Bug: 1394487

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on fuel-web (master)

Change abandoned by Aleksey Kasatkin (<email address hidden>) on branch: master
Review: https://review.openstack.org/135912
Reason: Covered by https://review.openstack.org/157344

Revision history for this message
Anastasia Palkina (apalkina) wrote :

Verified on ISO #226

"build_id": "2015-03-24_22-54-44", "ostf_sha": "a4cf5f218c6aea98105b10c97a4aed8115c15867", "build_number": "226", "release_versions": {"2014.2-6.1": {"VERSION": {"build_id": "2015-03-24_22-54-44", "ostf_sha": "a4cf5f218c6aea98105b10c97a4aed8115c15867", "build_number": "226", "api": "1.0", "nailgun_sha": "3b9f2c83c63653113701971492a6d8039db2cce2", "production": "docker", "python-fuelclient_sha": "3624051242c83fdbdd1df9a0e466797c06b75043", "astute_sha": "631f96d5a09cc48bfbddcbf056b946c8a80438f0", "feature_groups": ["mirantis"], "release": "6.1", "fuelmain_sha": "320b5f46fc1b2798f9e86ed7df51d3bda1686c10", "fuellib_sha": "acd7659a727ab890076de41272c2054744be4d20"}}}, "auth_required": true, "api": "1.0", "nailgun_sha": "3b9f2c83c63653113701971492a6d8039db2cce2", "production": "docker", "python-fuelclient_sha": "3624051242c83fdbdd1df9a0e466797c06b75043", "astute_sha": "631f96d5a09cc48bfbddcbf056b946c8a80438f0", "feature_groups": ["mirantis"], "release": "6.1", "fuelmain_sha": "320b5f46fc1b2798f9e86ed7df51d3bda1686c10", "fuellib_sha": "acd7659a727ab890076de41272c2054744be4d20"

Changed in fuel:
status: Fix Committed → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on fuel-main (master)

Change abandoned by Vladimir Kozhukalov (<email address hidden>) on branch: master
Review: https://review.openstack.org/136361

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.