swift-object-info should try harder on tombstones

Bug #1425679 reported by clayg
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Object Storage (swift)
Fix Released
Wishlist
Ricardo Ferreira

Bug Description

swift-object-info doesn't try to read any metadata from tombstones, but there's useful information in there:

vagrant@saio:~$ swift-object-info /srv/node1/sdb1/objects/407/65c/65f7b6abead7366196f1df7f904f665c/1424895101.02312.ts
Data file doesn't exist

^ not even trying

vagrant@saio:~$ python -c 'from swift.obj.diskfile import read_metadata; print read_metadata(open("/srv/node1/sdb1/objects/407/65c/65f7b6abead7366196f1df7f904f665c/1424895101.02312.ts"))'
{'X-Timestamp': '1424895101.02312', 'name': '/AUTH_test/container-9c30b832-5332-4ba9-969a-f0a4cac547e8/object-b9f6fa35-2124-46ef-ab47-034e3e0f5859'}

^ there's good stuff in there

swift-object-info should try harder to output the information it has on .ts files (and even .meta)

CVE References

clayg (clay-gerrard)
tags: added: low-hanging-fruit
Ricardo Ferreira (rsff)
Changed in swift:
assignee: nobody → Ricardo Ferreira (rsff)
clayg (clay-gerrard)
Changed in swift:
importance: Undecided → Wishlist
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to swift (master)

Fix proposed to branch: master
Review: https://review.openstack.org/162306

Changed in swift:
status: New → In Progress
Ricardo Ferreira (rsff)
Changed in swift:
status: In Progress → Fix Committed
Revision history for this message
Thierry Carrez (ttx) wrote :

Do not set to FixCommitted until the fix is merged. Thanks!

Changed in swift:
status: Fix Committed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to swift (master)

Reviewed: https://review.openstack.org/162306
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=9baafe7f305147c70d004402ebb05a22db85bc18
Submitter: Jenkins
Branch: master

commit 9baafe7f305147c70d004402ebb05a22db85bc18
Author: Ricardo Ferreira <email address hidden>
Date: Fri Mar 6 21:18:48 2015 +0000

    Allow swift-object-info to inspect .meta and .ts files

    Removing the check for ".data" extension makes it check
    *any* file for metadata, so it works with .meta and .ts
    filetypes.

    Change-Id: I43966d371218ad39414e9282cde579e48370a2a7
    Closes-Bug:1425679

Changed in swift:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to swift (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/165395

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to swift (feature/ec)

Fix proposed to branch: feature/ec
Review: https://review.openstack.org/166558

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on swift (feature/ec)

Change abandoned by paul luse (<email address hidden>) on branch: feature/ec
Review: https://review.openstack.org/166558

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to swift (feature/ec)

Fix proposed to branch: feature/ec
Review: https://review.openstack.org/166606

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on swift (feature/ec)

Change abandoned by paul luse (<email address hidden>) on branch: feature/ec
Review: https://review.openstack.org/166606

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to swift (master)

Reviewed: https://review.openstack.org/165395
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=15b83f67d2fc6645a5e68ce923ca9a2d4d1b9805
Submitter: Jenkins
Branch: master

commit 15b83f67d2fc6645a5e68ce923ca9a2d4d1b9805
Author: Alistair Coles <email address hidden>
Date: Wed Mar 18 12:43:03 2015 +0000

    Test swift-object-info opens meta and ts files

    Adds a unit test to verify the change made in [1], i.e. that
    swift-object-info will read from .meta and .ts files as well
    as .data files.

    [1] change I43966d371218ad39414e9282cde579e48370a2a7

    Change-Id: I82dde36e3a96db1a21cfe9a4cca0d941e543dfd0
    Related-Bug: 1425679

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to swift (feature/ec)

Fix proposed to branch: feature/ec
Review: https://review.openstack.org/167004

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on swift (feature/ec)

Change abandoned by John Dickinson (<email address hidden>) on branch: feature/ec
Review: https://review.openstack.org/167004
Reason: bah! didn't work

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to swift (feature/ec)

Related fix proposed to branch: feature/ec
Review: https://review.openstack.org/167429

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on swift (feature/ec)

Change abandoned by paul luse (<email address hidden>) on branch: feature/ec
Review: https://review.openstack.org/166558
Reason: in favor of a newer one

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to swift (feature/ec)
Download full text (9.5 KiB)

Reviewed: https://review.openstack.org/167429
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=5b11780a108b6024310efa1b8520b62ca1b318ef
Submitter: Jenkins
Branch: feature/ec

commit a288d05ca5d4c66dab36df40b4216e2ed87e5a91
Author: OpenStack Proposal Bot <email address hidden>
Date: Tue Mar 24 06:06:33 2015 +0000

    Imported Translations from Transifex

    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure

    Change-Id: I6ae0dd0c34ac1d7173acacdd0dd4896619592e0c

commit 0a467936621009418a9c94f636b33bfb6d947a89
Author: Lorcan <email address hidden>
Date: Thu Mar 12 15:40:39 2015 +0000

    Add swift-recon feature to track swift-drive-audit error count

    This is a follow-on from a previous commit which added recon info
    for swift-drive-audit (https://review.openstack.org/#/c/122468/).

    Here, the "--drievaudit" option is added to swift-recon tool. This
    feature gives the statistics for the system-wide drive errors flagged
    by swift-drive-audit. An example of the output is as follows:
    (verbose mode)

    swift-recon --driveaudit -v
    ===============================================================================
    --> Starting reconnaissance on 5 hosts
    ===============================================================================
    [2015-03-11 17:13:39] Checking drive-audit errors
    -> http://1.2.3.4:6000/recon/driveaudit: {'drive_audit_errors': 14}
    -> http://1.2.3.5:6000/recon/driveaudit: {'drive_audit_errors': 0}
    -> http://1.2.3.6:6000/recon/driveaudit: {'drive_audit_errors': 37}
    -> http://1.2.3.7:6000/recon/driveaudit: {'drive_audit_errors': 101}
    -> http://1.2.3.8:6000/recon/driveaudit: {'drive_audit_errors': 0}
    [drive_audit_errors] low: 0, high: 101, avg: 30.4, total: 152, Failed: 0.0%, no_result: 0, reported: 5
    ===============================================================================

    Change-Id: Ia16c52a9d613eeb3de1a5a428d88dd1233631912

commit 23d0842dec250905f68df601926ba8228392b322
Author: Thiago da Silva <email address hidden>
Date: Tue Feb 17 16:55:34 2015 -0500

    Refactoring the PUT method

    Extracting large chunks of the PUT method into smaller
    methods to improve maintainability and reuse of code.

    Based on the work that Clay Gerrard started:
    https://review.openstack.org/#/c/77812/

    Co-Authored-By: Clay Gerrard <email address hidden>

    Change-Id: Id479fc5b159a2782361ac4a6e4a6d8bbaee4fe85
    Signed-off-by: Thiago da Silva <email address hidden>

commit c3d2602c60d8a218409c9f95bef252e4811cbd29
Author: Mitsuhiro SHIGEMATSU <email address hidden>
Date: Sat Mar 21 08:42:23 2015 +0900

    Fix typo in swift/test/unit/account/test_backend.py

    Change-Id: I6cab110599019471b4005f1584fc26fa2a85d02e

commit 7e64c811fe9ecae738474239a9f73e3c6f95ddea
Author: Alistair Coles <email address hidden>
Date: Wed Feb 25 17:33:44 2015 +0000

    Enable in-process functional test policy to be configured

    Currently the in-process tests build a 2 replica, 4 partition,
    2 device object...

Read more...

Thierry Carrez (ttx)
Changed in swift:
milestone: none → 2.3.0-rc1
status: Fix Committed → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to swift (feature/crypto)

Related fix proposed to branch: feature/crypto
Review: https://review.openstack.org/175866

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to swift (feature/crypto)
Download full text (39.4 KiB)

Reviewed: https://review.openstack.org/175866
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=5bb7c286ebb4a54e4d2bd5a02845644d1c651183
Submitter: Jenkins
Branch: feature/crypto

commit e440d6aed8a40848584767ed36811bf09c738838
Author: Kota Tsuyuzaki <email address hidden>
Date: Wed Apr 15 11:25:13 2015 -0700

    Fix best response to return correct status

    Current best response could return "503 Internal Server Error".
    However, "503" means "Service Unavailable". (The status int of
    Internal Server Error is 500)

    This patch fixes the response status as "503 Service Unavailable"

    Change-Id: I88b8c52c26b19e9e76ba3375f1e16ced555ed54c

commit 57011d5699d49a47ae89073ff27b39140ab4d652
Author: Ricardo Ferreira <email address hidden>
Date: Thu Mar 12 23:18:33 2015 +0000

    More user-friendly output for object metadata

    Split out system, user and other metadata in swift-object-info. Print
    every position line by line instead of raw dict representation, so it
    would be easier to parse with tools such as grep.

    Co-Authored-By: Ricardo Ferreira <email address hidden>
    Co-Authored-By: Kamil Rykowski <email address hidden>

    Change-Id: Ia78da518c18f7e26016700aee87efb534fbd2040
    Closes-Bug: #1428866

commit a162c2bdd7be12daa29dd07230f84efcaf1cab37
Author: OpenStack Proposal Bot <email address hidden>
Date: Thu Apr 16 06:06:35 2015 +0000

    Imported Translations from Transifex

    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure

    Change-Id: I48ba06f4801ff2d7856d67e74d2b1f76c550fcf4

commit 52b102163e48dc04a6a492a3430efa1f7778d7ec
Author: Clay Gerrard <email address hidden>
Date: Wed Apr 15 15:31:06 2015 -0700

    Don't apply the wrong Etag validation to rebuilt fragments

    Because of the object-server's interaction with ssync sender's
    X-Backend-Replication-Headers when a object (or fragment archive) is
    pushed unmodified to another node it's ETag value is duped into the
    recieving ends metadata as Etag. This interacts poorly with the
    reconstructor's RebuildingECDiskFileStream which can not know ahead of
    time the ETag of the fragment archive being rebuilt.

    Don't send the Etag from the local source fragment archive being used as
    the basis for the rebuilt fragent archive's metadata along to ssync.

    Change-Id: Ie59ad93a67a7f439c9a84cd9cff31540f97f334a

commit 46bd6716ffae28aef53f15af170fd2df01b49843
Author: Kota Tsuyuzaki <email address hidden>
Date: Tue Apr 14 23:22:14 2015 -0700

    Small minor refactor on ec diskfile

    To be more helpful, add an inline comment and remove
    unnecessary assignment.

    Change-Id: Ia9c6993dfa03c238736955de8b0f5c1a7d5d1b65

commit 193fe9a5f906a2344bb5d328ad55b881e4086caa
Author: Lorcan <email address hidden>
Date: Wed Apr 15 11:32:32 2015 +0100

    Update swift-recon doc with more options

    The swit-recon tool has had several functional additions
    added recently but not all of these have been added to the docs.

    This change add...

Thierry Carrez (ttx)
Changed in swift:
milestone: 2.3.0-rc1 → 2.3.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.