Recon does not report each policy's information

Bug #1375327 reported by Daisuke Morita
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Object Storage (swift)
Fix Released
Undecided
Daisuke Morita

Bug Description

All policy-related recon actions do not respond each policy's information.

For example, the number of async pending files is different among policies, but the recon response shows only policy-0's information as follows:

{“async_pending”: 2} # it just shows policy-0's amount

Changed in swift:
assignee: nobody → Daisuke Morita (morita-daisuke)
Revision history for this message
Daisuke Morita (morita-daisuke) wrote :

My suggested response format is as follows:

{“async_pending”: 5, # shows total
  “policy”: {“policy-0”: {“async_pending”: 2}, “policy-1”: {“async_pending”: 3}} # shows each policy's information
}

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to swift (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/125179

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Related fix proposed to branch: master
Review: https://review.openstack.org/137769

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Related fix proposed to branch: master
Review: https://review.openstack.org/138697

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to swift (master)

Reviewed: https://review.openstack.org/137769
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=f8fa1a9234cd0c1d7466dd51d8528e2abd859c2b
Submitter: Jenkins
Branch: master

commit f8fa1a9234cd0c1d7466dd51d8528e2abd859c2b
Author: Daisuke Morita <email address hidden>
Date: Fri Nov 28 20:19:02 2014 +0900

    Show each policy's information on quarantined files in recon

    After the release of Swift ver. 2.0.0, some recon responses do not
    show each policy's information yet. To make things worse, some recon
    results only count on policy-0's score, therefore the total is not
    shown in the recon results.

    This patch makes the count of quarantined files policy-aware for recon
    requests. Suppose a number of quarantined objects for policy-0 is 2
    and a number for policy-1 is 3, recon sums up every policy's amount
    and shows information for each policy as follows.

    $ curl http://<host>:<port>/recon/quarantined
    {"accounts": 0, "containers": 0, "objects": 5, "policies": {"0":
    {"objects": 2}, "1": {"objects": 3}}}

    Moreover, this patch adds stats for each policy in CLI output.

    Change-Id: I07217c635f6fc4ea809ddbc3d859c4e81c4fde37
    Related-Bug: 1375327
    Related-Bug: 1375332

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to swift (feature/ec)

Related fix proposed to branch: feature/ec
Review: https://review.openstack.org/158087

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to swift (feature/ec)
Download full text (15.3 KiB)

Reviewed: https://review.openstack.org/158087
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=c84d50f43fdfbdc72ea9c0867769cfa7fdbb09c0
Submitter: Jenkins
Branch: feature/ec

commit db29ffc98384a9f61c2d4cc48bb2faf9f5dd0478
Author: Samuel Merritt <email address hidden>
Date: Fri Feb 20 11:04:24 2015 -0800

    Make proxy_logging close the WSGI iterator

    PEP 333 says that the WSGI framework will call .close() on the
    iterator returned by a WSGI application once it's done, provided such
    a method exists. So, if our code wraps an iterator, then we have to
    call .close() on it once we're done with it. proxy_logging wasn't.

    Since WSGIContext gets it right, I looked at making proxy_logging use
    WSGIContext. However, WSGIContext is all about forcing the first chunk
    out of the iterator so that it can capture the final HTTP status and
    headers; it doesn't help if you want to look at every chunk.
    proxy_logging wants every chunk so it can count the bytes sent.

    This didn't hurt anything in Swift, but pconstantine was complaining
    in IRC that our failure to call .close() was goofing up some other
    middleware he had.

    Change-Id: Ic6ea0795ccef6cda2b5c6737697ef7d58eac9ab4

commit 4ca08cc395e686265574366497a6869e94eebcb2
Author: Alistair Coles <email address hidden>
Date: Tue Feb 17 10:27:44 2015 +0000

    Update guest VM OS recommendation in SAIO doc

    The target development platform has changed to Ubuntu 14.04 [1].
    This patch makes the suggested SAIO platform the same.

    Also, remove pointer to wiki page for other platform install
    instructions that either redirects back to this SAIO doc or
    to another wike page and then a dead link.

    [1] I0a96bcf692bb240f3ab5aab7fefd294a07735a83

    DocImpact

    Change-Id: I9f96104b5437c1f1f28f924c048ef83cf03338f4

commit 7bc09dfdea0893a49e50005b22b426ae21c11d22
Author: Arnaud JOST <email address hidden>
Date: Wed Feb 18 11:56:11 2015 +0100

    Add support of x-remove- headers for container-sync

    If the used tool to send header doesn't support empty headers (older versions
    of curl), x-remove can be used to remove metadata.
    sync-key and sync-to metadata, used by container-sync, can now be removed using
    x-remove headers.

    Change-Id: I0edb4d5425a99d20a973aa4fceaf9af6c2ddecc0

commit 949804eda4a85c3c960b0baa090e16f1aa48e95e
Author: John Dickinson <email address hidden>
Date: Mon Feb 16 14:00:24 2015 -0800

    update the getting started doc

    Change-Id: I0a96bcf692bb240f3ab5aab7fefd294a07735a83

commit dd1a05f52765a6273906b5d6ce55f81e269bad12
Author: OpenStack Proposal Bot <email address hidden>
Date: Mon Feb 16 06:30:54 2015 +0000

    Imported Translations from Transifex

    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure

    Change-Id: I013976c6192a8bff891c9050f829ae7a1e2fec59

commit 7acc2911296f48f10165282eee6bfe8e5c817a69
Author: John Dickinson <email address hidden>
Date: Sun Feb 15 17:14:31 2015 -0800

    added swift_source to ratelimit info calls

    Change-Id: I2b4cc...

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to swift (feature/crypto)

Related fix proposed to branch: feature/crypto
Review: https://review.openstack.org/158370

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to swift (feature/crypto)
Download full text (17.0 KiB)

Reviewed: https://review.openstack.org/158370
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=24d3f51386965197c36d506228e34ca1c4100165
Submitter: Jenkins
Branch: feature/crypto

commit db29ffc98384a9f61c2d4cc48bb2faf9f5dd0478
Author: Samuel Merritt <email address hidden>
Date: Fri Feb 20 11:04:24 2015 -0800

    Make proxy_logging close the WSGI iterator

    PEP 333 says that the WSGI framework will call .close() on the
    iterator returned by a WSGI application once it's done, provided such
    a method exists. So, if our code wraps an iterator, then we have to
    call .close() on it once we're done with it. proxy_logging wasn't.

    Since WSGIContext gets it right, I looked at making proxy_logging use
    WSGIContext. However, WSGIContext is all about forcing the first chunk
    out of the iterator so that it can capture the final HTTP status and
    headers; it doesn't help if you want to look at every chunk.
    proxy_logging wants every chunk so it can count the bytes sent.

    This didn't hurt anything in Swift, but pconstantine was complaining
    in IRC that our failure to call .close() was goofing up some other
    middleware he had.

    Change-Id: Ic6ea0795ccef6cda2b5c6737697ef7d58eac9ab4

commit 4ca08cc395e686265574366497a6869e94eebcb2
Author: Alistair Coles <email address hidden>
Date: Tue Feb 17 10:27:44 2015 +0000

    Update guest VM OS recommendation in SAIO doc

    The target development platform has changed to Ubuntu 14.04 [1].
    This patch makes the suggested SAIO platform the same.

    Also, remove pointer to wiki page for other platform install
    instructions that either redirects back to this SAIO doc or
    to another wike page and then a dead link.

    [1] I0a96bcf692bb240f3ab5aab7fefd294a07735a83

    DocImpact

    Change-Id: I9f96104b5437c1f1f28f924c048ef83cf03338f4

commit 7bc09dfdea0893a49e50005b22b426ae21c11d22
Author: Arnaud JOST <email address hidden>
Date: Wed Feb 18 11:56:11 2015 +0100

    Add support of x-remove- headers for container-sync

    If the used tool to send header doesn't support empty headers (older versions
    of curl), x-remove can be used to remove metadata.
    sync-key and sync-to metadata, used by container-sync, can now be removed using
    x-remove headers.

    Change-Id: I0edb4d5425a99d20a973aa4fceaf9af6c2ddecc0

commit 949804eda4a85c3c960b0baa090e16f1aa48e95e
Author: John Dickinson <email address hidden>
Date: Mon Feb 16 14:00:24 2015 -0800

    update the getting started doc

    Change-Id: I0a96bcf692bb240f3ab5aab7fefd294a07735a83

commit dd1a05f52765a6273906b5d6ce55f81e269bad12
Author: OpenStack Proposal Bot <email address hidden>
Date: Mon Feb 16 06:30:54 2015 +0000

    Imported Translations from Transifex

    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure

    Change-Id: I013976c6192a8bff891c9050f829ae7a1e2fec59

commit 7acc2911296f48f10165282eee6bfe8e5c817a69
Author: John Dickinson <email address hidden>
Date: Sun Feb 15 17:14:31 2015 -0800

    added swift_source to ratelimit info calls

    Change-Id: I2...

Revision history for this message
Christopher Bartz (bartz) wrote :

The disk_usage stats option does also not work for object rings other than object.ring.gz . E.g., it is not possible to call

swift-recon object-2 -d

because 'object-2' is not member of the allowed check_types.

Revision history for this message
Thiago da Silva (thiagodasilva) wrote :

cbartz, what you mentioned in comment #10 seems to be a new bug. I'd like to fix this bug as 'Fix released' and create a new bug with the issue you mentioned.

Changed in swift:
status: New → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.