Bug #1266636 “swift-account-audit displays AttributeError stackt...” : Bugs : OpenStack Object Storage (swift)

Revision history for this message

Kun Huang (academicgareth) wrote on 2014-01-07:

#1

I think we could add a directory, for example bin_script, at <swift-repo>/test/ to prevent some tiny mistakes like this.

Daisuke Matsui (nanodayo) on 2014-10-10

Changed in swift:
assignee:	nobody → Daisuke Matsui (nanodayo)

Revision history for this message

Filippo Giunchedi (filippo) wrote on 2014-10-22:

#2

same here, confirmed

Daisuke Matsui (nanodayo) on 2014-11-07

Changed in swift:
assignee:	Daisuke Matsui (nanodayo) → nobody

Roman Vasilets (rvasilets) on 2015-05-12

Changed in swift:
assignee:	nobody → Roman Vasilets (rvasilets)

Revision history for this message

Tim Burke (1-tim-z) wrote on 2017-09-27:

#3

Wow... I never knew about that tool! It seems... not great?

Auditing account "AUTH_test"
Auditing container "c"
  Bad status HEADing object "/AUTH_test/c/s01" on 127.0.0.1/sdb2
  Bad status HEADing object "/AUTH_test/c/s01" on 127.0.0.1/sdb7
  Bad status HEADing object "/AUTH_test/c/s01" on 127.0.0.1/sdb1
  Bad status HEADing object "/AUTH_test/c/s01" on 127.0.0.1/sdb8
  Bad status HEADing object "/AUTH_test/c/s01" on 127.0.0.1/sdb6
  ETag mismatch for "/AUTH_test/c/s02" on 127.0.0.1/sdb7
  ETag mismatch for "/AUTH_test/c/s02" on 127.0.0.1/sdb7
  ETag mismatch for "/AUTH_test/c/s02" on 127.0.0.1/sdb7
  ETag mismatch for "/AUTH_test/c/s02" on 127.0.0.1/sdb7
  ETag mismatch for "/AUTH_test/c/s02" on 127.0.0.1/sdb7
  ETag mismatch for "/AUTH_test/c/s02" on 127.0.0.1/sdb7

Also, kinda confusing that we have both swift-account-audit and swift-account-auditor... time to revisit?

Changed in swift:
status:	New → Confirmed
importance:	Undecided → Low

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-10-09: Fix merged to swift (master)

#4

Reviewed: https://review.openstack.org/508000
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=4716d3da1188eb2f2971004461554b05d0061ec6
Submitter: Jenkins
Branch: master

commit 4716d3da1188eb2f2971004461554b05d0061ec6
Author: Tim Burke <email address hidden>
Date: Wed Sep 27 22:05:40 2017 +0000

swift-account-audit: compare each etag to the hash from container

...rather than only comparing the ETag from the last response over and
over again.

NB: This tool *does not* like EC data :-(

Change-Id: Idd37f94b07f607ab8a404dd986760361c39af029
Closes-Bug: 1266636

Changed in swift:
status:	Confirmed → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-10-13: Fix proposed to swift (feature/deep)

#5

Fix proposed to branch: feature/deep
Review: https://review.openstack.org/511941

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-10-13: Fix merged to swift (feature/deep)

#6

Download full text (8.2 KiB)

Reviewed: https://review.openstack.org/511941
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=ded0343892787e4a33eea7c4f0eb14a999ec63d8
Submitter: Jenkins
Branch: feature/deep

commit 407f5394f0f5cb422c06b4e5b2f9fbfdb07782d1
Author: OpenStack Proposal Bot <email address hidden>
Date: Thu Oct 12 08:12:38 2017 +0000

Imported Translations from Zanata

For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: I628cb09aa78d8e339b4762a3c9ed8aed43941261

commit 45ca39fc68cdb42b382c1638a92cc8d3cec5529a
Author: Clay Gerrard <email address hidden>
Date: Tue Oct 10 11:47:50 2017 -0700

add mangle_client_paths to example config

Change-Id: Ic1126fc95e8152025fccf25356c253facce3e3ec

commit 94bac4ab2fe65104d602378e8e49c37b8187a75d
Author: Tim Burke <email address hidden>
Date: Fri May 12 10:55:21 2017 -0400

domain_remap: stop mangling client-provided paths

    The root_path option for domain_remap seems to serve two purposes:
     - provide the first component (version) for the backend request
     - be an optional leading component for the client request, which
       should be stripped off

As a result, we have mappings like:

c.a.example.com/v1/o -> /v1/AUTH_a/c/o

instead of

c.a.example.com/v1/o -> /v1/AUTH_a/c/v1/o

which is rather bizarre. Why on earth did we *ever* start doing this?

Now, this second behavior is managed by a config option
(mangle_client_paths) with the default being to disable it.

Upgrade Consideration
=====================

If for some reason you *do* want to drop some parts of the
client-supplied path, add

mangle_client_paths = True

to the [filter:domain_remap] section of your proxy-server.conf. Do this
before upgrading to avoid any loss of availability.

UpgradeImpact
Change-Id: I87944bfbf8b767e1fc36dbc7910305fa1f11eeed

commit a4a5494fd2fe8a43a5d50a21a1951266cc7c4212
Author: Alistair Coles <email address hidden>
Date: Mon Oct 9 11:33:28 2017 +0100

test account autocreate listing format

Related-Change: Id3ce37aa0402e2d8dd5784ce329d7cb4fbaf700d
Change-Id: I50c22225bbebff71600bea9158bda1edd18b48b0

commit 8b7f15223cde4c19fd9cbbd97e8ad79a1b4afa8d
Author: Alistair Coles <email address hidden>
Date: Mon Oct 9 10:06:19 2017 +0100

Add example to container-sync-realms.conf.5 man page

Related-Change: I0760ce149e6d74f2b3f1badebac3e36da1ab7e77

Change-Id: I129de42f91d7924c7bcb9952f17fe8a1a10ae219

commit 816331155c624c444ed123bcab412821bd7854fb
Author: HCLTech-SSW <email address hidden>
Date: Fri Oct 6 01:37:34 2017 -0700

Added the man page for container-sync-realms.conf

Updated the comments of reviewers.

Change-Id: I0760ce149e6d74f2b3f1badebac3e36da1ab7e77
Closes-Bug: #1607026

commit 747b9d928624a3f44f1f9f0269489597cddc5997
Author: Jan Zerebecki <email address hidden>
Date: Wed Oct 4 21:14:03 2017 +0200

Fix swift-ring-builder set_weight with >1 device

...

Reviewed:  https://review.openstack.org/511941
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=ded0343892787e4a33eea7c4f0eb14a999ec63d8
Submitter: Jenkins
Branch:    feature/deep

commit 407f5394f0f5cb422c06b4e5b2f9fbfdb07782d1
Author: OpenStack Proposal Bot <openstack-infra@lists.openstack.org>
Date:   Thu Oct 12 08:12:38 2017 +0000

Imported Translations from Zanata
    
    For more information about this automatic import see:
    https://docs.openstack.org/i18n/latest/reviewing-translation-import.html
    
    Change-Id: I628cb09aa78d8e339b4762a3c9ed8aed43941261

commit 45ca39fc68cdb42b382c1638a92cc8d3cec5529a
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Tue Oct 10 11:47:50 2017 -0700

add mangle_client_paths to example config
    
    Change-Id: Ic1126fc95e8152025fccf25356c253facce3e3ec

commit 94bac4ab2fe65104d602378e8e49c37b8187a75d
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri May 12 10:55:21 2017 -0400

domain_remap: stop mangling client-provided paths
    
    The root_path option for domain_remap seems to serve two purposes:
     - provide the first component (version) for the backend request
     - be an optional leading component for the client request, which
       should be stripped off
    
    As a result, we have mappings like:
    
       c.a.example.com/v1/o -> /v1/AUTH_a/c/o
    
    instead of
    
       c.a.example.com/v1/o -> /v1/AUTH_a/c/v1/o
    
    which is rather bizarre. Why on earth did we *ever* start doing this?
    
    Now, this second behavior is managed by a config option
    (mangle_client_paths) with the default being to disable it.
    
    Upgrade Consideration
    =====================
    
    If for some reason you *do* want to drop some parts of the
    client-supplied path, add
    
       mangle_client_paths = True
    
    to the [filter:domain_remap] section of your proxy-server.conf. Do this
    before upgrading to avoid any loss of availability.
    
    UpgradeImpact
    Change-Id: I87944bfbf8b767e1fc36dbc7910305fa1f11eeed

commit a4a5494fd2fe8a43a5d50a21a1951266cc7c4212
Author: Alistair Coles <alistairncoles@gmail.com>
Date:   Mon Oct 9 11:33:28 2017 +0100

test account autocreate listing format
    
    Related-Change: Id3ce37aa0402e2d8dd5784ce329d7cb4fbaf700d
    Change-Id: I50c22225bbebff71600bea9158bda1edd18b48b0

commit 8b7f15223cde4c19fd9cbbd97e8ad79a1b4afa8d
Author: Alistair Coles <alistairncoles@gmail.com>
Date:   Mon Oct 9 10:06:19 2017 +0100

Add example to container-sync-realms.conf.5 man page
    
    Related-Change: I0760ce149e6d74f2b3f1badebac3e36da1ab7e77
    
    Change-Id: I129de42f91d7924c7bcb9952f17fe8a1a10ae219

commit 816331155c624c444ed123bcab412821bd7854fb
Author: HCLTech-SSW <hcl_ss_oss@hcl.com>
Date:   Fri Oct 6 01:37:34 2017 -0700

Added the man page for container-sync-realms.conf
    
    Updated the comments of reviewers.
    
    Change-Id: I0760ce149e6d74f2b3f1badebac3e36da1ab7e77
    Closes-Bug: #1607026

commit 747b9d928624a3f44f1f9f0269489597cddc5997
Author: Jan Zerebecki <jan.openstack@zerebecki.de>
Date:   Wed Oct 4 21:14:03 2017 +0200

Fix swift-ring-builder set_weight with >1 device
    
    When iterating over the (device, weight) tuples do not carry over the
    device from the previous iteration.
    
    Closes-Bug: 1454433
    Change-Id: Iba82519b0b2bc80e2c1abbed308b651c4da4b06a

commit 839c13003aea955c48e77269d4d40a567e07dd44
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Oct 4 18:59:49 2017 +0000

Stop clearing params for account_autocreate responses
    
    Otherwise, we send back a 204 where middlewares should be expecting
    a 200 and an empty JSON array.
    
    Change-Id: I05549342327108f71b60a316f734c55bc9589915
    Related-Change: Id3ce37aa0402e2d8dd5784ce329d7cb4fbaf700d

commit 4665c175be7f5299b577925e922a59dfa33ada8c
Author: Tim Burke <tim.burke@gmail.com>
Date:   Mon Oct 2 22:56:42 2017 +0000

Clean up SLO tests and docs
    
    Change-Id: If7087cb674d6c575c4073ba09b5ef056d908655b

commit 79905ae794db2da82c8834dc24177b1820b8c53a
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Sep 27 22:10:42 2017 +0000

Replace SOSO auth prefix in examples with more-standard AUTH
    
    Change-Id: I98643d6acf248840a8360f31e446bc8ecb834898

commit 4716d3da1188eb2f2971004461554b05d0061ec6
Author: Tim Burke <tim.burke@gmail.com>
Date:   Wed Sep 27 22:05:40 2017 +0000

swift-account-audit: compare each etag to the hash from container
    
    ...rather than only comparing the ETag from the last response over and
    over again.
    
    NB: This tool *does not* like EC data :-(
    
    Change-Id: Idd37f94b07f607ab8a404dd986760361c39af029
    Closes-Bug: 1266636

commit 36a843be73e2d58c3fe49a049d514b421124bd06
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Mon Jun 27 17:31:12 2016 -0700

Preserve X-Static-Large-Object from .data file after POST
    
    You can't modify the X-Static-Large-Object metadata with a POST, an
    object being a SLO is a property of the .data file.  Revert the change
    from 4500ff which attempts to correctly handle X-Static-Large-Object
    metadata on a POST, but is subject to a race if the most recent SLO
    .data isn't available during the POST.  Instead this change adjusts the
    reading of metadata such that the X-Static-Large-Object metadata is
    always preserved from the metadata on the datafile and bleeds through
    a .meta if any.
    
    Closes-bug: #1453807
    Closes-bug: #1634723
    
    Co-Authored-By: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
    Change-Id: Ie48a38442559229a2993443ab0a04dc84717ca59

commit c662e5fc8e7800ca516468aaab582c146063c3d6
Author: Alistair Coles <alistairncoles@gmail.com>
Date:   Tue Sep 26 10:15:59 2017 +0100

Add account_autocreate=true to internal-client.conf-sample
    
    Closes-Bug: #1698426
    Change-Id: I8a29a685bb12e60f4da4a0dc8270b408241ec415

commit f90ba1acb052ca5722eccbc8611d86efa81c3f6b
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri Mar 3 20:56:39 2017 +0000

Use swift3's check_signature function
    
    This adds support for v4 while getting us out of needing to know
    how signatures work.
    
    Related-Change: Iafb6114c12deb9a40d0f8324611de27b48ed95f6
    Change-Id: I14be2845101f6af8f73bc46a416c09e4b9449515

commit 9c3c3880916dced3e04165c6a3dd79ec5ebb281b
Author: Alistair Coles <alistairncoles@gmail.com>
Date:   Wed Jun 7 11:33:29 2017 +0100

Improve domain_remap docs
    
    * Make the conditions for remapping clearer
    * Mention the path_root
    * Mention '-' -> '_' replacement in account names
    * Make example consistent with default options
    
    Change-Id: Ifd3f3775bb8b13367d964010f35813018b5b41b3

commit 808ff4fff74415d94d620f74501d28d38f451990
Author: Tim Burke <tim.burke@gmail.com>
Date:   Thu Jun 15 16:08:43 2017 -0700

Ignore all auditor_status_*.json files in reconstructor
    
    ...just like we do for the replicator. This allows third parties to define
    custom audit types that re-use the object_audit_location_generator API
    without having the reconstructor yell at them.
    
    Change-Id: I4372a1712a112705c1f906386b1cb55901256295
    Related-Change: I2f3d0bd2f1e242db6eb263c7755f1363d1430048
    Related-Change: Ib15a0987288d9ee32432c1998aefe638ca3b223b

commit 849d204c596c9089dab606ece72c84092ad156ca
Author: Tim Burke <tim.burke@gmail.com>
Date:   Fri May 12 10:43:30 2017 -0400

domain_remap: be more careful about client-path mangling
    
    The root_path option for domain_remap seems to serve two purposes:
     - provide the first component (version) for the backend request
     - be an optional leading component for the client request, which
       should be stripped off
    
    As a result, we have mappings like:
    
     c.a.example.com/     -> /v1/AUTH_a/c/
     c.a.example.com/o    -> /v1/AUTH_a/c/o
     c.a.example.com/v1/o -> /v1/AUTH_a/c/o
    
    Currently, we don't really care about whether there was a full- or
    partial-match in that first component, leading to mappings like
    
     c.a.example.com/v1o  -> /v1/AUTH_a/c/o
    
    If we're going to continue supporting that second function, we should
    only consider full-matches, so we'll have
    
     c.a.example.com/v1o  -> /v1/AUTH_a/c/v1o
    
    Change-Id: Ibdc97bb8daf117ad46177617f170d03e481b0007

tags:

added: in-feature-deep

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-10-16: Fix proposed to swift (feature/s3api)

#7

Fix proposed to branch: feature/s3api
Review: https://review.openstack.org/512277

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-10-16:

#8

Fix proposed to branch: feature/s3api
Review: https://review.openstack.org/512283

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-10-16: Change abandoned on swift (feature/s3api)

#9

Change abandoned by Alistair Coles (<email address hidden>) on branch: feature/s3api
Review: https://review.openstack.org/512283
Reason: I was just trying to get sensible topic

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-10-18: Fix merged to swift (feature/s3api)

#10

Download full text (23.2 KiB)

Reviewed: https://review.openstack.org/512277
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=f94d6567a7e2e8b3ca1168b4a41c42c1ee371af5
Submitter: Zuul
Branch: feature/s3api

commit 24188beb81d39790034fa0902246163a7bf54c91
Author: Samuel Merritt <email address hidden>
Date: Thu Oct 12 16:13:25 2017 -0700

Remove some leftover threadpool cruft.

Change-Id: I43a1a428bd96a2e18aac334c03743a9f94f7d3e1

commit 1d67485c0b935719e0c8999eb353dfd84713add6
Author: Samuel Merritt <email address hidden>
Date: Fri Apr 15 12:43:44 2016 -0700

Move all monkey patching to one function

Change-Id: I2db2e53c50bcfa17f08a136581cfd7ac4958ada2

commit 407f5394f0f5cb422c06b4e5b2f9fbfdb07782d1
Author: OpenStack Proposal Bot <email address hidden>
Date: Thu Oct 12 08:12:38 2017 +0000

Imported Translations from Zanata

For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: I628cb09aa78d8e339b4762a3c9ed8aed43941261

commit 45ca39fc68cdb42b382c1638a92cc8d3cec5529a
Author: Clay Gerrard <email address hidden>
Date: Tue Oct 10 11:47:50 2017 -0700

add mangle_client_paths to example config

Change-Id: Ic1126fc95e8152025fccf25356c253facce3e3ec

commit 94bac4ab2fe65104d602378e8e49c37b8187a75d
Author: Tim Burke <email address hidden>
Date: Fri May 12 10:55:21 2017 -0400

domain_remap: stop mangling client-provided paths

    The root_path option for domain_remap seems to serve two purposes:
     - provide the first component (version) for the backend request
     - be an optional leading component for the client request, which
       should be stripped off

As a result, we have mappings like:

c.a.example.com/v1/o -> /v1/AUTH_a/c/o

instead of

c.a.example.com/v1/o -> /v1/AUTH_a/c/v1/o

which is rather bizarre. Why on earth did we *ever* start doing this?

Now, this second behavior is managed by a config option
(mangle_client_paths) with the default being to disable it.

Upgrade Consideration
=====================

If for some reason you *do* want to drop some parts of the
client-supplied path, add

mangle_client_paths = True

to the [filter:domain_remap] section of your proxy-server.conf. Do this
before upgrading to avoid any loss of availability.

UpgradeImpact
Change-Id: I87944bfbf8b767e1fc36dbc7910305fa1f11eeed

commit a4a5494fd2fe8a43a5d50a21a1951266cc7c4212
Author: Alistair Coles <email address hidden>
Date: Mon Oct 9 11:33:28 2017 +0100

test account autocreate listing format

Related-Change: Id3ce37aa0402e2d8dd5784ce329d7cb4fbaf700d
Change-Id: I50c22225bbebff71600bea9158bda1edd18b48b0

commit 8b7f15223cde4c19fd9cbbd97e8ad79a1b4afa8d
Author: Alistair Coles <email address hidden>
Date: Mon Oct 9 10:06:19 2017 +0100

Add example to container-sync-realms.conf.5 man page

Related-Change: I0760ce149e6d74f2b3f1badebac3e36da1ab7e77

Change-Id: I129de42f91d7924c7bcb9952f17fe8a1a10ae219

commit 816331155c624c444ed123bcab412...

Reviewed:  https://review.openstack.org/512277
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=f94d6567a7e2e8b3ca1168b4a41c42c1ee371af5
Submitter: Zuul
Branch:    feature/s3api

commit 24188beb81d39790034fa0902246163a7bf54c91
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Thu Oct 12 16:13:25 2017 -0700

Remove some leftover threadpool cruft.
    
    Change-Id: I43a1a428bd96a2e18aac334c03743a9f94f7d3e1

commit 1d67485c0b935719e0c8999eb353dfd84713add6
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Fri Apr 15 12:43:44 2016 -0700

Move all monkey patching to one function
    
    Change-Id: I2db2e53c50bcfa17f08a136581cfd7ac4958ada2