sample internal client config can not create dot accounts

Bug #1698426 reported by clayg on 2017-06-16
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Object Storage (swift)
Wishlist
Alistair Coles

Bug Description

The example internal-client.conf (and expirer and reconciler historically) does to have `account_autocreate = true` which means it takes the default account_autocreate = false

Which sounds weird/bad - we *also* have the auto_create_account_prefix option which defaults to . - and a lot of the places we *use* internal-client.conf (and it's cousins object-expirer.conf, container-reconciler.conf) operate on these allegedly "auto-created magic dot-accounts"

... but it's not possible to create dot account using InternalClient/internal-client.conf unless you turn on the `account_autocreate = true` config option (i.e. the default config doesn't "work")

... except it *does* work. I mean... my *production* object-expirer.conf and container-reconciler.conf don't have account_autocreate = true and it's not all horribly broken?

But in my reading this is at least half on accident.

As near as I can tell for a few reasons the object-expirer and container-reconciler end up creating their .expiring_objects and .misplaced_object accounts via direct container PUTs - the container server special cases dot accounts and the container-updater creates the accounts.

Then there's a little gem in proxy.controllers.base that lets get_container_info return a container's cache info before the container-updater has auto-vivified the account when that is needed (a PUT to a container in an auto-created account head of the container-updater).

But you definately can *not* create containers and upload objects into "auto created" dot-accounts using InternalClient/internal-client.conf unless you set `account_autocreate = true` (or happen to only create these "auto-created" .account/container's using direct container requests to the back-end storage)

Seems like the default example config should be more conducive to getting stuff done?

/me shurgs

clayg (clay-gerrard) on 2017-06-16
tags: added: low-hanging-fruit
Alistair Coles (alistair-coles) wrote :

The container sharder uses internal client to create shard clients and needs account_autocreate=true

Changed in swift:
status: New → Confirmed
assignee: nobody → Alistair Coles (alistair-coles)

Fix proposed to branch: master
Review: https://review.openstack.org/507447

Changed in swift:
status: Confirmed → In Progress

Reviewed: https://review.openstack.org/507447
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=c662e5fc8e7800ca516468aaab582c146063c3d6
Submitter: Jenkins
Branch: master

commit c662e5fc8e7800ca516468aaab582c146063c3d6
Author: Alistair Coles <email address hidden>
Date: Tue Sep 26 10:15:59 2017 +0100

    Add account_autocreate=true to internal-client.conf-sample

    Closes-Bug: #1698426
    Change-Id: I8a29a685bb12e60f4da4a0dc8270b408241ec415

Changed in swift:
status: In Progress → Fix Released
Download full text (8.2 KiB)

Reviewed: https://review.openstack.org/511941
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=ded0343892787e4a33eea7c4f0eb14a999ec63d8
Submitter: Jenkins
Branch: feature/deep

commit 407f5394f0f5cb422c06b4e5b2f9fbfdb07782d1
Author: OpenStack Proposal Bot <email address hidden>
Date: Thu Oct 12 08:12:38 2017 +0000

    Imported Translations from Zanata

    For more information about this automatic import see:
    https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

    Change-Id: I628cb09aa78d8e339b4762a3c9ed8aed43941261

commit 45ca39fc68cdb42b382c1638a92cc8d3cec5529a
Author: Clay Gerrard <email address hidden>
Date: Tue Oct 10 11:47:50 2017 -0700

    add mangle_client_paths to example config

    Change-Id: Ic1126fc95e8152025fccf25356c253facce3e3ec

commit 94bac4ab2fe65104d602378e8e49c37b8187a75d
Author: Tim Burke <email address hidden>
Date: Fri May 12 10:55:21 2017 -0400

    domain_remap: stop mangling client-provided paths

    The root_path option for domain_remap seems to serve two purposes:
     - provide the first component (version) for the backend request
     - be an optional leading component for the client request, which
       should be stripped off

    As a result, we have mappings like:

       c.a.example.com/v1/o -> /v1/AUTH_a/c/o

    instead of

       c.a.example.com/v1/o -> /v1/AUTH_a/c/v1/o

    which is rather bizarre. Why on earth did we *ever* start doing this?

    Now, this second behavior is managed by a config option
    (mangle_client_paths) with the default being to disable it.

    Upgrade Consideration
    =====================

    If for some reason you *do* want to drop some parts of the
    client-supplied path, add

       mangle_client_paths = True

    to the [filter:domain_remap] section of your proxy-server.conf. Do this
    before upgrading to avoid any loss of availability.

    UpgradeImpact
    Change-Id: I87944bfbf8b767e1fc36dbc7910305fa1f11eeed

commit a4a5494fd2fe8a43a5d50a21a1951266cc7c4212
Author: Alistair Coles <email address hidden>
Date: Mon Oct 9 11:33:28 2017 +0100

    test account autocreate listing format

    Related-Change: Id3ce37aa0402e2d8dd5784ce329d7cb4fbaf700d
    Change-Id: I50c22225bbebff71600bea9158bda1edd18b48b0

commit 8b7f15223cde4c19fd9cbbd97e8ad79a1b4afa8d
Author: Alistair Coles <email address hidden>
Date: Mon Oct 9 10:06:19 2017 +0100

    Add example to container-sync-realms.conf.5 man page

    Related-Change: I0760ce149e6d74f2b3f1badebac3e36da1ab7e77

    Change-Id: I129de42f91d7924c7bcb9952f17fe8a1a10ae219

commit 816331155c624c444ed123bcab412821bd7854fb
Author: HCLTech-SSW <email address hidden>
Date: Fri Oct 6 01:37:34 2017 -0700

    Added the man page for container-sync-realms.conf

    Updated the comments of reviewers.

    Change-Id: I0760ce149e6d74f2b3f1badebac3e36da1ab7e77
    Closes-Bug: #1607026

commit 747b9d928624a3f44f1f9f0269489597cddc5997
Author: Jan Zerebecki <email address hidden>
Date: Wed Oct 4 21:14:03 2017 +0200

    Fix swift-ring-builder set_weight with >1 device

  ...

Read more...

tags: added: in-feature-deep

Fix proposed to branch: feature/s3api
Review: https://review.openstack.org/512283

Change abandoned by Alistair Coles (<email address hidden>) on branch: feature/s3api
Review: https://review.openstack.org/512283
Reason: I was just trying to get sensible topic

Download full text (23.2 KiB)

Reviewed: https://review.openstack.org/512277
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=f94d6567a7e2e8b3ca1168b4a41c42c1ee371af5
Submitter: Zuul
Branch: feature/s3api

commit 24188beb81d39790034fa0902246163a7bf54c91
Author: Samuel Merritt <email address hidden>
Date: Thu Oct 12 16:13:25 2017 -0700

    Remove some leftover threadpool cruft.

    Change-Id: I43a1a428bd96a2e18aac334c03743a9f94f7d3e1

commit 1d67485c0b935719e0c8999eb353dfd84713add6
Author: Samuel Merritt <email address hidden>
Date: Fri Apr 15 12:43:44 2016 -0700

    Move all monkey patching to one function

    Change-Id: I2db2e53c50bcfa17f08a136581cfd7ac4958ada2

commit 407f5394f0f5cb422c06b4e5b2f9fbfdb07782d1
Author: OpenStack Proposal Bot <email address hidden>
Date: Thu Oct 12 08:12:38 2017 +0000

    Imported Translations from Zanata

    For more information about this automatic import see:
    https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

    Change-Id: I628cb09aa78d8e339b4762a3c9ed8aed43941261

commit 45ca39fc68cdb42b382c1638a92cc8d3cec5529a
Author: Clay Gerrard <email address hidden>
Date: Tue Oct 10 11:47:50 2017 -0700

    add mangle_client_paths to example config

    Change-Id: Ic1126fc95e8152025fccf25356c253facce3e3ec

commit 94bac4ab2fe65104d602378e8e49c37b8187a75d
Author: Tim Burke <email address hidden>
Date: Fri May 12 10:55:21 2017 -0400

    domain_remap: stop mangling client-provided paths

    The root_path option for domain_remap seems to serve two purposes:
     - provide the first component (version) for the backend request
     - be an optional leading component for the client request, which
       should be stripped off

    As a result, we have mappings like:

       c.a.example.com/v1/o -> /v1/AUTH_a/c/o

    instead of

       c.a.example.com/v1/o -> /v1/AUTH_a/c/v1/o

    which is rather bizarre. Why on earth did we *ever* start doing this?

    Now, this second behavior is managed by a config option
    (mangle_client_paths) with the default being to disable it.

    Upgrade Consideration
    =====================

    If for some reason you *do* want to drop some parts of the
    client-supplied path, add

       mangle_client_paths = True

    to the [filter:domain_remap] section of your proxy-server.conf. Do this
    before upgrading to avoid any loss of availability.

    UpgradeImpact
    Change-Id: I87944bfbf8b767e1fc36dbc7910305fa1f11eeed

commit a4a5494fd2fe8a43a5d50a21a1951266cc7c4212
Author: Alistair Coles <email address hidden>
Date: Mon Oct 9 11:33:28 2017 +0100

    test account autocreate listing format

    Related-Change: Id3ce37aa0402e2d8dd5784ce329d7cb4fbaf700d
    Change-Id: I50c22225bbebff71600bea9158bda1edd18b48b0

commit 8b7f15223cde4c19fd9cbbd97e8ad79a1b4afa8d
Author: Alistair Coles <email address hidden>
Date: Mon Oct 9 10:06:19 2017 +0100

    Add example to container-sync-realms.conf.5 man page

    Related-Change: I0760ce149e6d74f2b3f1badebac3e36da1ab7e77

    Change-Id: I129de42f91d7924c7bcb9952f17fe8a1a10ae219

commit 816331155c624c444ed123bcab412...

tags: added: in-feature-s3api
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers