No way to specify swift.conf location when looking for swift_hash_path_suffix

Bug #1091007 reported by Mikolaj Golub
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Object Storage (swift)
Low
Unassigned

Bug Description

When looking for swift_hash_path_suffix the Swith uses config location /etc/swift/swift.conf, hard coded in common/utils.py.

So there is no a way to change the location, while sometime it may be handy, e.g. when testing and deploying swift under ordinary user.

It would be nice if the location could be redefined e.g. via environment variable (see the attached patch).

Revision history for this message
Mikolaj Golub (to-my-trociny) wrote :
Revision history for this message
clayg (clay-gerrard) wrote :

Someone else is already working on this here: https://review.openstack.org/#/c/17004/ - can you see test out that change and comment if it would resolve your issue?

Revision history for this message
Mikolaj Golub (to-my-trociny) wrote :

Thanks. It works for me!

Revision history for this message
Flavio Percoco (flaper87) wrote :

Just found out that a bug was filled for this. The patch is still waiting for approval!

Changed in swift:
status: New → In Progress
assignee: nobody → Flavio Percoco Premoli (flaper87)
importance: Undecided → Low
Revision history for this message
Kun Huang (academicgareth) wrote :

A bit confusing
To set SWIFT_DIR in some kind conf ie better?
We set SWIFT_DIR for swift.conf, but proxy-server, account-server, container-server use the first arg as conf without checking some dir like SWIFT_DIR. So, is setting SWIFT_DIR a little useless?

Revision history for this message
Mikolaj Golub (to-my-trociny) wrote :

I can start servers (proxy, account, etc...) specifying their config file in command line, like:

  /usr/local/bin/swift-proxy-server /path/to/proxy-server.conf

I can set swift_dir in the config. And this mostly works except swift_hash_path_suffix, which location is hard coded to /etc/swift/swift.conf

I am running a test script (as non-root), which deploys mini-swift in a user local directory, but for this to work I still need to create /etc/swift/swift.conf as a root and set swift_hash_path_suffix, which is globally used by all test environments on this host.

My (simplified) approach to fix this was to allow to set the location of swift.conf with swift_hash_path_suffix via SWIFT_DIR environment variable. Flavio provided much more general solution. I hope it will be committed eventually.

Revision history for this message
Kun Huang (academicgareth) wrote :

I know your case, and Flavio's patch could let us choose SWIFT_DIR. But my question is that whether this solution is good for general cases?

The value in env is only stable with setting in ~/.bashrc or /etc/rc3.d/S99rc.local. Is it better to set it in config file?

According to that SWIFT_DIR is property of all swift services and proxy-server is just one of those, we can't set it in proxy-server.conf. That seems a environment setting is better..

If trying to fix it as a bug, I have to think about general cases more.

Revision history for this message
clayg (clay-gerrard) wrote :

What if swift daemons would accept a conf.d style config "directory" on the command line as the first argument in place of a specific config file (which would of course continue to work in a backwards-compatible way as long as the configs were in /etc/swift)

Most daemons already have a default filename for their config if you don't pass one in... if the first arg is a directory - at the least you could set swift_dir; munge in the default conf filename; and continue as normal.

Revision history for this message
Kun Huang (academicgareth) wrote :

In another word, should we treat /etc/swift/swift.conf (not /etc/swift only) as a kind of config?

Revision history for this message
Sergio Rubio (rubiojr) wrote :

Hey folks, bitten by this too.

Trying to use the Ring class from outside Swift and I need to either patch Swift or create /etc/swift/swift.conf as stated above.

[9349][rubiojr.napoleon] cat test.py
from swift.common.ring import Ring

Ring('.', ring_name = 'object')

[~/tmp/pyfoo]
[9349][rubiojr.napoleon] python test.py
Error: [swift-hash]: both swift_hash_path_suffix and swift_hash_path_prefix are missing from /etc/swift/swift.conf

Let me know if a simple patch to encapsulate the code in common/constraints.py and common/utils.py to read the config from a configurable path would help to move things forward.

Revision history for this message
Sergio Rubio (rubiojr) wrote :

In my particular case, monkey patching works well enough apparently in case anyone is interested:

import swift.common.utils
swift.common.utils.HASH_PATH_SUFFIX = '0xxxxxxxxxxxxxxxxx'
swift.common.utils.HASH_PATH_PREFIX = '6xxxxxxxxxxxxxxxxx'

from swift.common.ring import Ring

ring = Ring('./', ring_name = 'object')
print ring.get_nodes('AUTH_7xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx')

Revision history for this message
Samuel Merritt (torgomatic) wrote :

Hm, sounds like there may be a couple of different use cases here.

For running Swift under a different user or in a different location, I think making the location of swift.conf configurable is a reasonable idea. Probably an environment variable is the way to go there, because otherwise you wind up needing a config file to find your config file.

For using swift.common.ring.Ring outside of Swift, I think a better way to go is to modify the Ring class to be able to take a prefix and a suffix at instantiation time and use those instead of the defaults.

That is, something like:

    ring = Ring('/path/to/file', ring_name='object', hash_path_prefix="Two peanuts vere...", hash_path_suffix="...und von vas assaulted... peanut.")

That'd need a little bit of plumbing work to get the prefix into hash_path, but nothing too awful.

Revision history for this message
Sergio Rubio (rubiojr) wrote :

Thanks Samuel, adding that to the Ring class for proper encapsulation sounds like a good approach.

clayg (clay-gerrard)
Changed in swift:
status: In Progress → Confirmed
assignee: Flavio Percoco (flaper87) → nobody
Revision history for this message
John Dickinson (notmyname) wrote :

needs to be reverified. Last activity on this was about 2 years ago, and this part of the code has changed a _lot_ since then

Changed in swift:
status: Confirmed → Incomplete
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for OpenStack Object Storage (swift) because there has been no activity for 60 days.]

Changed in swift:
status: Incomplete → Expired
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers