Groups of swift daemons are all forced to use the same config

[Liam Young]

On swift storage servers there are three groups of services: account, container and object.

Each of these groups is comprised of a number of services, for instance: server, auditor, replicator etc

Each service has its own init script but all the services in a group are configured to use the same group config file eg swift-account, swift-account-auditor, swift-account-reaper & swift-account-replicator all use /etc/swift/account-server.conf.

Obviously this causes a problem when different services need different config. In the case of a swift cluster performing global replication the replication server need "
replication_server = true" where as the auditor needs "replication_server = false"

[Corey Bryant (corey.bryant)]

We may want to fix this along with https://bugs.launchpad.net/bugs/1800676. Sahid already took a stab at it a while back but that got delayed by me.

Liam, do you need this prior to ussuri?

[Liam Young (gnuoy)]

Hi Cory, the init script update is to support swift global replication. The upstream code and the proposed changes to the charm support the feature in mitaka so ideally the support would go right back to trusty-mitaka.

[Corey Bryant (corey.bryant)]

The switch to systemd unit files is a big change (LP: #1800676) which we can do at this point in the ussuri cycle, but I think we'll need to figure out how to adapt the existing sysvinit approach for SRUs.

[Sahid Orentino (sahid-ferdjaoui)]

Hello Liam,

I probably missed something but in the account-server.conf file there are sections, normally we should be able to set a specific value for replication_server and that for swift server and replicator.

[account-server]
replication_server = true

[account-auditor]
replication_server = false

Can you share more details on how you think that should be working?

Thanks,
s.

[Liam Young (gnuoy)]

Hi Sahid,

In our deployment for swift global replication we have two account services.
One for local and one for replication:

# cat /etc/swift/account-server/1.conf
[DEFAULT]
bind_ip = 0.0.0.0
bind_port = 6002
workers = 1

[pipeline:main]
pipeline = recon account-server

[filter:recon]
use = egg:swift#recon
recon_cache_path = /var/cache/swift

[app:account-server]
use = egg:swift#account

[account-auditor]

[account-reaper]
#
# cat /etc/swift/account-server/2.conf
[DEFAULT]
bind_ip = 0.0.0.0
bind_port = 6012
workers = 1

[pipeline:main]
pipeline = recon account-server

[filter:recon]
use = egg:swift#recon
recon_cache_path = /var/cache/swift

[app:account-server]
use = egg:swift#account
replication_server = true
#

I believe these two config files are mutually exclusive as they have different
values for the same key in both the 'DEFAULT' and 'app:account-server'
sections.

Similarly, I believe the config file for the local account service is
incompatable with the local config file for the local container service.

# cat /etc/swift/account-server/1.conf
[DEFAULT]
bind_ip = 0.0.0.0
bind_port = 6002
workers = 1

[pipeline:main]
pipeline = recon account-server

[filter:recon]
use = egg:swift#recon
recon_cache_path = /var/cache/swift

[app:account-server]
use = egg:swift#account

[account-auditor]

[account-reaper]
#
# cat /etc/swift/container-server/1.conf
[DEFAULT]
bind_ip = 0.0.0.0
bind_port = 6001
workers = 1

[pipeline:main]
pipeline = recon container-server

[filter:recon]
use = egg:swift#recon
recon_cache_path = /var/cache/swift

[app:container-server]
use = egg:swift#container
allow_versions = true

[container-updater]

[container-auditor]

I believe these two config files are mutually exclusive as they have different
values for the same key in both the 'DEFAULT' and 'pipeline:main' sections.
sections.

[Corey Bryant (corey.bryant)]

I found these docs pretty useful:

https://opendev.org/openstack/swift/src/branch/master/doc/source/replication_network.rst
https://opendev.org/openstack/swift/src/branch/master/bin/swift-init#L23

[Liam Young (gnuoy)]

Sahid pointed out that the swift-init will traverse a search path and start a daemon for every config file it finds so no change to the init script is needed. Initial tests suggest this completely covers my use case. I will continue testing and report back. I will mark the bug as invalid for the moment. Thanks Sahid !