nova-placement-api default config files is too strict

[Corey Bryant]

If nova.conf doesn't exist in the typical location of /etc/nova/nova.conf and OS_PLACEMENT_CONFIG_DIR isn't set, nova-placement-api's wsgi application will fail. In our case with the OpenStack snap, we have two possible paths we may pick nova.conf up from, based on what --config-file specifies. I think the right answer here is to be a bit more flexible and not set the default config file if it's path doesn't exist.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/472378

[Sylvain Bauza (sylvain-bauza)]

Like I said in the review, after looking at PEP-3333, I think Nova already supports defining other places for nova.conf by defining the OS_PLACEMENT_CONFIG_DIR environment variable.
I don't really see why you couldn't want to set this specific envvar ?

[Chris Dent (cdent)]

As I said in response to you on the review, the goal is to be able to express more than one place to look for the config, which the environment variable currently does not do.

[Sylvain Bauza (sylvain-bauza)]

Let's set to Confirm because we don't have a provision yet.

[Corey Bryant (corey.bryant)]

It looks like there's more to this. From what I can tell, the placement api config doesn't actually make its way through from the placement wsgi script to keystonemiddleware via OS_PLACEMENT_CONFIG_DIR or via my attempt to fix above by passing sys.argv. While setting debug=true makes it look like the config gets applied, it doesn't actually get passed along.

See nova/api/openstack/placement/deploy.py:

# Do not provide global conf to middleware here.
        auth_middleware = auth_token.filter_factory(
            {}, oslo_config_project=project_name)

I think that results in the ConfigOpts class of oslo_config having to call find_config_files() and find_config_dirs() which will search standard config locations (~/.nova, ~/, /etc/nova, /etc) for config.

For the nova snap package, the config will either be in /snap/nova/current/etc/nova/ or /var/snap/nova/common/etc/nova, neither of which would be in oslo_config's standard search paths.

It seems that this should be fixed in nova, but oslo_config could also be updated to be a bit more flexible in their standard search paths, perhaps with snap package directories or via environment variables.

[Chris Dent (cdent)]

Corey, good digging. I'm not sure what the right approach is here.

n.a.o.p.deploy makes a lot of effort to minimize the amount of configuration is used by the wsgi stack (with a long term goal of placement being internally as minimally configurable as possible (point it at a database, make it aware of where keystone is, put something in the service catalog, done)).

Do you have an idea on how to deal with this?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/473615

[Corey Bryant (corey.bryant)]

Chris,

It seems like the right approach to this is to fix oslo.config to handle snap-specific directories. I'll visit that soon.

Corey

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/474212

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/474212
Committed: https://git.openstack.org/cgit/openstack/oslo.config/commit/?id=21e70e28b3015c1619dacfad8a2218b09e8109ec
Submitter: Jenkins
Branch: master

commit 21e70e28b3015c1619dacfad8a2218b09e8109ec
Author: Corey Bryant <email address hidden>
Date: Wed Jun 14 09:57:23 2017 -0400

    Add snap package paths to default config dirs

    With snap packaging (see snapcraft.io) the package is installed into a
    read-only squashfs filesystem, which includes the default config. For
    example, $SNAP/etc/nova/nova.conf. To override the defaults, a separate
    writable directory is used, and this directory is also unique to the snap.
    For example, either $SNAP_COMMON/etc/nova/nova.conf, or
    $SNAP_COMMON/etc/nova/nova.conf.d/ can be used to override config.

    This patch adds these snap directories to the default config paths where
    oslo looks for config.

    For more details on $SNAP and $SNAP_COMMON please refer to
    https://snapcraft.io/docs/reference/env.

    Change-Id: I83627e0f215382aedc7b32163e0303b39e8bccf8
    Closes-Bug: 1696830

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/oslo.config 4.6.0 release.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/476139

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/473615
Committed: https://git.openstack.org/cgit/openstack/snap-nova/commit/?id=f011198f5bd93d20fd37a124140d9addbf1a4c29
Submitter: Jenkins
Branch: master

commit f011198f5bd93d20fd37a124140d9addbf1a4c29
Author: Corey Bryant <email address hidden>
Date: Tue Jun 13 13:14:00 2017 +0000

    Enable nova-placement-api config and logging

    nova-placement-api is a bit tricky. The wsgi application doesn't accept
    CLI arguments, so we can't pass --config-file, --config-dir, or
    --log-file via uwsgi's pyargv. Additionally, nova-placement-api doesn't
    pass the [keystone_authtoken] from the config file found in
    OS_PLACEMENT_CONFIG_DIR to keystonemiddleware.

    We continue to use the OS_PLACEMENT_CONFIG_DIR environment variable to
    point at the default nova.conf in $SNAP/etc/nova.conf. This default
    nova.conf includes logdir=$SNAP_COMMON/log/, enabling nova-placement-api.log
    to be written to the correct directory.

    Passing [keystone_authtoken] to keystonemiddleware is fixed in oslo.config.
    The prior commit titled "Patch (_COMMON) dirs into oslo.config defaults"
    includes a patch for this. keystonemiddleware gets the [keystone_authtoken]
    config via oslo.config's find_config_files() and find_config_dirs(), which
    are patched to include the $SNAP and $SNAP_COMMON config directories.

    Change-Id: I02e2a5d9df91eb8cbf36beb474809bc1e80fe80b
    Closes-Bug: 1696830

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/476139
Committed: https://git.openstack.org/cgit/openstack/oslo.config/commit/?id=982cf476d0d7b6fcc96ecbefb1c2f70685cb84a7
Submitter: Jenkins
Branch: stable/ocata

commit 982cf476d0d7b6fcc96ecbefb1c2f70685cb84a7
Author: Corey Bryant <email address hidden>
Date: Wed Jun 14 09:57:23 2017 -0400

    Add snap package paths to default config dirs

    With snap packaging (see snapcraft.io) the package is installed into a
    read-only squashfs filesystem, which includes the default config. For
    example, $SNAP/etc/nova/nova.conf. To override the defaults, a separate
    writable directory is used, and this directory is also unique to the snap.
    For example, either $SNAP_COMMON/etc/nova/nova.conf, or
    $SNAP_COMMON/etc/nova/nova.conf.d/ can be used to override config.

    This patch adds these snap directories to the default config paths where
    oslo looks for config.

    For more details on $SNAP and $SNAP_COMMON please refer to
    https://snapcraft.io/docs/reference/env.

    Change-Id: I83627e0f215382aedc7b32163e0303b39e8bccf8
    Closes-Bug: 1696830
    (cherry picked from commit 21e70e28b3015c1619dacfad8a2218b09e8109ec)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/oslo.config 3.22.2 release.

[Chris Dent (cdent)]

See https://review.openstack.org/#/c/619049/ for a related change on config file handling for the placement-api.

[Chris Dent (cdent)]

After speaking with coreycb we're going to drop nova/placement from this as it no longer quite fits and the existing snap related bugs are sufficient to provide an aide-mémoire.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Eric Fried (<email address hidden>) on branch: master
Review: https://review.opendev.org/472378
Reason: Placement is deleted from nova. If this is still needed, please propose it to openstack/placement.