The local.conf and devstack.local.conf configuration files are being merged incorrectly, some values are lost when merging

[Castulo J. Martinez]

When Grenade runs, it uses a function fetch_devstacks to fetch the base and target Devstacks and their configuration files. As part of this process it merges several config files into one final local.conf file to be used for installing Devstack.
When these files are merging some config values are being overwritten/deleted messing up the devstack installation.

As an example, this is how my devstack.local.conf file looks like:

[[local|localrc]]
# Multihost configuration
MULTI_HOST=1
HOST_IP=10.0.1.74

# Replication of gate configuration. Below helps you get to a
# configuration that looks a lot like what we use in the gate
USE_SCREEN=False

# Neutron needs a lot of manual hand holding to get rolling, we
# specify a lot of config in the gate in neutron jobs related to
# networks.
Q_USE_DEBUG_COMMAND=True
NETWORK_GATEWAY=192.168.0.1
FIXED_RANGE=192.168.0.0/20
FLOATING_RANGE=172.24.5.0/24
PUBLIC_NETWORK_GATEWAY=172.24.5.1

# Services
# --------
# Pre-requisite
ENABLED_SERVICES=rabbit,mysql,key
# Nova
ENABLED_SERVICES+=,n-api,n-crt,n-obj,n-cpu,n-cond,n-sch
# Glance
ENABLED_SERVICES+=,g-api,g-reg
# Neutron
ENABLED_SERVICES+=,q-svc,q-agt,q-dhcp,q-l3,q-meta,neutron,q-ip-availability,q-metering
# Since Swift is being enabled in the devstack.localrc.base we need to disable it here
disable_service s-proxy s-object s-container s-account
# Tempest
enable_service tempest

And this is how the local.conf file ends up after merging with devstack.local.conf:

[[local|localrc]]
# localrc for DevStack base installation

# Default to selecting an interface via the default route
unset HOST_IP

DEST=/opt/stack/newton
DATA_DIR=/opt/stack/data
SWIFT_DATA_DIR=${DATA_DIR}/swift

# Timeouts
ACTIVE_TIMEOUT=120
ASSOCIATE_TIMEOUT=60
BOOT_TIMEOUT=120
SERVICE_TIMEOUT=120

ADMIN_PASSWORD=0penstack
API_RATE_LIMIT=False
LOGDAYS=1
LOGFILE=$DEST/logs/stack.sh.log
MYSQL_PASSWORD=superstack
RABBIT_HOST=localhost
RABBIT_PASSWORD=superstack
SCREEN_LOGDIR=$DEST/logs/screen
SCREEN_NAME=${SCREEN_NAME:-stack}
SERVICE_PASSWORD=$ADMIN_PASSWORD
SERVICE_TOKEN=cd0d1a03-b701-4fcb-801a-8b4d0bc3d06e
STACK_LOG=stack.sh.log
VOLUME_BACKING_FILE_SIZE=10000M

enable_service s-account s-container s-object s-proxy
SWIFT_HASH=8213897fads879789asdf789
SWIFT_REPLICAS=1
MULTI_HOST=1
HOST_IP=10.0.1.74
USE_SCREEN=False
Q_USE_DEBUG_COMMAND=True
NETWORK_GATEWAY=192.168.0.1
FIXED_RANGE=192.168.0.0/20
FLOATING_RANGE=172.24.5.0/24
PUBLIC_NETWORK_GATEWAY=172.24.5.1
ENABLED_SERVICES=rabbit,mysql,key
ENABLED_SERVICES+=,q-svc,q-agt,q-dhcp,q-l3,q-meta,neutron,q-ip-availability,q-metering
disable_service s-proxy s-object s-container s-account
enable_service tempest

So these two lines were deleted or overwritten probably during the merge process, and they should not have been:

ENABLED_SERVICES+=,n-api,n-crt,n-obj,n-cpu,n-cond,n-sch
ENABLED_SERVICES+=,g-api,g-reg

This bug was introduced with this patch:
https://review.openstack.org/#/c/421302/

[Castulo J. Martinez (castulo-martinez)]

The problem is not related specifically to Grenade, but to a project Grenade is using: devstack-tools. I've submitted a patch to fix this behavior: https://review.openstack.org/#/c/428412/