merge_config_file fail without core components

[Toshiaki Takahashi]

We created separated OpenStack controller node which include some component (actually, Tacker) but not include core component (Nova, Neutron...).
In this case, post-config setting leads to the following error.

=====================================================================
2020-12-20 16:56:56.687 | + inc/meta-config:merge_config_file:97 : awk -v 'configfile=$TACKER_CONF' '
2020-12-20 16:56:56.687 | BEGIN {
2020-12-20 16:56:56.687 | section = ""
2020-12-20 16:56:56.687 | last_section = ""
2020-12-20 16:56:56.687 | section_count = 0
2020-12-20 16:56:56.687 | }
2020-12-20 16:56:56.687 | /^\[.+\]/ {
2020-12-20 16:56:56.687 | gsub("[][]", "", $1);
2020-12-20 16:56:56.687 | section=$1
2020-12-20 16:56:56.687 | next
2020-12-20 16:56:56.687 | }
2020-12-20 16:56:56.687 | /^ *\#/ {
2020-12-20 16:56:56.687 | next
2020-12-20 16:56:56.687 | }
2020-12-20 16:56:56.687 | /^[^ \t]+/ {
2020-12-20 16:56:56.687 | # get offset of first = in $0
2020-12-20 16:56:56.687 | eq_idx = index($0, "=")
2020-12-20 16:56:56.687 | # extract attr & value from $0
2020-12-20 16:56:56.687 | attr = substr($0, 1, eq_idx - 1)
2020-12-20 16:56:56.687 | value = substr($0, eq_idx + 1)
2020-12-20 16:56:56.688 | # only need to strip trailing whitespace from attr
2020-12-20 16:56:56.688 | sub(/[ \t]*$/, "", attr)
2020-12-20 16:56:56.688 | # need to strip leading & trailing whitespace from value
2020-12-20 16:56:56.688 | sub(/^[ \t]*/, "", value)
2020-12-20 16:56:56.688 | sub(/[ \t]*$/, "", value)
2020-12-20 16:56:56.688 |
2020-12-20 16:56:56.688 | # cfg_attr_count: number of config lines per [section, attr]
2020-12-20 16:56:56.688 | # cfg_attr: three dimensional array to keep all the config lines per [section, attr]
2020-12-20 16:56:56.688 | # cfg_section: keep the section names in the same order as they appear in local.conf
2020-12-20 16:56:56.688 | # cfg_sec_attr_name: keep the attr names in the same order as they appear in local.conf
2020-12-20 16:56:56.688 | if (! (section, attr) in cfg_attr_count) {
2020-12-20 16:56:56.688 | if (section != last_section) {
2020-12-20 16:56:56.688 | cfg_section[section_count++] = section
2020-12-20 16:56:56.688 | last_section = section
2020-12-20 16:56:56.688 | }
2020-12-20 16:56:56.688 | attr_count = cfg_sec_attr_count[section_count - 1]++
2020-12-20 16:56:56.688 | cfg_sec_attr_name[section_count - 1, attr_count] = attr
2020-12-20 16:56:56.688 |
2020-12-20 16:56:56.688 | cfg_attr[section, attr, 0] = value
2020-12-20 16:56:56.688 | cfg_attr_count[section, attr] = 1
2020-12-20 16:56:56.688 | } else {
2020-12-20 16:56:56.688 | lno = cfg_attr_count[section, attr]++
2020-12-20 16:56:56.688 | cfg_attr[section, attr, lno] = value
2020-12-20 16:56:56.688 | }
2020-12-20 16:56:56.688 | }
2020-12-20 16:56:56.688 | END {
2020-12-20 16:56:56.688 | # Process each section in order
2020-12-20 16:56:56.688 | for (sno = 0; sno < section_count; sno++) {
2020-12-20 16:56:56.688 | section = cfg_section[sno]
2020-12-20 16:56:56.688 | # The ini routines simply append a config item immediately
2020-12-20 16:56:56.688 | # after the section header. To keep the same order as defined
2020-12-20 16:56:56.688 | # in local.conf, invoke the ini routines in the reverse order
2020-12-20 16:56:56.688 | for (attr_no = cfg_sec_attr_count[sno] - 1; attr_no >=0; attr_no--) {
2020-12-20 16:56:56.688 | attr = cfg_sec_attr_name[sno, attr_no]
2020-12-20 16:56:56.688 | if (cfg_attr_count[section, attr] == 1)
2020-12-20 16:56:56.688 | print "iniset " configfile " " section " " attr " \"" cfg_attr[section, attr, 0] "\""
2020-12-20 16:56:56.688 | else {
2020-12-20 16:56:56.688 | # For multiline, invoke the ini routines in the reverse order
2020-12-20 16:56:56.688 | count = cfg_attr_count[section, attr]
2020-12-20 16:56:56.689 | print "inidelete " configfile " " section " " attr
2020-12-20 16:56:56.689 | print "iniset " configfile " " section " " attr " \"" cfg_attr[section, attr, count - 1] "\""
2020-12-20 16:56:56.689 | for (l = count -2; l >= 0; l--)
2020-12-20 16:56:56.689 | print "iniadd_literal " configfile " " section " " attr " \"" cfg_attr[section, attr, l] "\""
2020-12-20 16:56:56.689 | }
2020-12-20 16:56:56.689 | }
2020-12-20 16:56:56.689 | }
2020-12-20 16:56:56.689 | }
2020-12-20 16:56:56.689 | '
2020-12-20 16:56:56.689 | + inc/meta-config:merge_config_file:164 : read a
2020-12-20 16:56:56.689 | awk: line 31: syntax error at or near ,
2020-12-20 16:56:56.689 | awk: line 41: syntax error at or near }
2020-12-20 16:56:56.689 | awk: line 45: syntax error at or near }
=====================================================================

The following is local.conf.

=====================================================================
[[local|localrc]]
disable_all_services
enable_service q-agt
enable_service tacker
enable_service tacker-conductor
ADMIN_PASSWORD="secretadmin"
CELLSV2_SETUP="singleconductor"
DATABASE_HOST="158.69.65.183"
DATABASE_PASSWORD="secretdatabase"
DATABASE_TYPE="mysql"
ERROR_ON_CLONE="True"
FIXED_RANGE="10.1.0.0/20"
FLOATING_RANGE="172.24.5.0/24"
GLANCE_HOSTPORT="158.69.65.183:9292"
HOST_IP="158.69.70.4"
IPV4_ADDRS_SAFE_TO_USE="10.1.0.0/20"
IS_ZUUL_FT="True"
KEYSTONE_SERVICE_HOST="158.69.65.183"
LIBVIRT_TYPE="qemu"
LOGFILE="/opt/stack/logs/devstacklog.txt"
LOG_COLOR="False"
MYSQL_HOST="158.69.65.183"
NETWORK_GATEWAY="10.1.0.1"
NOVA_VNC_ENABLED="True"
NOVNC_FROM_PACKAGE="True"
OVS_BRIDGE_MAPPINGS="public:br-ex,mgmtphysnet0:br-infra"
PHYSICAL_NETWORK="mgmtphysnet0"
PUBLIC_BRIDGE_MTU="1430"
PUBLIC_NETWORK_GATEWAY="172.24.5.1"
Q_DVR_MODE="dvr"
Q_HOST="158.69.65.183"
RABBIT_HOST="158.69.65.183"
RABBIT_PASSWORD="secretrabbit"
SERVICE_HOST="158.69.65.183"
SERVICE_PASSWORD="secretservice"
TACKER_HOST="158.69.70.4"
TACKER_MODE="standalone"
VERBOSE="True"
VERBOSE_NO_TIMESTAMP="True"
LIBS_FROM_GIT=aodh,barbican,blazar,blazar-nova,cinder,devstack,glance,heat,horizon,keystone,mistral,mistral-dashboard,networking-sfc,neutron,nova,placement,python-barbicanclient,python-blazarclient,python-mistralclient,python-tackerclient,requirements,swift,tacker,tacker-horizon,fenix
enable_plugin tacker https://opendev.org/openstack/tacker

[[post-config|$TACKER_CONF]]
[connect_grant]
base_url = http://127.0.0.1:9990/grant/v1/grants

[connect_vnf_packages]
base_url = http://127.0.0.1:9990/vnfpkgm/v1/vnf_packages
pipeline = package_content,vnfd,artifacts
=====================================================================

My guess is that this is because gawk is not installed in the environment, which seems Nova's requirement.

[Toshiaki Takahashi (takahashi-tsc)]

I proposed a patch for fix.

https://review.opendev.org/c/openstack/devstack/+/768363

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/devstack/+/768363
Committed: https://opendev.org/openstack/devstack/commit/d207ba9015f3210812468bfbf7d06b1491392554
Submitter: "Zuul (22348)"
Branch: master

commit d207ba9015f3210812468bfbf7d06b1491392554
Author: Toshiaki Takahashi <email address hidden>
Date: Wed Dec 23 17:40:57 2020 +0000

    Move gawk into general for post-config

    Devstack script for setting post-config needs gawk.
    So this patch moves gawk from files/*/nova into files/*/general.

    Closes-Bug: #1909041
    Change-Id: I06a1a5524f146a8d7337963e846b5a6b7561be13

[Martin Kopec (mkopec)]

https://review.opendev.org/c/openstack/devstack/+/768363 got merged