Override of DEST in local.conf not honoured for SERVICE_DIR / DATA_DIR

[Daniel Berrange]

The online docs http://devstack.org/configuration.html helpfully illustrate setting DEST=/some/dir in local.conf to stop the default /opt/stack being used.

Unfortunately it seems that DATA_DIR and SERVICE_DIR in stackrc will substitute in the value of $DEST *before* your local.conf settings are loaded. So you end up with

 DEST=/some/dir
 DATA_DIR=/opt/stack/data
 SERVICE_DIR=/opt/stack/status

devstack will then mysteriously fail later because the /opt/stack hierarchy won't have been created.

Either DATA_DIR & SERVICE_DIR need to honour the local.conf $DEST value, or we need to sanity-check the user's settings and warn the user if they have set DEST but not also set DATA_DIR & SERVICE_DIR in their local.conf.

[Ian Wienand (iwienand)]

Hi Daniel,

I'm not seeing this.

stackrc should set the default then source localrc/local.conf to override that [1]. Is it possible you had an env variable exported or similar?

[1] https://github.com/openstack-dev/devstack/blob/master/stackrc

[Kashyap Chamarthy (kashyapc)]

I think I noticed what Dan described on a Fedora 20 Devstack environment with the below localrc (later on I changed the DATA_DIR path, though):

   $ git log | head -1
    83f7736dcf9039d9114dd4568687f28221bac5d9

    $ cat localrc
    DESTDIR=$HOME/src/openstack

    Q_USE_DEBUG_COMMAND=True
    NETWORK_GATEWAY=10.1.0.1

    ACTIVE_TIMEOUT=90
    BOOT_TIMEOUT=90
    ASSOCIATE_TIMEOUT=60
    TERMINATE_TIMEOUT=60
    MYSQL_PASSWORD=secret
    DATABASE_PASSWORD=secret
    RABBIT_PASSWORD=secret
    ADMIN_PASSWORD=secret
    SERVICE_PASSWORD=secret
    SERVICE_TOKEN=111222333444
    #SWIFT_HASH=1234123412341234
    ROOTSLEEP=0
    ERROR_ON_CLONE=False
    ENABLED_SERVICES=g-api,g-reg,key,n-api,n-crt,n-cpu,n-sch,mysql,qpid,sysstat,quantum,q-svc,q-agt,q-dhcp,q-l3,q-meta
    SKIP_EXERCISES=boot_from_volume,bundle,client-env,euca
    SERVICE_HOST=127.0.0.1
    # Screen console logs will capture service logs.
    SYSLOG=False
    SCREEN_LOGDIR=$DATA_DIR/new/screen-logs
    LOGFILE=/$DATA_DIR/new/devstacklog.txt
    VERBOSE=True
    FIXED_RANGE=10.1.0.0/24
    FIXED_NETWORK_SIZE=256
    VIRT_DRIVER=libvirt
    #SWIFT_REPLICAS=1
    LOG_COLOR=False
    PIP_USE_MIRRORS=False
    USE_GET_PIP=1
    # Don't reset the requirements.txt files after g-r updates
    UNDO_REQUIREMENTS=False
    #CINDER_PERIODIC_INTERVAL=10
    export OS_NO_CACHE=True
    #CINDER_SECURE_DELETE=False
    API_RATE_LIMIT=False
    VOLUME_BACKING_FILE_SIZE=24G
    #TEMPEST_HTTP_IMAGE=http://127.0.0.1/
    USE_SCREEN=False
    FORCE_CONFIG_DRIVE=always

    IMAGE_URLS="http://download.cirros-cloud.net/0.3.1/cirros-0.3.1-x86_64-uec.tar.gz"

[Daniel Berrange (berrange)]

By code inspection alone I don't see how stackrc can possibly handle this correctly

On line 4, 10 and 13 the vars are set to default values:

DEST=/opt/stack
DATA_DIR=${DEST}/data
SERVICE_DIR=${DEST}/status

Now on line 70

# allow local overrides of env variables, including repo config
if [[ -f $RC_DIR/localrc ]]; then
    # Old-style user-supplied config
    source $RC_DIR/localrc
elif [[ -f $RC_DIR/.localrc.auto ]]; then
    # New-style user-supplied config extracted from local.conf
    source $RC_DIR/.localrc.auto
fi

How can sourcing a new value for $DEST at line 70, possibly change the value of DATA_DIR / SERVICE_DIR which were set on line 10/13.

A quick debug addition to the end of stackrc demonstrates this pretty clearly

$ git diff
diff --git a/stackrc b/stackrc
index 49fb26b..2ddcf21 100644
--- a/stackrc
+++ b/stackrc
@@ -321,6 +321,11 @@ INSTALL_TESTONLY_PACKAGES=${INSTALL_TESTONLY_PACKAGES:-False}
 # Undo requirements changes by global requirements
 UNDO_REQUIREMENTS=${UNDO_REQUIREMENTS:-True}

+echo "$DEST"
+echo "$SERVICE_DIR"
+echo "$DATA_DIR"
+exit
+
 # Local variables:
 # mode: shell-script
 # End:

$ cat local.conf
[[local|localrc]]
DEST=/home/berrange

$ ./stack.sh
/home/berrange
/opt/stack/status
/opt/stack/data

Those last two should have either been /home/berrange/status & /home/berrange/data, or stack.sh should have warned that setting $DEST on its own doesn't work & you must set all 3 vars at once.

[Pavel Chekin (pchekin)]

Because DEST, DATA_DIR and SERVICE_DIR are set in stackrc before processing local.conf (or or localrc if it exists), changing DEST only does not affect DATA_DIR and SERVICE_DIR. Also these variables may be referenced in local.conf even if DEST remains unchanged.

As a solution, the documentation can be updated by the advice to change both DATA_DIR and SERVICE_DIR variables, if DEST is changed from the default value.

[Pavlo Shchelokovskyy (pshchelo)]

Reviving this old discussion, I still see this on fresh master of DevStack.

Additionally to the above mentioned SERVICE_DIR and DATA_DIR, now I also have to explicitly set SUBUNIT_OUTPUT too (not very critical, w/o it if there are no errors while deploying itself, everything is deployed fine except the error to create the devstack.subunit file, and stack.sh returns non-0.)

As a minimal reproducer, I have a folder /home/<my-user-name>/stack/ where I've cloned DevStack as 'devstack' subfolder, and want all the components installed in /home/<my-user-name>/stack/ too.

example superminimal Keystone-only install that does the wrong things and fails:

$ pwd
/home/pshchelo/stack

$ ls
devstack

$ ls -R /opt
/opt/:

$ # super-minimal config, Keystone only

$ cd devstack && cat local.conf
[[local|localrc]]
DEST=/home/pshchelo/stack
ADMIN_PASSWORD=secret
MYSQL_PASSWORD=secret
RABBIT_PASSWORD=secret
SERVICE_PASSWORD=secret
disable_all_services
enable_service mysql key

$ ./stack.sh

.....

+functions-common:init_service_check:1455 SCREEN_NAME=stack
+functions-common:init_service_check:1456 SERVICE_DIR=/opt/stack/status
+functions-common:init_service_check:1458 [[ ! -d /opt/stack/status/stack ]]
+functions-common:init_service_check:1459 mkdir -p /opt/stack/status/stack
mkdir: cannot create directory '/opt/stack/status': Permission denied
+functions-common:init_service_check:1 exit_trap
+./stack.sh:exit_trap:480 local r=1
++./stack.sh:exit_trap:481 jobs -p
+./stack.sh:exit_trap:481 jobs=
+./stack.sh:exit_trap:484 [[ -n '' ]]
+./stack.sh:exit_trap:490 kill_spinner
+./stack.sh:kill_spinner:376 '[' '!' -z '' ']'
+./stack.sh:exit_trap:492 [[ 1 -ne 0 ]]
+./stack.sh:exit_trap:493 echo 'Error on exit'
Error on exit
+./stack.sh:exit_trap:494 generate-subunit 1471168810 70 fail
./stack.sh: line 494: /opt/stack/devstack.subunit: Permission denied

$ ls -R /opt/
/opt/:
stack

/opt/stack:
data

/opt/stack/data:

$

Note the SERVICE_DIR and location of subunit file, they were not affected by me setting only DEST in local.conf.
Also, a folder /opt/stack/data is created, instead of '$DEST/data'

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/355234

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/355234
Committed: https://git.openstack.org/cgit/openstack-dev/devstack/commit/?id=b08b673def8f5c80b7a91ba2127edbb13c39c26a
Submitter: Jenkins
Branch: master

commit b08b673def8f5c80b7a91ba2127edbb13c39c26a
Author: Pavlo Shchelokovskyy <email address hidden>
Date: Sun Aug 14 13:26:13 2016 +0300

    Allow properly overriding DEST

    Change the order of variable declarations in stackrc so that setting
    custom DEST in local.conf is also affecting DATA_DIR, SERVICE_DIR and
    SUBUNIT_OUTPUT.

    Change-Id: I00847bb6733febf105855ae6fc577a7c904ec4b4
    Closes-Bug: #1285720

[Ken'ichi Ohmichi (oomichi)]

This patch made testr_result.html disappear on the gate tests, so we revert this.

[Pavlo Shchelokovskyy (pshchelo)]

please change the status of this bug back to confirmed as the fix was reverted