Defect: Assembly stuck in BUILDING state using Devstack+Neutron

[Adrian Otto]

I am using Devstack with Docker installed on Centos 6.5 on a RAX cloud server (8GB performance).

My plan file (vanilla ex1):

[devstack@devstack ~]$ cat ex1.yaml
name: ex1
description: Nodejs express.
artifacts:
- name: nodeus
  artifact_type: application.heroku
  content:
    href: https://github.com/paulczar/example-nodejs-express.git

My solum resources:

devstack@devstack ~]$ solum app list
+--------------------------------------+------+-----------------+
| uuid | name | description |
+--------------------------------------+------+-----------------+
| 6d13069f-b51f-46b3-8021-6ce8af2ffd1e | ex1 | Nodejs express. |
+--------------------------------------+------+-----------------+
[devstack@devstack ~]$ solum assembly list
+--------------------------------------+------+-------------+----------+
| uuid | name | description | status |
+--------------------------------------+------+-------------+----------+
| e7957f3f-1bee-47c4-be39-bc903cf40187 | ex1 | None | BUILDING |
+--------------------------------------+------+-------------+----------+
[devstack@devstack ~]$ solum assembly show e7957f3f-1bee-47c4-be39-bc903cf40187
+-----------------+----------------------------------------------------------------------+
| Property | Value |
+-----------------+----------------------------------------------------------------------+
| status | BUILDING |
| description | None |
| application_uri | None |
| name | ex1 |
| trigger_uri | http://23.253.72.14:9777/v1/public/triggers/f70c391a-1ddf-4f80-a06e- |
| | fa76b74cd32a |
| uuid | e7957f3f-1bee-47c4-be39-bc903cf40187 |
+-----------------+----------------------------------------------------------------------+

Expected Result: The devstack system builds the app, makes an image, stores it in glance, etc.
Actual Result: The build process never completes, so no build image is created.
Steps to Reproduce:

cd ${HOME}
git clone git://git.openstack.org/stackforge/solum
SOLUM_DIR="${HOME}/solum"
cat << EOF > SOLUM_DIR/contrib/devstack/localrc
#
# Default ${DEVSTACK_DIR}/localrc file for Solum
#

disable_service n-net
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
enable_service neutron
enable_service solum
enable_service solum-api
enable_service solum-build-api
enable_service solum-conductor
enable_service solum-deployer
enable_service solum-worker

ENABLED_SERVICES+=,heat,h-api,h-api-cfn,h-api-cw,h-eng
SOLUM_INSTALL_CEDARISH=True
FLOATING_RANGE=192.168.15.0/27
FLAT_INTERFACE=eth0
Q_FLOATING_ALLOCATION_POOL=start=192.168.15.10,end=192.168.15.30
PUBLIC_NETWORK_GATEWAY=192.168.15.1
DATABASE_PASSWORD=solum
RABBIT_PASSWORD=solum
SERVICE_TOKEN=solum
SERVICE_PASSWORD=solum
EOF

cd /path/to/devstack
./unstack.sh
SOLUM_DIR="${HOME}/solum"
DEVSTACK_DIR="${HOME}/devstack"
echo "Getting Solum Code"
cd ${HOME}
#git clone git://git.openstack.org/stackforge/solum
echo "Getting Devstack Code"
git clone https://github.com/openstack-dev/devstack.git
echo "Setting up Devstack for Solum"
cd ~/solum/contrib/devstack
cp lib/solum ${DEVSTACK_DIR}/lib
cp extras.d/70-solum.sh ${DEVSTACK_DIR}/extras.d
cp localrc ${DEVSTACK_DIR}
echo "Starting Devstack"
cd ${DEVSTACK_DIR} && ./stack.sh

wget https://raw.githubusercontent.com/stackforge/solum/master/examples/plans/ex1.yaml
solum app create ex1.yaml
solum assembly create <uuid_of_app>

[Adrian Otto (aotto)]

@paulczar indicated he has the same problem.

[Paul Czarkowski (paulcz)]

output from solum-deployer

{"explanation": "The server could not comply with the request since it is either malformed or otherwise incorrect.", "code": 400, "error": {"message": "The Parameter (private_subnet) was not provided.", "traceback": "Traceback (most recent call last):\n\n File \"/opt/stack/heat/heat/engine/service.py\", line 62, in wrapped\n return func(self, ctx, *args, **kwargs)\n\n File \"/opt/stack/heat/heat/engine/service.py\", line 538, in create_stack\n env, **common_params)\n\n File \"/opt/stack/heat/heat/engine/parser.py\", line 105, in __init__\n context=context)\n\n File \"/opt/stack/heat/heat/engine/hot/template.py\", line 143, in parameters\n context=context)\n\n File \"/opt/stack/heat/heat/engine/parameters.py\", line 394, in __init__\n user_parameters))\n\n File \"/opt/stack/heat/heat/engine/parameters.py\", line 392, in <genexpr>\n self.params = dict((p.name,\n\n File \"/opt/stack/heat/heat/engine/parameters.py\", line 389, in <genexpr>\n user_parameters = (user_parameter(si) for si in schemata.iteritems())\n\n File \"/opt/stack/heat/heat/engine/parameters.py\", line 382, in user_parameter\n validate_value)\n\n File \"/opt/stack/heat/heat/engine/parameters.py\", line 196, in __new__\n return ParamClass(name, schema, value, validate_value)\n\n File \"/opt/stack/heat/heat/engine/parameters.py\", line 213, in __init__\n raise exception.UserParameterMissing(key=self.name)\n\nUserParameterMissing: The Parameter (private_subnet) was not provided.\n", "type": "UserParameterMissing"}, "title": "Bad Request"}
 log_http_response /opt/stack/python-heatclient/heatclient/common/http.py:122
2014-05-02 17:57:51.149 15005 ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling: ERROR: The Parameter (private_subnet) was not provided.
2014-05-02 17:57:51.149 15005 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
2014-05-02 17:57:51.149 15005 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply
2014-05-02 17:57:51.149 15005 TRACE oslo.messaging.rpc.dispatcher incoming.message))
2014-05-02 17:57:51.149 15005 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 176, in _dispatch
2014-05-02 17:57:51.149 15005 TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args)
2014-05-02 17:57:51.149 15005 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 122, in _do_dispatch
2014-05-02 17:57:51.149 15005 TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint, method)(ctxt, **new_args)
2014-05-02 17:57:51.149 15005 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/solum/solum/deployer/handlers/heat.py", line 111, in deploy
2014-05-02 17:57:51.149 15005 TRACE oslo.messaging.rpc.dispatcher parameters=parameters)
2014-05-02 17:57:51.149 15005 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/python-heatclient/heatclient/v1/stacks.py", line 119, in create
2014-05-02 17:57:...

[Paul Czarkowski (paulcz)]

I get these errors in both vm-slug and docker modes.

my localrc for vms is:

LOGFILE=/opt/stack/logs/stack.sh.log
FLAT_INTERFACE=eth1
PUBLIC_INTERFACE=eth1
DATABASE_PASSWORD=solum
RABBIT_PASSWORD=solum
SERVICE_TOKEN=solum
SERVICE_PASSWORD=solum
ADMIN_PASSWORD=solum
NOVNC_FROM_PACKAGE=false
SCREEN_LOGDIR=/opt/stack/logs/screen
SOLUM_INSTALL_CEDARISH=True
ENABLED_SERVICES+=,heat,h-api,h-api-cfn,h-api-cw,h-eng
IMAGE_URLS=http://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-uec.tar.gz,http://cc42a68525a2dda0151c-9a7653a0ba84bd9342f239dc5349667e.r38.cf1.rackcdn.com/cedarish.qcow2

enable_service solum
enable_service solum-api
enable_service solum-conductor
enable_service solum-deployer
enable_service solum-worker

disable_service n-net
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
enable_service neutron

Q_FLOATING_ALLOCATION_POOL=start=192.168.76.50,end=192.168.76.70
FLOATING_RANGE=192.168.76.0/24
PUBLIC_NETWORK_GATEWAY=192.168.76.11

[Adrian Otto (aotto)]

I got this symptom again today using the M2 Demo setup:

https://wiki.openstack.org/wiki/Solum/solum_m2_demo

Murali also had this issue on his M2 demo system.

[Angus Salkeld (asalkeld)]

this fixes the bug for me: https://review.openstack.org/#/c/92171/

[Openstack Gerrit (openstack-gerrit)]

Reviewed: https://review.openstack.org/92171
Committed: https://git.openstack.org/cgit/stackforge/solum/commit/?id=3994e65322d82ff2cd9ed91f08607e8c7eb6ae44
Submitter: Jenkins
Branch: master

commit 3994e65322d82ff2cd9ed91f08607e8c7eb6ae44
Author: Angus Salkeld <email address hidden>
Date: Tue May 6 10:57:57 2014 +1000

    Update status and reason in conductor method.

    The fields status and reason did not reflect actual
    attributes of Image objects. Changing these to state
    and description to fix this so that actual fields
    are updated.

    Fixes-bug: #1315459
    Change-Id: I0b57871a6bcf4ef35fe3d60fca552e41aa3e78f1

[rohit mathur (rohit-mathur1)]

I am still facing the same issue. Can anyone post the solution?

[rohit mathur (rohit-mathur1)]

hi guys,
I am new to Solum and installing it first time on Ubuntu 12.04.

I have followed the steps mentioned below for installation

git clone git://git.openstack.org/stackforge/solum
git clone https://github.com/openstack-dev/devstack.git
SOLUM_DIR="${HOME}/solum"
DEVSTACK_DIR="${HOME}/devstack"
cd ~/solum/contrib/devstack
cp lib/solum ${DEVSTACK_DIR}/lib
cp extras.d/70-solum.sh ${DEVSTACK_DIR}/extras.d
cp localrc ${DEVSTACK_DIR}
echo "Starting Devstack"
cd ${DEVSTACK_DIR}
./stack.sh

Following the steps, Solum gets installed properly but when i try to create an assembly my dot net application then its gets stuck in the Build stage and nothing happens.

Can anyone tell me the steps to debug this?

[Adrian Otto (aotto)]

Rohit,

Thanks for your comment. I have opened a separate bug ticket for your comment:

https://bugs.launchpad.net/solum/+bug/1319406

We will respond to that concern there.

Thanks,

Adrian