Comment 0 for bug 1529691

Revision history for this message
Artem Hrechanychenko (agrechanichenko) wrote : Stop deployment while adding node to existing clusterleads to the destruction of the cluster

ISO # 361

Steps to reproduce :
1. Deploy any environment with 3 controllers, 1 compute and NeutronTUN or NeutronVLAN
2. Add 1 compute to the cluster
3. Deploy changes
4. Stop deployment at the middle of re-deployment of non-primary controllers
5. Run OSTF
6. Deploy changes once again
7. Run OSTF

Actual result:
After stopped deployment process - non primary controller's are deleted and new compute - become "ready".

From astute log:
2015-12-28 18:15:04 INFO [847] Replace running task 56f35e3d-7018-40d3-989d-0f9fbaa674f6 to new 644c1467-89ee-4b2d-bd6a-6d97c4625ebe with data:
{"args"=>
  {"engine"=>
    {"url"=>"http://10.109.5.2:80/cobbler_api",
     "username"=>"cobbler",
     "password"=>"rGKTvfux",
     "master_ip"=>"10.109.5.2"},
   "nodes"=>
    [{"admin_ip"=>"10.109.5.10",
      "uid"=>"3",
      "roles"=>["controller"],
      "slave_name"=>"node-3"},
     {"admin_ip"=>"10.109.5.6",
      "uid"=>"2",
      "roles"=>["controller"],
      "slave_name"=>"node-2"}],
   "stop_task_uuid"=>"56f35e3d-7018-40d3-989d-0f9fbaa674f6",
   "task_uuid"=>"644c1467-89ee-4b2d-bd6a-6d97c4625ebe"},
 "respond_to"=>"stop_deployment_resp",
 "method"=>"remove_nodes",
 "api_version"=>"1"}
2015-12-28 18:15:04 INFO [847] Processing RPC call 'stop_deploy_task'

There is no way to run OSTF via WEB-UI, only via CLI with --force flag

[root@nailgun ~]# fuel --env 1 health --check smoke --force
[ 1 of 12] [failure] 'Create instance flavor' (0.0 s) Can not set proxy for Health Check.Make sure that network configuration for controllers is correct
[ 2 of 12] [failure] 'Check create, update and delete image actions using Glance v2' (0.0 s) Can not set proxy for Health Check.Make sure that network configuration for controllers is correct
[ 3 of 12] [failure] 'Create volume and boot instance from it' (0.0 s) Can not set proxy for Health Check.Make sure that network configuration for controllers is correct
[ 4 of 12] [failure] 'Create volume and attach it to instance' (0.0 s) Can not set proxy for Health Check.Make sure that network configuration for controllers is correct
[ 5 of 12] [failure] 'Check network connectivity from instance via floating IP' (0.0 s) Can not set proxy for Health Check.Make sure that network configuration for controllers is correct
[ 6 of 12] [failure] 'Create keypair' (0.0 s) Can not set proxy for Health Check.Make sure that network configuration for controllers is correct
[ 7 of 12] [failure] 'Create security group' (0.0 s) Can not set proxy for Health Check.Make sure that network configuration for controllers is correct
[ 8 of 12] [failure] 'Check network parameters' (0.0 s) Can not set proxy for Health Check.Make sure that network configuration for controllers is correct
[ 9 of 12] [failure] 'Launch instance' (0.0 s) Can not set proxy for Health Check.Make sure that network configuration for controllers is correct
[10 of 12] [failure] 'Launch instance with file injection' (0.0 s) Can not set proxy for Health Check.Make sure that network configuration for controllers is correct
[11 of 12] [failure] 'Launch instance, create snapshot, launch instance from snapshot' (0.0 s) Can not set proxy for Health Check.Make sure that network configuration for controllers is correct
[12 of 12] [failure] 'Create user and authenticate with it.' (0.0 s) Can not set proxy for Health Check.Make sure that network configuration for controllers is correct

Keystone service - is completely dead on primary controller
root@node-1:~# source openrc
root@node-1:~# keystone catalog
/usr/lib/python2.7/dist-packages/keystoneclient/shell.py:64: DeprecationWarning: The keystone CLI is deprecated in favor of python-openstackclient. For a Python library, continue using python-keystoneclient.
  'python-keystoneclient.', DeprecationWarning)
/usr/lib/python2.7/dist-packages/keystoneclient/v2_0/client.py:145: DeprecationWarning: Constructing an instance of the keystoneclient.v2_0.client.Client class without a session is deprecated as of the 1.7.0 release and may be removed in the 2.0.0 release.
  'the 2.0.0 release.', DeprecationWarning)
/usr/lib/python2.7/dist-packages/keystoneclient/v2_0/client.py:147: DeprecationWarning: Using the 'tenant_name' argument is deprecated in version '1.7.0' and will be removed in version '2.0.0', please use the 'project_name' argument instead
  super(Client, self).__init__(**kwargs)
/usr/lib/python2.7/dist-packages/debtcollector/renames.py:43: DeprecationWarning: Using the 'tenant_id' argument is deprecated in version '1.7.0' and will be removed in version '2.0.0', please use the 'project_id' argument instead
  return f(*args, **kwargs)
/usr/lib/python2.7/dist-packages/keystoneclient/httpclient.py:376: DeprecationWarning: Constructing an HTTPClient instance without using a session is deprecated as of the 1.7.0 release and may be removed in the 2.0.0 release.
  'the 2.0.0 release.', DeprecationWarning)
Authorization Failed: Unable to establish connection to http://10.109.7.3:5000/v2.0/tokens
root@node-1:~# glance image-list
Unable to establish connection to http://10.109.7.3:5000/v2.0/tokens