deployment cluster with enabled ssl on master node failed with failed tasks: Task[top-role-cinder/6]

[Artem Hrechanychenko]

Detailed bug description:

16-05-24 14:28:48 DEBUG [30547] Node 6 has failed to deploy. There is no more retries for puppet run.
2016-05-24 14:28:48 DEBUG [30547] Node 6(top-role-cinder) status: error
2016-05-24 14:28:48 DEBUG [30547] 98173ec2-680a-4b8c-b09c-e98f13b6f15c: MC agent 'puppetd', method 'last_run_summary', results:
{:sender=>"6",
 :statuscode=>0,
 :statusmsg=>"OK",
 :data=>
  {:time=>
    {"anchor"=>0.000109392,
     "cinder_config"=>0.8466862199999999,
     "config_retrieval"=>2.601160097,
     "exec"=>0.020808378,
     "file"=>0.004546865,
     "filebucket"=>0.000153289,
     "package"=>217.16670322800002,
     "schedule"=>0.000821023,
     "total"=>220.64098849200002,
     "last_run"=>1464100127},
   :resources=>
    {"changed_resources"=>
      "Package[cinder],File[create-policy-rc.d],Package[tgt],Package[cinder-backup],Package[cinder-volume],Exec[remove-policy-rc.d],Package[python-mysqldb]",
     "failed_resources"=>
      "Cinder_config[DEFAULT/host],Cinder_config[oslo_messaging_rabbit/rabbit_ha_queues],Cinder_config[oslo_messaging_rabbit/amqp_durable_queues],Cinder_config[DEFAULT/enable_v2_api],Cinder_config[oslo_messaging_rabbit/heartbeat_rate],Cinder_config[DEFAULT/backup_swift_auth_url],Cinder_config[DEFAULT/backup_swift_container],Cinder_config[DEFAULT/use_syslog],Cinder_config[DEFAULT/api_paste_config],Cinder_config[DEFAULT/enable_v3_api],Cinder_config[DEFAULT/enabled_backends],Cinder_config[DEFAULT/backup_driver],Cinder_config[DEFAULT/use_stderr],Cinder_config[DEFAULT/debug],Cinder_config[DEFAULT/storage_availability_zone],Cinder_config[DEFAULT/syslog_log_facility],Cinder_config[oslo_messaging_rabbit/rabbit_use_ssl],Cinder_config[DEFAULT/log_dir],Cinder_config[DEFAULT/glance_api_servers],Cinder_config[oslo_messaging_rabbit/rabbit_hosts],Cinder_config[oslo_concurrency/lock_path],Cinder_config[DEFAULT/default_availability_zone],Cinder_config[DEFAULT/verbose],Cinder_config[DEFAULT/control_exchange],Cinder_config[LVM-backend/volume_backend_name],Cinder_config[LVM-backend/volumes_dir],Cinder_config[DEFAULT/auth_strategy],Cinder_config[LVM-backend/iscsi_helper],Cinder_config[keymgr/fixed_key],Cinder_config[LVM-backend/volume_group],Cinder_config[LVM-backend/volume_driver],Cinder_config[DEFAULT/use_syslog_rfc_format],Cinder_config[DEFAULT/enable_v1_api],Cinder_config[oslo_messaging_rabbit/heartbeat_timeout_threshold],Cinder_config[oslo_messaging_rabbit/rabbit_virtual_host],Cinder_config[DEFAULT/glance_api_version],Cinder_config[oslo_messaging_rabbit/rabbit_password],Cinder_config[DEFAULT/rpc_backend],Cinder_config[oslo_messaging_rabbit/rabbit_userid],Cinder_config[LVM-backend/iscsi_ip_address],Cinder_config[DEFAULT/backup_swift_url],Cinder_config[database/idle_timeout],Cinder_config[database/max_pool_size],Cinder_config[database/max_retries],Cinder_config[database/max_overflow],Cinder_config[database/connection],Cinder_config[DEFAULT/default_log_levels]",
     "failed"=>47,
     "changed"=>7,
     "total"=>108,
     "restarted"=>0,
     "out_of_sync"=>54,
     "failed_to_restart"=>0,
     "scheduled"=>0,
     "skipped"=>3},
   :changes=>{"total"=>7},
   :events=>{"failure"=>47, "success"=>7, "total"=>54},
   :version=>{"config"=>1464099906, "puppet"=>"3.8.5"},
   :err_msg=>
    "Process not running but not empty lockfile is present. Trying to remove lockfile...ok.",
   :status=>"stopped",
   :running=>0,
   :enabled=>1,
   :idling=>0,
   :stopped=>1,
   :lastrun=>1464100127,
   :runtime=>1,
   :output=>"Currently stopped; last completed run 1 seconds ago"}}

2016-05-24 14:28:48 WARNING [30547] Validation of node:
{"uid"=>"5",
 "status"=>"error",
 "error_type"=>"deploy",
 "error_msg"=>
  "All nodes are finished. Failed tasks: Task[top-role-cinder/6] Stopping the deployment process!"}
 for report failed: Task status provided '' is not supported; Task name is not provided
2016-05-24 14:28:48 INFO [30547] Cluster[]: All nodes are finished. Failed tasks: Task[top-role-cinder/6] Stopping the deployment process!
2016-05-24 14:28:48 DEBUG [30547] Graph[virtual_sync_node]: Found failed tasks: post_deployment_start, post_deployment_end, deploy_end
2016-05-24 14:28:48 DEBUG [30547] Graph[6]: Found failed tasks: enable_cinder_volume_service, cgroups, upload_nodes_info, dns-client, top-role-cinder, update_hosts, ntp-client
2016-05-24 14:28:48 DEBUG [30547] Graph[4]: Found failed tasks: ironic_post_swift_key, dump_rabbitmq_definitions, ceph_create_pools, updatedb, ceilometer-radosgw-user, vcenter_compute_zones_create, enable_rados, ntp-check, disable_keystone_service_token, upload_cirros, update_hosts, cgroups, openstack-network-routers-ha, upload_murano_package, ironic_upload_images, dns-client, public_vip_ping, upload_nodes_info, ceph_ready_check, enable_quorum, ntp-server

Steps to reproduce:
1)Deploy master node

2)Disable plain http for nailgun
Add to the /etc/fuel/astute.yaml
"SSL":
"force_https": "true"

3)restart nginx container using command 'service restart nginx'

4)turn on 3 vms and wait while it is appears as discovered nodes

5)create cluster :
Ubuntu
neutron tun
1 controller
1 compute
1 cinder

6)deploy it <<<<<<<<<<<<<<<<<<<<<failed here
7)Run ostf

Expected results:
  deployment successfully

Actual result:
Deployment has failed. All nodes are finished. Failed tasks: Task[top-role-cinder/6] Stopping the deployment process!

 Reproducibility:
 Workaround:

 Impact:
 acceptance

 Description of the environment:
 cat /etc/fuel_build_id:
 376
cat /etc/fuel_build_number:
 376
cat /etc/fuel_release:
 9.0
cat /etc/fuel_openstack_version:
 mitaka-9.0
rpm -qa | egrep 'fuel|astute|network-checker|nailgun|packetary|shotgun':
 fuel-release-9.0.0-1.mos6346.noarch
 fuel-bootstrap-cli-9.0.0-1.mos281.noarch
 fuel-migrate-9.0.0-1.mos8376.noarch
 rubygem-astute-9.0.0-1.mos745.noarch
 fuel-misc-9.0.0-1.mos8376.noarch
 network-checker-9.0.0-1.mos72.x86_64
 fuel-mirror-9.0.0-1.mos136.noarch
 fuel-openstack-metadata-9.0.0-1.mos8693.noarch
 fuel-notify-9.0.0-1.mos8376.noarch
 nailgun-mcagents-9.0.0-1.mos745.noarch
 fuel-provisioning-scripts-9.0.0-1.mos8693.noarch
 python-fuelclient-9.0.0-1.mos315.noarch
 fuelmenu-9.0.0-1.mos270.noarch
 fuel-9.0.0-1.mos6346.noarch
 fuel-utils-9.0.0-1.mos8376.noarch
 fuel-setup-9.0.0-1.mos6346.noarch
 fuel-library9.0-9.0.0-1.mos8376.noarch
 shotgun-9.0.0-1.mos88.noarch
 fuel-agent-9.0.0-1.mos281.noarch
 fuel-ui-9.0.0-1.mos2688.noarch
 fuel-ostf-9.0.0-1.mos934.noarch
 python-packetary-9.0.0-1.mos136.noarch
 fuel-nailgun-9.0.0-1.mos8693.noarch

[Michael Polenchuk (mpolenchuk)]

"Network is unreachable" on node-6
I believe it's temporary network issue,
please try to reproduce it again.

[Dmitry Pyzhov (dpyzhov)]

Bug is incomplete status for a month. Changing status to invalid. Please reopen the issue if you have more data.