Dear All,
My Kubernetes Cluster is timing out after 60 mins (default timeout).
Following is the update I am getting in
magnum.log : --------------------------------------------------------------------------------------------
2024-07-04 17:31:27.861 28315 DEBUG heatclient.common.http [None req-b33789b1-588d-4c96-aa1c-f8c287eed979 - - - - - -] curl -g -i -X GET -H 'Content-Type: application/json' -H 'Accept: application/json' -H 'User-Agent: python-heatclient' -H 'X-Auth-Token: {SHA1}2a32154864629c1cf3898b05ffb7695131992650' -H 'X-Auth-Url: http://controller:5000/v3' http://10.0.0.10:8004/v1/38b8901ceeb94cb8af93cfd6b5b33f7e/stacks/1376de41-8d66-4eac-9829-d229b70f1fb3 log_curl_request /usr/lib/python3.9/site-packages/heatclient/common/http.py:143
2024-07-04 17:31:27.877 28315 DEBUG heatclient.common.http [None req-b33789b1-588d-4c96-aa1c-f8c287eed979 - - - - - -]
HTTP/1.1 302 Found
Location: http://10.0.0.10:8004/v1/38b8901ceeb94cb8af93cfd6b5b33f7e/stacks/deb-qybmbhpbf6oi/1376de41-8d66-4eac-9829-d229b70f1fb3?resolve_outputs=False
Content-Length: 423
Content-Type: application/json
X-Openstack-Request-Id: req-4562290a-b608-4fd0-a9b4-4b93e022b239
Date: Thu, 04 Jul 2024 16:31:27 GMT
Connection: keep-alive
{"message": "The resource was found at <a href=\"http://10.0.0.10:8004/v1/38b8901ceeb94cb8af93cfd6b5b33f7e/stacks/deb-qybmbhpbf6oi/1376de41-8d66-4eac-9829-d229b70f1fb3?resolve_outputs=False\">http://10.0.0.10:8004/v1/38b8901ceeb94cb8af93cfd6b5b33f7e/stacks/deb-qybmbhpbf6oi/1376de41-8d66-4eac-9829-d229b70f1fb3?resolve_outputs=False</a>;\nyou should be redirected automatically.\n\n", "code": "302 Found", "title": "Found"}
log_http_response /usr/lib/python3.9/site-packages/heatclient/common/http.py:156
2024-07-04 17:31:27.878 28315 DEBUG heatclient.common.http [None req-b33789b1-588d-4c96-aa1c-f8c287eed979 - - - - - -] curl -g -i -X GET -H 'Content-Type: application/json' -H 'Accept: application/json' -H 'User-Agent: python-heatclient' -H 'X-Auth-Token: {SHA1}2a32154864629c1cf3898b05ffb7695131992650' -H 'X-Auth-Url: http://controller:5000/v3' http://10.0.0.10:8004/v1/38b8901ceeb94cb8af93cfd6b5b33f7e/stacks/deb-qybmbhpbf6oi/1376de41-8d66-4eac-9829-d229b70f1fb3?resolve_outputs=False log_curl_request /usr/lib/python3.9/site-packages/heatclient/common/http.py:143
2024-07-04 17:31:27.900 28315 DEBUG heatclient.common.http [None req-b33789b1-588d-4c96-aa1c-f8c287eed979 - - - - - -]
HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 7681
X-Openstack-Request-Id: req-ed9275f1-be26-4f6f-87c2-e4cf2b02dba7
Date: Thu, 04 Jul 2024 16:31:27 GMT
Connection: keep-alive
{"stack": {"stack_name": "deb-qybmbhpbf6oi", "id": "1376de41-8d66-4eac-9829-d229b70f1fb3", "links": [{"href": "http://10.0.0.10:8004/v1/38b8901ceeb94cb8af93cfd6b5b33f7e/stacks/deb-qybmbhpbf6oi/1376de41-8d66-4eac-9829-d229b70f1fb3", "rel": "self"}], "creation_time": "2024-07-04T15:31:23Z", "updated_time": null, "deletion_time": null, "notification_topics": [], "parameters": {"OS::stack_id": "1376de41-8d66-4eac-9829-d229b70f1fb3", "OS::project_id": "38b8901ceeb94cb8af93cfd6b5b33f7e", "OS::stack_name": "deb-qybmbhpbf6oi", "existing_master_private_ip": "", "is_cluster_stack": "True", "master_role": "master", "worker_role": "worker", "existing_security_group": "", "ssh_key_name": "mykey", "ssh_public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCWod6tojAxdE7cBvi/BBI2I58ps46rRA1yxn4/DwK/851Uni6oIgYX4fJA5f1vwg/oJD0SnsHXXe1FR2nMgGKCeHxD9UE057UtT6XnAZdwRgu51cAfKq4ny7B6wUur9J4UWFFqk+rtZTz7kMWg8/U2XEdryMOjDMiehO7JLFsUsJikdtjcd6tT1tyeA48wIVNiYZdtwai9kNfNMLCzqbPhzbH6rPvusbPsJRfS5VU1ct5ztyWtMfTx7cUKPW3JOoW/GM1TqLMomf0q8z7JbUJ6OOIz4PIAHvksXEDubWloT/yb3xfBoSgv9cHkO8FtvKPieJBPvAEfmImr0+v85SIV5KkVvRLWsDRhmlHRKOSRNlZuTrDGC1P/Vtf7d2B+B8Tkb1Qlwew331IFVmEQn79l8G1QbGntZdRCdGnEZrI1n9IKDOZg4ZGjP1IudztoC7o2yo93/69EgjCNpdkBSr3Fq6YgUFr0GWcz5qYn53JiEHEVXChz7lt0YpDtoeQrtpE= root@controller", "external_network": "4299ca53-3a6c-4e65-bb7b-5c22bb4f85de", "fixed_network": "", "fixed_network_name": "deb", "fixed_subnet": "", "fixed_subnet_cidr": "10.0.0.0/24", "master_image": "fedora-coreos-latest", "minion_image": "fedora-coreos-latest", "master_flavor": "m1.small", "master_nodegroup_name": "default-master", "worker_nodegroup_name": "default-worker", "heapster_enabled": "False", "metrics_server_enabled": "True", "metrics_server_chart_tag": "v3.7.0", "minion_flavor": "m1.small", "prometheus_monitoring": "False", "grafana_admin_passwd": "******", "dns_nameserver": "8.8.8.8", "number_of_masters": "1", "number_of_minions": "1", "portal_network_cidr": "10.254.0.0/16", "network_driver": "flannel", "flannel_network_cidr": "10.100.0.0/16", "flannel_network_subnetlen": "24", "flannel_backend": "vxlan", "system_pods_initial_delay": "30", "system_pods_timeout": "5", "admission_control_list": "NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,PersistentVolumeClaimResize,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,RuntimeClass", "kube_allow_priv": "true", "boot_volume_size": "0", "boot_volume_type": "", "etcd_volume_size": "0", "etcd_volume_type": "", "docker_volume_size": "0", "docker_volume_type": "", "docker_storage_driver": "overlay2", "cgroup_driver": "cgroupfs", "traefik_ingress_controller_tag": "v1.7.28", "wait_condition_timeout": "6000", "minions_to_remove": "", "discovery_url": "https://discovery.etcd.io/43b8c1daeb5852880873752d4b5d05aa", "registry_enabled": "False", "registry_port": "5000", "swift_region": "", "registry_container": "container", "registry_insecure": "True", "registry_chunksize": "5242880", "volume_driver": "", "region_name": "RegionOne", "username": "admin", "password": "******", "loadbalancing_protocol": "TCP", "tls_disabled": "False", "kube_dashboard_enabled": "True", "influx_grafana_dashboard_enabled": "False", "verify_ca": "True", "kubernetes_port": "6443", "cluster_uuid": "f4886345-ed56-4cdc-9678-b3e359a4a1e9", "magnum_url": "http://10.0.0.10:9511/v1", "http_proxy": "", "https_proxy": "", "no_proxy": "", "trustee_domain_id": "1a5f9ba13a0c4a809e75e0cbcb1b68b2", "trustee_user_id": "6e012773ab0c4ec0b4f1ff18dc8e26e7", "trustee_username": "f4886345-ed56-4cdc-9678-b3e359a4a1e9_38b8901ceeb94cb8af93cfd6b5b33f7e", "trustee_password": "******", "trust_id": "******", "auth_url": "http://controller:5000/v3", "hyperkube_prefix": "docker.io/rancher/", "kube_tag": "v1.23.3-rancher1", "master_kube_tag": "v1.23.3-rancher1", "minion_kube_tag": "v1.23.3-rancher1", "cloud_provider_tag": "v1.23.1", "cloud_provider_enabled": "True", "etcd_tag": "v3.4.6", "coredns_tag": "1.6.6", "flannel_tag": "v0.15.1", "flannel_cni_tag": "v0.3.0", "kube_version": "v1.18.16", "kube_dashboard_version": "v2.0.0", "metrics_scraper_tag": "v1.0.4", "insecure_registry_url": "", "container_infra_prefix": "", "dns_service_ip": "10.254.0.10", "dns_cluster_domain": "cluster.local", "openstack_ca": "******", "nodes_affinity_policy": "soft-anti-affinity", "availability_zone": "nova", "cert_manager_api": "False", "ca_key": "******", "calico_tag": "v3.21.2", "calico_kube_controllers_tag": "v1.0.3", "calico_ipv4pool": "10.100.0.0/16", "calico_ipv4pool_ipip": "Off", "pods_network_cidr": "10.100.0.0/16", "ingress_controller": "", "ingress_controller_role": "ingress", "octavia_ingress_controller_tag": "v1.18.0", "kubelet_options": "", "kubeapi_options": "", "kubecontroller_options": "", "kubeproxy_options": "", "kubescheduler_options": "", "octavia_enabled": "False", "octavia_provider": "amphora", "octavia_lb_algorithm": "ROUND_ROBIN", "octavia_lb_healthcheck": "True", "kube_service_account_key": "******", "kube_service_account_private_key": "******", "prometheus_tag": "v1.8.2", "grafana_tag": "5.1.5", "heat_container_agent_tag": "wallaby-stable-1", "keystone_auth_enabled": "True", "keystone_auth_default_policy": "{}", "k8s_keystone_auth_tag": "v1.18.0", "monitoring_enabled": "False", "monitoring_retention_days": "14", "monitoring_retention_size": "14", "monitoring_interval_seconds": "30", "monitoring_storage_class_name": "", "monitoring_ingress_enabled": "False", "cluster_basic_auth_secret": "", "cluster_root_domain_name": "localhost", "prometheus_operator_chart_tag": "v8.12.13", "prometheus_adapter_enabled": "True", "prometheus_adapter_chart_tag": "1.4.0", "prometheus_adapter_configmap": "", "project_id": "38b8901ceeb94cb8af93cfd6b5b33f7e", "tiller_enabled": "False", "tiller_tag": "v2.16.7", "tiller_namespace": "magnum-tiller", "helm_client_url": "", "helm_client_sha256": "018f9908cb950701a5d59e757653a790c66d8eda288625dbb185354ca6f41f6b", "helm_client_tag": "v3.2.1", "auto_healing_enabled": "False", "auto_healing_controller": "draino", "magnum_auto_healer_tag": "v1.18.0", "auto_scaling_enabled": "False", "cinder_csi_enabled": "True", "cinder_csi_plugin_tag": "v1.23.0", "csi_attacher_tag": "v3.3.0", "csi_provisioner_tag": "v3.0.0", "csi_snapshotter_tag": "v4.2.1", "csi_resizer_tag": "v1.3.0", "csi_node_driver_registrar_tag": "v2.4.0", "csi_liveness_probe_tag": "v2.5.0", "node_problem_detector_tag": "v0.6.2", "nginx_ingress_controller_tag": "0.32.0", "nginx_ingress_controller_chart_tag": "4.0.17", "draino_tag": "abf028a", "autoscaler_tag": "v1.18.1", "min_node_count": "0", "max_node_count": "2", "update_max_batch_size": "1", "npd_enabled": "True", "ostree_remote": "", "ostree_commit": "", "use_podman": "True", "selinux_mode": "enforcing", "kube_image_digest": "", "container_runtime": "host-docker", "containerd_version": "1.4.4", "containerd_tarball_url": "", "containerd_tarball_sha256": "96641849cb78a0a119223a427dfdc1ade88412ef791a14193212c8c8e29d447b", "post_install_manifest_url": "", "master_lb_allowed_cidrs": ""}, "description": "This template will boot a Kubernetes cluster with one or more minions (as specified by the number_of_minions parameter, which defaults to 1).\n", "template_description": "This template will boot a Kubernetes cluster with one or more minions (as specified by the number_of_minions parameter, which defaults to 1).\n", "capabilities": [], "disable_rollback": true, "timeout_mins": 60, "stack_owner": "admin", "parent": null, "stack_user_project_id": "596348e1847b480f90166064778f3bd3", "tags": [], "stack_status": "CREATE_FAILED", "stack_status_reason": "Timed out"}}
log_http_response /usr/lib/python3.9/site-packages/heatclient/common/http.py:156
2024-07-04 17:31:27.904 28315 DEBUG magnum.drivers.heat.template_def [None req-b33789b1-588d-4c96-aa1c-f8c287eed979 - - - - - -] cluster f4886345-ed56-4cdc-9678-b3e359a4a1e9, status CREATE_IN_PROGRESS, stack 1376de41-8d66-4eac-9829-d229b70f1fb3 does not have output_key api_address get_output_value /usr/lib/python3.9/site-packages/magnum/drivers/heat/template_def.py:130
2024-07-04 17:31:27.904 28315 DEBUG magnum.drivers.heat.k8s_template_def [None req-b33789b1-588d-4c96-aa1c-f8c287eed979 - - - - - -] Using heat_output: kube_minions set_output /usr/lib/python3.9/site-packages/magnum/drivers/heat/k8s_template_def.py:71
2024-07-04 17:31:27.904 28315 DEBUG magnum.drivers.heat.template_def [None req-b33789b1-588d-4c96-aa1c-f8c287eed979 - - - - - -] cluster f4886345-ed56-4cdc-9678-b3e359a4a1e9, status CREATE_IN_PROGRESS, stack 1376de41-8d66-4eac-9829-d229b70f1fb3 does not have output_key kube_minions get_output_value /usr/lib/python3.9/site-packages/magnum/drivers/heat/template_def.py:130
2024-07-04 17:31:27.927 28315 DEBUG magnum.drivers.heat.template_def [None req-b33789b1-588d-4c96-aa1c-f8c287eed979 - - - - - -] cluster f4886345-ed56-4cdc-9678-b3e359a4a1e9, status CREATE_IN_PROGRESS, stack 1376de41-8d66-4eac-9829-d229b70f1fb3 does not have output_key api_address get_output_value /usr/lib/python3.9/site-packages/magnum/drivers/heat/template_def.py:130
2024-07-04 17:31:27.927 28315 DEBUG magnum.drivers.heat.k8s_template_def [None req-b33789b1-588d-4c96-aa1c-f8c287eed979 - - - - - -] Using heat_output: kube_minions set_output /usr/lib/python3.9/site-packages/magnum/drivers/heat/k8s_template_def.py:71
2024-07-04 17:31:27.928 28315 DEBUG magnum.drivers.heat.template_def [None req-b33789b1-588d-4c96-aa1c-f8c287eed979 - - - - - -] cluster f4886345-ed56-4cdc-9678-b3e359a4a1e9, status CREATE_IN_PROGRESS, stack 1376de41-8d66-4eac-9829-d229b70f1fb3 does not have output_key kube_minions get_output_value /usr/lib/python3.9/site-packages/magnum/drivers/heat/template_def.py:130
2024-07-04 17:31:27.935 28315 ERROR magnum.drivers.heat.driver [None req-b33789b1-588d-4c96-aa1c-f8c287eed979 - - - - - -] Nodegroup error, stack status: CREATE_FAILED, stack_id: 1376de41-8d66-4eac-9829-d229b70f1fb3, reason: Timed out
2024-07-04 17:31:27.953 28315 DEBUG magnum.service.periodic [None req-b33789b1-588d-4c96-aa1c-f8c287eed979 - - - - - -] Status for cluster 20 updated to CREATE_FAILED (default-master failed, default-worker failed) update_status /usr/lib/python3.9/site-packages/magnum/service/periodic.py:84
--------------------------------------------------------------------------------------------------------
heat-engine.log : --------------------------------------------------------------------------------------
2024-07-04 17:31:22.375 28174 DEBUG heat.engine.scheduler [None req-fed20218-6210-4c2a-94ac-04a0d2c07074 admin - - - - -] Task create from SoftwareDeployment "master_config_deployment" Stack "deb-qybmbhpbf6oi-kube_masters-xvpu7lm3crhk-0-rfrlvlj63szd" [8c6d2928-5e9a-4150-b698-d2961b719e4e] running step /usr/lib/python3.9/site-packages/heat/engine/scheduler.py:207
2024-07-04 17:31:22.386 28174 DEBUG heat.engine.scheduler [None req-fed20218-6210-4c2a-94ac-04a0d2c07074 admin - - - - -] Task create from SoftwareDeployment "master_config_deployment" Stack "deb-qybmbhpbf6oi-kube_masters-xvpu7lm3crhk-0-rfrlvlj63szd" [8c6d2928-5e9a-4150-b698-d2961b719e4e] sleeping _sleep /usr/lib/python3.9/site-packages/heat/engine/scheduler.py:148
2024-07-04 17:31:22.824 28174 DEBUG heat.engine.scheduler [None req-fed20218-6210-4c2a-94ac-04a0d2c07074 admin - - - - -] Task create from ResourceGroup "kube_masters" Stack "deb-qybmbhpbf6oi" [1376de41-8d66-4eac-9829-d229b70f1fb3] running step /usr/lib/python3.9/site-packages/heat/engine/scheduler.py:207
2024-07-04 17:31:22.828 28174 DEBUG heat.engine.scheduler [None req-fed20218-6210-4c2a-94ac-04a0d2c07074 admin - - - - -] Task create from ResourceGroup "kube_masters" Stack "deb-qybmbhpbf6oi" [1376de41-8d66-4eac-9829-d229b70f1fb3] sleeping _sleep /usr/lib/python3.9/site-packages/heat/engine/scheduler.py:148
2024-07-04 17:31:22.873 28175 DEBUG heat.engine.scheduler [None req-332508e7-ab11-44f2-b537-c66dfa63020d - - - - - -] Task create from TemplateResource "0" Stack "deb-qybmbhpbf6oi-kube_masters-xvpu7lm3crhk" [d456ad99-83f8-4813-a63e-02cfd550835a] running step /usr/lib/python3.9/site-packages/heat/engine/scheduler.py:207
2024-07-04 17:31:22.876 28175 DEBUG heat.engine.scheduler [None req-332508e7-ab11-44f2-b537-c66dfa63020d - - - - - -] Task create from TemplateResource "0" Stack "deb-qybmbhpbf6oi-kube_masters-xvpu7lm3crhk" [d456ad99-83f8-4813-a63e-02cfd550835a] sleeping _sleep /usr/lib/python3.9/site-packages/heat/engine/scheduler.py:148
2024-07-04 17:31:23.388 28174 DEBUG heat.engine.scheduler [None req-fed20218-6210-4c2a-94ac-04a0d2c07074 admin - - - - -] Task create from SoftwareDeployment "master_config_deployment" Stack "deb-qybmbhpbf6oi-kube_masters-xvpu7lm3crhk-0-rfrlvlj63szd" [8c6d2928-5e9a-4150-b698-d2961b719e4e] running step /usr/lib/python3.9/site-packages/heat/engine/scheduler.py:207
2024-07-04 17:31:23.398 28174 DEBUG heat.engine.scheduler [None req-fed20218-6210-4c2a-94ac-04a0d2c07074 admin - - - - -] Task create from SoftwareDeployment "master_config_deployment" Stack "deb-qybmbhpbf6oi-kube_masters-xvpu7lm3crhk-0-rfrlvlj63szd" [8c6d2928-5e9a-4150-b698-d2961b719e4e] sleeping _sleep /usr/lib/python3.9/site-packages/heat/engine/scheduler.py:148
2024-07-04 17:31:23.829 28174 INFO heat.engine.scheduler [None req-fed20218-6210-4c2a-94ac-04a0d2c07074 admin - - - - -] Task create from ResourceGroup "kube_masters" Stack "deb-qybmbhpbf6oi" [1376de41-8d66-4eac-9829-d229b70f1fb3] timed out
2024-07-04 17:31:23.830 28174 DEBUG heat.engine.resources.stack_resource [None req-fed20218-6210-4c2a-94ac-04a0d2c07074 admin - - - - -] Cancelling CREATE of ResourceGroup "kube_masters" [d456ad99-83f8-4813-a63e-02cfd550835a] Stack "deb-qybmbhpbf6oi" [1376de41-8d66-4eac-9829-d229b70f1fb3] _handle_cancel /usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py:551
2024-07-04 17:31:23.839 28173 INFO heat.engine.service [None req-fed20218-6210-4c2a-94ac-04a0d2c07074 admin - - - - -] Starting cancel of updating stack deb-qybmbhpbf6oi-kube_masters-xvpu7lm3crhk
2024-07-04 17:31:23.877 28175 DEBUG heat.engine.scheduler [None req-332508e7-ab11-44f2-b537-c66dfa63020d - - - - - -] Task create from TemplateResource "0" Stack "deb-qybmbhpbf6oi-kube_masters-xvpu7lm3crhk" [d456ad99-83f8-4813-a63e-02cfd550835a] running step /usr/lib/python3.9/site-packages/heat/engine/scheduler.py:207
2024-07-04 17:31:23.880 28175 DEBUG heat.engine.scheduler [None req-332508e7-ab11-44f2-b537-c66dfa63020d - - - - - -] Task create from TemplateResource "0" Stack "deb-qybmbhpbf6oi-kube_masters-xvpu7lm3crhk" [d456ad99-83f8-4813-a63e-02cfd550835a] sleeping _sleep /usr/lib/python3.9/site-packages/heat/engine/scheduler.py:148
2024-07-04 17:31:23.899 28173 INFO heat.engine.stack [None req-fed20218-6210-4c2a-94ac-04a0d2c07074 admin - - - - -] Stack CREATE FAILED (deb-qybmbhpbf6oi-kube_masters-xvpu7lm3crhk): Stack CREATE cancelled
2024-07-04 17:31:23.902 28173 DEBUG heat.engine.stack [None req-fed20218-6210-4c2a-94ac-04a0d2c07074 admin - - - - -] Persisting stack deb-qybmbhpbf6oi-kube_masters-xvpu7lm3crhk status CREATE FAILED _send_notification_and_add_event /usr/lib/python3.9/site-packages/heat/engine/stack.py:1109
2024-07-04 17:31:23.931 28174 INFO heat.engine.stack [None req-fed20218-6210-4c2a-94ac-04a0d2c07074 admin - - - - -] Stack CREATE FAILED (deb-qybmbhpbf6oi): Timed out
2024-07-04 17:31:23.934 28174 DEBUG heat.engine.stack [None req-fed20218-6210-4c2a-94ac-04a0d2c07074 admin - - - - -] Persisting stack deb-qybmbhpbf6oi status CREATE FAILED _send_notification_and_add_event /usr/lib/python3.9/site-packages/heat/engine/stack.py:1109
2024-07-04 17:31:23.985 28173 INFO heat.engine.stack [None req-fed20218-6210-4c2a-94ac-04a0d2c07074 - - - - - -] Stack CREATE FAILED (deb-qybmbhpbf6oi-kube_masters-xvpu7lm3crhk-0-rfrlvlj63szd): Stack CREATE cancelled
2024-07-04 17:31:23.989 28173 DEBUG heat.engine.stack [None req-fed20218-6210-4c2a-94ac-04a0d2c07074 - - - - - -] Persisting stack deb-qybmbhpbf6oi-kube_masters-xvpu7lm3crhk-0-rfrlvlj63szd status CREATE FAILED _send_notification_and_add_event /usr/lib/python3.9/site-packages/heat/engine/stack.py:1109
2024-07-04 17:31:24.399 28174 DEBUG heat.engine.scheduler [None req-fed20218-6210-4c2a-94ac-04a0d2c07074 admin - - - - -] Task create from SoftwareDeployment "master_config_deployment" Stack "deb-qybmbhpbf6oi-kube_masters-xvpu7lm3crhk-0-rfrlvlj63szd" [8c6d2928-5e9a-4150-b698-d2961b719e4e] running step /usr/lib/python3.9/site-packages/heat/engine/scheduler.py:207
2024-07-04 17:31:24.414 28174 DEBUG heat.engine.scheduler [None req-fed20218-6210-4c2a-94ac-04a0d2c07074 admin - - - - -] Task create from SoftwareDeployment "master_config_deployment" Stack "deb-qybmbhpbf6oi-kube_masters-xvpu7lm3crhk-0-rfrlvlj63szd" [8c6d2928-5e9a-4150-b698-d2961b719e4e] sleeping _sleep /usr/lib/python3.9/site-packages/heat/engine/scheduler.py:148
2024-07-04 17:31:24.882 28175 DEBUG heat.engine.scheduler [None req-332508e7-ab11-44f2-b537-c66dfa63020d - - - - - -] Task create from TemplateResource "0" Stack "deb-qybmbhpbf6oi-kube_masters-xvpu7lm3crhk" [d456ad99-83f8-4813-a63e-02cfd550835a] running step /usr/lib/python3.9/site-packages/heat/engine/scheduler.py:207
2024-07-04 17:31:24.885 28175 INFO heat.engine.resource [None req-332508e7-ab11-44f2-b537-c66dfa63020d - - - - - -] CREATE: TemplateResource "0" [8c6d2928-5e9a-4150-b698-d2961b719e4e] Stack "deb-qybmbhpbf6oi-kube_masters-xvpu7lm3crhk" [d456ad99-83f8-4813-a63e-02cfd550835a]
2024-07-04 17:31:24.885 28175 ERROR heat.engine.resource Traceback (most recent call last):
2024-07-04 17:31:24.885 28175 ERROR heat.engine.resource File "/usr/lib/python3.9/site-packages/heat/engine/resource.py", line 922, in _action_recorder
2024-07-04 17:31:24.885 28175 ERROR heat.engine.resource yield
2024-07-04 17:31:24.885 28175 ERROR heat.engine.resource File "/usr/lib/python3.9/site-packages/heat/engine/resource.py", line 1034, in _do_action
2024-07-04 17:31:24.885 28175 ERROR heat.engine.resource yield from self.action_handler_task(action, args=handler_args)
2024-07-04 17:31:24.885 28175 ERROR heat.engine.resource File "/usr/lib/python3.9/site-packages/heat/engine/resource.py", line 984, in action_handler_task
2024-07-04 17:31:24.885 28175 ERROR heat.engine.resource done = check(handler_data)
2024-07-04 17:31:24.885 28175 ERROR heat.engine.resource File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 408, in check_create_complete
2024-07-04 17:31:24.885 28175 ERROR heat.engine.resource return self._check_status_complete(self.CREATE)
2024-07-04 17:31:24.885 28175 ERROR heat.engine.resource File "/usr/lib/python3.9/site-packages/heat/engine/resources/stack_resource.py", line 450, in _check_status_complete
2024-07-04 17:31:24.885 28175 ERROR heat.engine.resource raise exception.ResourceFailure(status_reason, self,
2024-07-04 17:31:24.885 28175 ERROR heat.engine.resource heat.common.exception.ResourceFailure: resources[0]: Stack CREATE cancelled
2024-07-04 17:31:24.885 28175 ERROR heat.engine.resource
2024-07-04 17:31:24.936 28175 INFO heat.engine.stack [None req-332508e7-ab11-44f2-b537-c66dfa63020d - - - - - -] Stack CREATE FAILED (deb-qybmbhpbf6oi-kube_masters-xvpu7lm3crhk): Resource CREATE failed: resources[0]: Stack CREATE cancelled
2024-07-04 17:31:24.939 28175 DEBUG heat.engine.stack [None req-332508e7-ab11-44f2-b537-c66dfa63020d - - - - - -] Persisting stack deb-qybmbhpbf6oi-kube_masters-xvpu7lm3crhk status CREATE FAILED _send_notification_and_add_event /usr/lib/python3.9/site-packages/heat/engine/stack.py:1109
2024-07-04 17:31:24.957 28175 DEBUG heat.db.api [None req-fed20218-6210-4c2a-94ac-04a0d2c07074 - - - - - -] Did not set stack state with values {'action': 'CREATE', 'status': 'FAILED', 'status_reason': 'Resource CREATE failed: resources[0]: Stack CREATE cancelled'}, stack id: d456ad99-83f8-4813-a63e-02cfd550835a with expected traversal: fe3019c2-d9b5-4167-af55-e8ff7a85948c stack_update /usr/lib/python3.9/site-packages/heat/db/api.py:843
2024-07-04 17:31:24.959 28175 INFO heat.engine.stack [None req-fed20218-6210-4c2a-94ac-04a0d2c07074 - - - - - -] Stack deb-qybmbhpbf6oi-kube_masters-xvpu7lm3crhk traversal fe3019c2-d9b5-4167-af55-e8ff7a85948c no longer active; not setting state to CREATE_FAILED
--------------------------------------------------------------------------------------------------------
