StarlingX

Ansible Error - StarlingX AIO-DX Deployment

Bug #1959637 reported by Mohammed AA Abuibaid
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
StarlingX
Invalid
Low
Ramaswamy Subramanian

Bug Description

Hello Stx Community,

We are running into an error which in brief says: "No handlers could be found for logger \"cgtsclient.common.http\". We have adjusted the parameter “standard_root_disk_size: “ parameter in main.yml from the directory ”/usr/share/ansiblestz-ansible/playbooks/roles/bootstrap/prepare-env/vars/”. Previous it was 500, we adjusted it to 100, then 50 and then 1 but still received the same error. This seems more like an HTTP issue more than a physical disk problem. The entire console output of stderr is attached, formatted a bit for readability. There is a ticket from last week but it includes “Disk size requirements not met” which isn’t included in our output so it’s likely different. http://lists.starlingx.io/pipermail/starlingx-discuss/2021-January/010647.html

Any help this is would be appreciated.

Kind regards,
Mohammed
PhD Student @ CarletonU

Tags: stx.config
Revision history for this message
Mohammed AA Abuibaid (abuibaid) wrote :
Revision history for this message
Ghada Khalil (gkhalil) wrote :

screening: Please try to get help from the <email address hidden> mailing list

Revision history for this message
Ghada Khalil (gkhalil) wrote :

It looks like there is a thread related to this issue on the mailing list:
http://lists.starlingx.io/pipermail/starlingx-discuss/2022-February/012740.html

Revision history for this message
Ghada Khalil (gkhalil) wrote :

@Mohammed, can you provide more information on the build you are using? Is it a CENGN build or a private build? Can you include some more info as per the guidelines here: https://docs.starlingx.io/developer_resources/debug_issues.html

tags: added: stx.config
Revision history for this message
Mohammed AA Abuibaid (abuibaid) wrote :

Thanks, Ghada, for opening the discussion about the case here. Yes, we are using CENGN Build. We have problems with the ansible YAML config files that refer to outdated Helm releases. More precisely, it gets stuck during the installation when it attempts to download images; looking at the output, it says it cannot download "ghcr.io/kubernetes-helm/tiller:v2.16.1500". To fix this, we changed it to "ghcr.io/helm/tiller:v2.16.1" in two YAML files:

In /usr/share/ansible/stx-ansible/playbooks/roles/common/load-images-information/vars/k8s-v1.18.1/system-images.yml we changed "tiller_img: " to "tiller_imag : ghcr.io/helm/tiller:v2.16.1"

In /usr/share/ansible/stx-ansible/playbooks/roles/common/armada-helm/templates/armada-overrides.yaml.j2 we changed "tiller: " to "tiller : registry.local:9001/ghcr.io/helm/tiller:v2.16.1"

Revision history for this message
Elijah Mendez (ssyntaxerror) wrote :

Log files request by John Kung from the StarlingX Listserv.

Revision history for this message
Ramaswamy Subramanian (rsubrama) wrote :
Download full text (6.5 KiB)

It appears like disk is too small. Was there an attempt to increase the disk space allocated and attempt the installation again?

Below are details from the logs.

sysinv-api.log --> The API 'POST /v1/ihosts HTTP/1.0' is responding with status '500'.
---
sysinv 2022-02-02 10:40:53.234 79558 INFO sysinv.api.hooks.auditor [req-cfe7e9e1-1b81-40c3-ac20-1edb409e8ae8 bb57778856b248b895a9c07028e003ab 908189cfae0b4755b3638bb161bff250] ::ffff:127.0.0.1 "POST /v1/ihosts HTTP/1.0" status: 500 len: 730 time: 0.115689992905 POST: {u'tboot': u'false', u'console': u'tty0', u'mgmt_ip': u'192.168.204.2', u'install_output': u'text', u'hostname': u'controller-0', u'operational': u'disabled', u'invprovision': u'provisioning', u'boot_device': u'/dev/disk/by-path/pci-0000:01:01.0-scsi-0:0:0:0', u'rootfs_device': u'/dev/disk/by-path/pci-0000:01:01.0-scsi-0:0:0:0', u'administrative': u'locked', u'availability': u'offline', u'personality': u'controller', u'mgmt_mac': u'00:00:00:00:00:00'} host:127.0.0.1:6385 agent:Python-httplib2/0.9.2 (gzip) user: admin tenant: admin domain: Default

---

The processing of the API resulted in the below error.

---
sysinv 2022-02-02 10:40:53.220 76654 INFO sysinv.conductor.manager [-] Disk size : 60 ... disk too small
sysinv 2022-02-02 10:40:53.221 76654 ERROR sysinv.openstack.common.rpc.amqp [-] Exception during message handling: SysinvException: Disk size requirements not met.
2022-02-02 10:40:53.221 76654 ERROR sysinv.openstack.common.rpc.amqp Traceback (most recent call last):
2022-02-02 10:40:53.221 76654 ERROR sysinv.openstack.common.rpc.amqp File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/rpc/amqp.py", line 437, in _process_data
2022-02-02 10:40:53.221 76654 ERROR sysinv.openstack.common.rpc.amqp **args)
2022-02-02 10:40:53.221 76654 ERROR sysinv.openstack.common.rpc.amqp File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/rpc/dispatcher.py", line 172, in dispatch
2022-02-02 10:40:53.221 76654 ERROR sysinv.openstack.common.rpc.amqp result = getattr(proxyobj, method)(ctxt, **kwargs)
2022-02-02 10:40:53.221 76654 ERROR sysinv.openstack.common.rpc.amqp File "/usr/lib64/python2.7/site-packages/sysinv/conductor/manager.py", line 8402, in create_controller_filesystems
2022-02-02 10:40:53.221 76654 ERROR sysinv.openstack.common.rpc.amqp raise exception.SysinvException("Disk size requirements not met.")
2022-02-02 10:40:53.221 76654 ERROR sysinv.openstack.common.rpc.amqp SysinvException: Disk size requirements not met.
2022-02-02 10:40:53.221 76654 ERROR sysinv.openstack.common.rpc.amqp
sysinv 2022-02-02 10:40:53.223 76654 ERROR sysinv.openstack.common.rpc.common [-] Returning exception Disk size requirements not met. to caller: SysinvException: Disk size requirements not met.
sysinv 2022-02-02 10:40:53.224 76654 ERROR sysinv.openstack.common.rpc.common [-] ['Traceback (most recent call last):\n', ' File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/rpc/amqp.py", line 437, in _process_data\n **args)\n', ' File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/rpc/dispatcher.py", line 172, in dispatch\n result = getattr(proxyobj, method)(ctxt, **kwargs)\...

Read more...

Revision history for this message
Elijah Mendez (ssyntaxerror) wrote : RE: [Bug 1959637] Re: Ansible Error - StarlingX AIO-DX Deployment
Download full text (8.7 KiB)

Hi Ramaswamy,

Thank you for the help identifying the problem, we had not realized the 500gb was a hard requirement. This bug can be closed we'll be pivoting to a different setup due to extenuating time and hardware constraints.

-Elijah

-----Original Message-----
From: <email address hidden> <email address hidden> On Behalf Of Ramaswamy Subramanian
Sent: February 3, 2022 4:42 PM
To: Elijah Mendez <email address hidden>
Subject: [Bug 1959637] Re: Ansible Error - StarlingX AIO-DX Deployment

[External Email]

It appears like disk is too small. Was there an attempt to increase the disk space allocated and attempt the installation again?

Below are details from the logs.

sysinv-api.log --> The API 'POST /v1/ihosts HTTP/1.0' is responding with status '500'.
---
sysinv 2022-02-02 10:40:53.234 79558 INFO sysinv.api.hooks.auditor [req-cfe7e9e1-1b81-40c3-ac20-1edb409e8ae8 bb57778856b248b895a9c07028e003ab 908189cfae0b4755b3638bb161bff250] ::ffff:127.0.0.1 "POST /v1/ihosts HTTP/1.0" status: 500 len: 730 time: 0.115689992905 POST: {u'tboot': u'false', u'console': u'tty0', u'mgmt_ip': u'192.168.204.2', u'install_output': u'text', u'hostname': u'controller-0', u'operational': u'disabled', u'invprovision': u'provisioning', u'boot_device': u'/dev/disk/by-path/pci-0000:01:01.0-scsi-0:0:0:0', u'rootfs_device': u'/dev/disk/by-path/pci-0000:01:01.0-scsi-0:0:0:0', u'administrative': u'locked', u'availability': u'offline', u'personality': u'controller', u'mgmt_mac': u'00:00:00:00:00:00'} host:127.0.0.1:6385 agent:Python-httplib2/0.9.2 (gzip) user: admin tenant: admin domain: Default

---

The processing of the API resulted in the below error.

---
sysinv 2022-02-02 10:40:53.220 76654 INFO sysinv.conductor.manager [-] Disk size : 60 ... disk too small sysinv 2022-02-02 10:40:53.221 76654 ERROR sysinv.openstack.common.rpc.amqp [-] Exception during message handling: SysinvException: Disk size requirements not met.
2022-02-02 10:40:53.221 76654 ERROR sysinv.openstack.common.rpc.amqp Traceback (most recent call last):
2022-02-02 10:40:53.221 76654 ERROR sysinv.openstack.common.rpc.amqp File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/rpc/amqp.py", line 437, in _process_data
2022-02-02 10:40:53.221 76654 ERROR sysinv.openstack.common.rpc.amqp **args)
2022-02-02 10:40:53.221 76654 ERROR sysinv.openstack.common.rpc.amqp File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/rpc/dispatcher.py", line 172, in dispatch
2022-02-02 10:40:53.221 76654 ERROR sysinv.openstack.common.rpc.amqp result = getattr(proxyobj, method)(ctxt, **kwargs)
2022-02-02 10:40:53.221 76654 ERROR sysinv.openstack.common.rpc.amqp File "/usr/lib64/python2.7/site-packages/sysinv/conductor/manager.py", line 8402, in create_controller_filesystems
2022-02-02 10:40:53.221 76654 ERROR sysinv.openstack.common.rpc.amqp raise exception.SysinvException("Disk size requirements not met.")
2022-02-02 10:40:53.221 76654 ERROR sysinv.openstack.common.rpc.amqp SysinvException: Disk size requirements not met.
2022-02-02 10:40:53.221 76654 ERROR sysinv.openstack.common.rpc.amqp sysinv 2022-02-02 10:40:53.223 76654 ERROR sysinv...

Read more...

Revision history for this message
Ghada Khalil (gkhalil) wrote :

Closing the LP based on the notes above. The system used does not meet the minimum hardware requirements for StarlingX.

Changed in starlingx:
importance: Undecided → Low
status: New → Invalid
assignee: nobody → Ramaswamy Subramanian (rsubrama)
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.