Comment 0 for bug 2002346

Revision history for this message
Fabiano Correa Mercer (fcorream) wrote : AWS subcloud not operational after the initial deployment

Brief Description
-----------------
Subcloud came up non-operational, e.g., unable to acquire Keystone administrative privileges ($ source /etc/platform/openrc), after the initial deployment.

Severity
--------
Minor.

Steps to Reproduce
------------------
Deploy 250 subclouds in parallel

Expected Behavior
------------------
subcloud439 online/unlock/operational after the deployment

Actual Behavior
----------------
subcloud439 offline/non-operational after the deployment.

Reproducibility
---------------
1 out of 500

System Configuration
--------------------
Distributed Cloud - IPv6

Branch/Pull Time/Commit
-----------------------
N/A

Last Pass
---------
2022-11-29_22-00-05

Timestamp/Logs
--------------
2022-12-12T18:45:33.533 ip-10-229-177-192 sysinv-api[53042]: info Dec 12 18:45:33 ERROR: Unable to run system show, trying direct request on sysinv-api URL (sysinv-api) 2022-12-12T18:45:33.677 ip-10-229-177-192 sysinv-api[53042]: info Dec 12 18:45:33 ERROR: Unable to communicate with the System Inventory Service (sysinv-api)

daemon-cfg.log:

2022-12-12T20:35:48.166 controller-0 OCF_IPaddr2(management-ip)[440833]: info INFO: IP status = no, IP_CIP=
...
2022-12-12T20:35:48.553 controller-0 OCF_IPaddr2(management-ip)[441642]: info INFO: Adding inet6 address 2620:10a:a001:ac12::36e2/123 to device ens6 (with preferred_lft forever)
2022-12-12T20:35:48.559 controller-0 OCF_IPaddr2(management-ip)[441642]: err ERROR: RTNETLINK answers: File exists
2022-12-12T20:35:48.563 controller-0 OCF_IPaddr2(management-ip)[441642]: err ERROR: Failed to add 2620:10a:a001:ac12::36e2

Test Activity
-------------
Scalability Testing

Workaround
----------
Reboot the subcloud