| 2023-01-11 17:31:13 |
Fabiano Correa Mercer |
description |
Brief Description
-----------------
Subcloud came up non-operational, e.g., unable to acquire Keystone administrative privileges ($ source /etc/platform/openrc), after the initial deployment.
Severity
--------
Minor.
Steps to Reproduce
------------------
Deploy 250 subclouds in parallel
Expected Behavior
------------------
subcloud439 online/unlock/operational after the deployment
Actual Behavior
----------------
subcloud439 offline/non-operational after the deployment.
Reproducibility
---------------
1 out of 500
System Configuration
--------------------
Distributed Cloud - IPv6
Branch/Pull Time/Commit
-----------------------
N/A
Last Pass
---------
2022-11-29_22-00-05
Timestamp/Logs
--------------
2022-12-12T18:45:33.533 ip-10-229-177-192 sysinv-api[53042]: info Dec 12 18:45:33 ERROR: Unable to run system show, trying direct request on sysinv-api URL (sysinv-api) 2022-12-12T18:45:33.677 ip-10-229-177-192 sysinv-api[53042]: info Dec 12 18:45:33 ERROR: Unable to communicate with the System Inventory Service (sysinv-api)
daemon-cfg.log:
2022-12-12T20:35:48.166 controller-0 OCF_IPaddr2(management-ip)[440833]: info INFO: IP status = no, IP_CIP=
...
2022-12-12T20:35:48.553 controller-0 OCF_IPaddr2(management-ip)[441642]: info INFO: Adding inet6 address 2620:10a:a001:ac12::36e2/123 to device ens6 (with preferred_lft forever)
2022-12-12T20:35:48.559 controller-0 OCF_IPaddr2(management-ip)[441642]: err ERROR: RTNETLINK answers: File exists
2022-12-12T20:35:48.563 controller-0 OCF_IPaddr2(management-ip)[441642]: err ERROR: Failed to add 2620:10a:a001:ac12::36e2
Test Activity
-------------
Scalability Testing
Workaround
----------
Reboot the subcloud |
Brief Description
-----------------
In rare situations, the add_interface may fail with RTNETLINK error.
Add logs to help the investigation if this error occurs again.
Log to show the device link status and all IP address configured for
the specific device.
Add log for IP address deletion from device, to be sure that
ipaddr2 start/stop sequence were executed.
Severity
--------
Minor.
Steps to Reproduce
------------------
Deploy 250 subclouds in parallel
Expected Behavior
------------------
subcloud439 online/unlock/operational after the deployment
Actual Behavior
----------------
subcloud439 offline/non-operational after the deployment.
Reproducibility
---------------
1 out of 2000
System Configuration
--------------------
Distributed Cloud - IPv6
Branch/Pull Time/Commit
-----------------------
N/A
Last Pass
---------
2022-11-29_22-00-05
Timestamp/Logs
--------------
2022-12-12T18:45:33.533 ip-10-229-177-192 sysinv-api[53042]: info Dec 12 18:45:33 ERROR: Unable to run system show, trying direct request on sysinv-api URL (sysinv-api) 2022-12-12T18:45:33.677 ip-10-229-177-192 sysinv-api[53042]: info Dec 12 18:45:33 ERROR: Unable to communicate with the System Inventory Service (sysinv-api)
daemon-cfg.log:
2022-12-12T20:35:48.166 controller-0 OCF_IPaddr2(management-ip)[440833]: info INFO: IP status = no, IP_CIP=
...
2022-12-12T20:35:48.553 controller-0 OCF_IPaddr2(management-ip)[441642]: info INFO: Adding inet6 address 2620:10a:a001:ac12::36e2/123 to device ens6 (with preferred_lft forever)
2022-12-12T20:35:48.559 controller-0 OCF_IPaddr2(management-ip)[441642]: err ERROR: RTNETLINK answers: File exists
2022-12-12T20:35:48.563 controller-0 OCF_IPaddr2(management-ip)[441642]: err ERROR: Failed to add 2620:10a:a001:ac12::36e2
Test Activity
-------------
Scalability Testing
Workaround
----------
Reboot the subcloud |
|
