Appfwk operation fails due to AttributeError in RPC dispatcher

Bug #1991701 reported by Leonardo Fagundes Luz Serrano
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
StarlingX
Fix Released
Medium
Leonardo Fagundes Luz Serrano

Bug Description

Brief Description
-----------------
App operation (apply, remove) fails with following message on sysinv.log:

"AttributeError: 'ConductorManager' object has no attribute 'RPC_API_NAMESPACE'"
File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/rpc/dispatcher.py", line 151, in dispatch

Oddly, the failure point has a catch clause specifically for AttributeError exceptions
which should be preventing this error.

App will often get stuck with an 'apply-failed' or 'remove-failed' state.

Severity
--------
Major: System/Feature is usable but degraded

Steps to Reproduce
------------------
Intermittent. Most often seen when trying to apply a large app.

Expected Behavior
------------------
App operation does not fail due to this error

Actual Behavior
----------------
Operation fails

Reproducibility
---------------
Intermittent, but seen quite often.

System Configuration
--------------------
DX ipv4

Branch/Pull Time/Commit
-----------------------
BUILD_DATE="2022-09-13 18:10:00 -0400"

Last Pass
---------
This bug might be associated to the transition from Armada to FluxCD around May or June 2022

Timestamp/Logs
--------------
AttributeError: 'ConductorManager' object has no attribute 'RPC_API_NAMESPACE'
4163:2022-09-20 22:43:37.161 175829 ERROR sysinv.conductor.kube_app Traceback (most recent call last):
4164:2022-09-20 22:43:37.161 175829 ERROR sysinv.conductor.kube_app File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/rpc/dispatcher.py", line 151, in dispatch
4165:2022-09-20 22:43:37.161 175829 ERROR sysinv.conductor.kube_app cb_namespace = proxyobj.RPC_API_NAMESPACE
4166:2022-09-20 22:43:37.161 175829 ERROR sysinv.conductor.kube_app AttributeError: 'ConductorManager' object has no attribute 'RPC_API_NAMESPACE'
4167:2022-09-20 22:43:37.161 175829 ERROR sysinv.conductor.kube_app

Test Activity
-------------
[Sanity, Feature Testing, Regression Testing, Developer Testing, Evaluation, Other - Please specify]

Workaround
----------
system application-remove --force <app>
(Might need to delete the app as well with application-delete)
Then re-apply

Changed in starlingx:
status: New → In Progress
Ghada Khalil (gkhalil)
tags: added: stx.apps stx.containers
Changed in starlingx:
assignee: nobody → Leonardo Fagundes Luz Serrano (lfagunde)
importance: Undecided → Medium
tags: added: stx.8.0
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to config (master)

Reviewed: https://review.opendev.org/c/starlingx/config/+/860103
Committed: https://opendev.org/starlingx/config/commit/015df527fed192b8eb10609309241472cc8c8ac1
Submitter: "Zuul (22348)"
Branch: master

commit 015df527fed192b8eb10609309241472cc8c8ac1
Author: Leonardo Fagundes Luz Serrano <email address hidden>
Date: Mon Oct 3 12:13:23 2022 -0300

    Fixed AttributeError handling in rpc dispatcher

    Exception handling is failing to catch an AttributeError
    exception. Code snippet is replaced here by an equivalent
    approach using a built-in python function.

    The message always comes coupled with another error message.
    It is an artifact of the appfwk and provides
    no information about the root cause or how to debug it.
    Should be removed to avoid confusion.

    Test Plan:
    PASS Following steps to reproduce the message no longer
         trigger it.

    Closes-Bug: 1991701
    Signed-off-by: Leonardo Fagundes Luz Serrano <email address hidden>
    Change-Id: Ia37cf4c5340b58e7f318683669eaf35871558134

Changed in starlingx:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.