StarlingX

application-remove of user application failed

Bug #1836922 reported by Peng Peng on 2019-07-17

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	StarlingX	Fix Released	High	Bob Church

Bug Description

Brief Description
-----------------
as title

Severity
--------
Major

Steps to Reproduce
------------------
apply application to system
Remove applied app

TC-name: z_containers/test_custom_containers.py::test_launch_app_via_sysinv

Expected Behavior
------------------

Actual Behavior
----------------

Reproducibility
---------------
Reproducible

System Configuration
--------------------
Two node system
Multi-node system

Lab-name: IP_1-4/ WCP_63-66

Branch/Pull Time/Commit
-----------------------
stx master as of 20190717T013000Z

Last Pass
---------
20190715T233000Z

Timestamp/Logs
--------------
[2019-07-17 08:41:27,801] 301 DEBUG MainThread ssh.send :: Send 'system --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.2:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne application-list'
[2019-07-17 08:41:29,359] 423 DEBUG MainThread ssh.expect :: Output:
+---------------------+--------------------------------+-------------------------------+--------------------+---------+-----------+
| application | version | manifest name | manifest file | status | progress |
+---------------------+--------------------------------+-------------------------------+--------------------+---------+-----------+
| hello-kitty | 1.0 | hello-kitty | manifest.yaml | applied | completed |
| platform-integ-apps | 1.0-7 | platform-integration-manifest | manifest.yaml | applied | completed |
| stx-openstack | 1.0-17-centos-stable-versioned | armada-manifest | stx-openstack.yaml | applied | completed |
+---------------------+--------------------------------+-------------------------------+--------------------+---------+-----------+
[sysadmin@controller-1 ~(keystone_admin)]$
[2019-07-17 08:41:29,359] 301 DEBUG MainThread ssh.send :: Send 'echo $?'
[2019-07-17 08:41:29,462] 423 DEBUG MainThread ssh.expect :: Output:
0
[sysadmin@controller-1 ~(keystone_admin)]$
[2019-07-17 08:41:29,463] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_1_4
[2019-07-17 08:41:29,463] 466 DEBUG MainThread ssh.exec_cmd:: Executing command...
[2019-07-17 08:41:29,463] 301 DEBUG MainThread ssh.send :: Send 'system --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.2:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne application-remove hello-kitty'
[2019-07-17 08:41:31,002] 423 DEBUG MainThread ssh.expect :: Output:
+---------------+----------------------------------+
| Property | Value |
+---------------+----------------------------------+
| active | True |
| app_version | 1.0 |
| created_at | 2019-07-17T08:40:13.080450+00:00 |
| manifest_file | manifest.yaml |
| manifest_name | hello-kitty |
| name | hello-kitty |
| progress | None |
| status | removing |
| updated_at | 2019-07-17T08:41:06.149033+00:00 |
+---------------+----------------------------------+

[2019-07-17 08:41:37,764] 301 DEBUG MainThread ssh.send :: Send 'system --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.2:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne application-list'
[2019-07-17 08:41:39,372] 423 DEBUG MainThread ssh.expect :: Output:
+---------------------+--------------------------------+-------------------------------+--------------------+---------------+------------------------------------------+
| application | version | manifest name | manifest file | status | progress |
+---------------------+--------------------------------+-------------------------------+--------------------+---------------+------------------------------------------+
| hello-kitty | 1.0 | hello-kitty | manifest.yaml | remove-failed | operation aborted, check logs for detail |

In /var/log/armada/hello-kitty-delete.log
2019-07-17 10:57:19.213 98 ERROR armada.cli [-] Caught unexpected exception: FileNotFoundError: [Errno 2] No such file or directory: '/manifests/hello-kitty/1.0/hello-kitty-manifest-del.yaml'
2019-07-17 10:57:19.213 98 ERROR armada.cli Traceback (most recent call last):
2019-07-17 10:57:19.213 98 ERROR armada.cli File "/usr/local/lib/python3.6/dist-packages/armada/cli/__init__.py", line 39, in safe_invoke
2019-07-17 10:57:19.213 98 ERROR armada.cli self.invoke()
2019-07-17 10:57:19.213 98 ERROR armada.cli File "/usr/local/lib/python3.6/dist-packages/armada/cli/delete.py", line 99, in invoke
2019-07-17 10:57:19.213 98 ERROR armada.cli self.handle(tiller)
2019-07-17 10:57:19.213 98 ERROR armada.cli File "/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py", line 79, in func_wrapper
2019-07-17 10:57:19.213 98 ERROR armada.cli return future.result()
2019-07-17 10:57:19.213 98 ERROR armada.cli File "/usr/lib/python3.6/concurrent/futures/_base.py", line 425, in result
2019-07-17 10:57:19.213 98 ERROR armada.cli return self.__get_result()
2019-07-17 10:57:19.213 98 ERROR armada.cli File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
2019-07-17 10:57:19.213 98 ERROR armada.cli raise self._exception
2019-07-17 10:57:19.213 98 ERROR armada.cli File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
2019-07-17 10:57:19.213 98 ERROR armada.cli result = self.fn(*self.args, **self.kwargs)
2019-07-17 10:57:19.213 98 ERROR armada.cli File "/usr/local/lib/python3.6/dist-packages/armada/cli/delete.py", line 125, in handle
2019-07-17 10:57:19.213 98 ERROR armada.cli with open(self.manifest) as f:
2019-07-17 10:57:19.213 98 ERROR armada.cli FileNotFoundError: [Errno 2] No such file or directory: '/manifests/h2019-07-17 10:57:19.213 98 ERROR armada.cli [-] Caught unexpected exception: FileNotFoundError: [Errno 2] No such file or directory: '/manifests/hello-kitty/1.0/hello-kitty-manifest-del.yaml'
2019-07-17 10:57:19.213 98 ERROR armada.cli Traceback (most recent call last):
2019-07-17 10:57:19.213 98 ERROR armada.cli File "/usr/local/lib/python3.6/dist-packages/armada/cli/__init__.py", line 39, in safe_invoke
2019-07-17 10:57:19.213 98 ERROR armada.cli self.invoke()
2019-07-17 10:57:19.213 98 ERROR armada.cli File "/usr/local/lib/python3.6/dist-packages/armada/cli/delete.py", line 99, in invoke
2019-07-17 10:57:19.213 98 ERROR armada.cli self.handle(tiller)
2019-07-17 10:57:19.213 98 ERROR armada.cli File "/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py", line 79, in func_wrapper
2019-07-17 10:57:19.213 98 ERROR armada.cli return future.result()
2019-07-17 10:57:19.213 98 ERROR armada.cli File "/usr/lib/python3.6/concurrent/futures/_base.py", line 425, in result
2019-07-17 10:57:19.213 98 ERROR armada.cli return self.__get_result()
2019-07-17 10:57:19.213 98 ERROR armada.cli File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
2019-07-17 10:57:19.213 98 ERROR armada.cli raise self._exception
2019-07-17 10:57:19.213 98 ERROR armada.cli File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
2019-07-17 10:57:19.213 98 ERROR armada.cli result = self.fn(*self.args, **self.kwargs)
2019-07-17 10:57:19.213 98 ERROR armada.cli File "/usr/local/lib/python3.6/dist-packages/armada/cli/delete.py", line 125, in handle
2019-07-17 10:57:19.213 98 ERROR armada.cli with open(self.manifest) as f:
2019-07-17 10:57:19.213 98 ERROR armada.cli FileNotFoundError: [Errno 2] No such file or directory: '/manifests/hello-kitty/1.0/hello-kitty-manifest-del.yaml'
2019-07-17 10:57:19.213 98 ERROR armada.cli ^[[00mello-kitty/1.0/hello-kitty-manifest-del.yaml'
2019-07-17 10:57:19.213 98 ERROR armada.cli ^[[00m

Test Activity
-------------
Sanity

Tags:

Revision history for this message

Peng Peng (ppeng) wrote on 2019-07-17:

ALL_NODES_20190717.150848.tar Edit (23.7 MiB, application/x-tar)

Revision history for this message

Frank Miller (sensfan22) wrote on 2019-07-17:

Issue introduced by recent commit. Marking as stx.2.0 gating.

Changed in starlingx:
importance:	Undecided → High
assignee:	nobody → Bob Church (rchurch)
tags:	added: stx.2.0 stx.containers
summary:	- application-remove failed + application-remove of user application failed

Ghada Khalil (gkhalil) on 2019-07-17

Changed in starlingx:
status:	New → Confirmed
status:	Confirmed → Triaged

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-07-18: Fix proposed to config (master)

Fix proposed to branch: master
Review: https://review.opendev.org/671376

Changed in starlingx:
status:	Triaged → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-07-18: Fix merged to config (master)

Reviewed: https://review.opendev.org/671376
Committed: https://git.openstack.org/cgit/starlingx/config/commit/?id=156a254ef851229825dce0b2e6d020e0d67d8347
Submitter: Zuul
Branch: master

commit 156a254ef851229825dce0b2e6d020e0d67d8347
Author: Robert Church <email address hidden>
Date: Wed Jul 17 14:14:45 2019 -0400

Fix generic application deletions

Extend the use of the ArmadaManifestOperator to generic applications
even though they are not delivered with helm plugin support.

This will provide a unified approach to using a deletion manifest for
all applications during 'armada delete' operations.

    Change-Id: I5a23cdbc940c7fbe1a51a026728893b90a4bdb85
    Closes-Bug: #1836922
    Signed-off-by: Robert Church <email address hidden>

Changed in starlingx:
status:	In Progress → Fix Released

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Bug attachments

ALL_NODES_20190717.150848.tar Edit

Add attachment

Remote bug watches

Bug watches keep track of this bug in other bug trackers.