Wrong armada image in use

Bug #1928994 reported by Thiago Paiva Brito
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
StarlingX
Fix Released
Medium
Thiago Paiva Brito

Bug Description

Brief Description
-----------------
After the merge of https://review.opendev.org/c/starlingx/integ/+/783083 we are still seeing some instances of the evicted pods being waited forever on manual tests. Investigating the issue we figured that despite those changes, the wrong version of the armada image (2 years old) is still in use.

Severity
--------
Minor

Steps to Reproduce
------------------
Apply stx-openstack on AIO-SX

Expected Behavior
------------------
Application is succeeded

Actual Behavior
----------------
Application fails by timeout waiting on Evicted pods

Reproducibility
---------------
Intermittend: 1 out of 4

System Configuration
--------------------
AIO-SX, AIO-DX, Standard

Timestamp/Logs
--------------
kubectl describe on the armada pod shows

```
  Containers:
   armada-api:
    Image: registry.local:9001/quay.io/airshipit/armada:8a1638098f88d92bf799ef4934abe569789b885e-ubuntu_bionic
```

Test Activity
-------------
Developer Testing

Workaround
----------
- "kubectl delete" evicted pods and reapply
- Remove and reapply the application

description: updated
Changed in starlingx:
assignee: nobody → Thiago Paiva Brito (outbrito)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ansible-playbooks (master)
Changed in starlingx:
status: New → In Progress
Ghada Khalil (gkhalil)
tags: added: stx.containers
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to platform-armada-app (master)

Reviewed: https://review.opendev.org/c/starlingx/platform-armada-app/+/793972
Committed: https://opendev.org/starlingx/platform-armada-app/commit/7d30242557401bfdb0be5f75bd29ce05708d250f
Submitter: "Zuul (22348)"
Branch: master

commit 7d30242557401bfdb0be5f75bd29ce05708d250f
Author: Thiago Brito <email address hidden>
Date: Tue Jun 1 10:47:33 2021 -0300

    Fix chart syntax to comply with Go 1.14

    When testing patch [1] it was found that the usage of parenthesis on
    this chart doesn't comply with the Tiller implementation on Go 1.14.
    This fix will allow the upversion of Tiller to > 2.16.8 which is
    required for the Armada upversioning.

    [1] https://review.opendev.org/c/starlingx/ansible-playbooks/+/792297

    Partial-Bug: #1928994
    Signed-off-by: Thiago Brito <email address hidden>
    Change-Id: I8532036cd70b3e20a143dceba19119779413eb39

Revision history for this message
Ghada Khalil (gkhalil) wrote :

Screening: marking as stx.6.0 / medium priority as the issue frequency is rather high at 25%. This should be fixed in the master branch.

Changed in starlingx:
importance: Undecided → Medium
tags: added: stx.6.0
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ansible-playbooks (master)

Reviewed: https://review.opendev.org/c/starlingx/ansible-playbooks/+/792297
Committed: https://opendev.org/starlingx/ansible-playbooks/commit/fcde313c6b8c91ae76eb71f2963c7928db6cbec2
Submitter: "Zuul (22348)"
Branch: master

commit fcde313c6b8c91ae76eb71f2963c7928db6cbec2
Author: Thiago Brito <email address hidden>
Date: Wed May 19 19:23:43 2021 -0300

    Update armada image version to 7ef4b86

    After the merge of [1] we are still seeing some instances of the armada
    waiting indefinitely on Evicted pods until stx-openstack fails to apply.
    Investigating this further, I figured that the armada pods were me
    still using the old armada version due to the version of the images
    declared on these ansible playbooks.

    This commit upversion the Armada image to 7ef4b86 and Tiller to 2.16.9.

    Tested in a custom build in conjunction with [2], I was able to
    correctly deploy StarlingX and apply stx-openstack.

    [1] https://review.opendev.org/c/starlingx/integ/+/783083
    [2] https://review.opendev.org/c/starlingx/platform-armada-app/+/793972

    Depends-On: I8532036cd70b3e20a143dceba19119779413eb39
    Closes-bug: #1928994
    Signed-off-by: Thiago Brito <email address hidden>
    Change-Id: Id06d805229b55a471a52c3173b2aa8b62957fcf9

Changed in starlingx:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to platform-armada-app (r/stx.5.0)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ansible-playbooks (r/stx.5.0)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to platform-armada-app (r/stx.5.0)

Reviewed: https://review.opendev.org/c/starlingx/platform-armada-app/+/805812
Committed: https://opendev.org/starlingx/platform-armada-app/commit/8f770bdddac396d53fdd740d69610d6fca8fcd65
Submitter: "Zuul (22348)"
Branch: r/stx.5.0

commit 8f770bdddac396d53fdd740d69610d6fca8fcd65
Author: Thiago Brito <email address hidden>
Date: Tue Jun 1 10:47:33 2021 -0300

    Fix chart syntax to comply with Go 1.14

    When testing patch [1] it was found that the usage of parenthesis on
    this chart doesn't comply with the Tiller implementation on Go 1.14.
    This fix will allow the upversion of Tiller to > 2.16.8 which is
    required for the Armada upversioning.

    [1] https://review.opendev.org/c/starlingx/ansible-playbooks/+/792297

    Partial-Bug: #1928994
    Signed-off-by: Thiago Brito <email address hidden>
    Change-Id: I8532036cd70b3e20a143dceba19119779413eb39
    (cherry picked from commit 7d30242557401bfdb0be5f75bd29ce05708d250f)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ansible-playbooks (r/stx.5.0)

Reviewed: https://review.opendev.org/c/starlingx/ansible-playbooks/+/805813
Committed: https://opendev.org/starlingx/ansible-playbooks/commit/1d637ea6273f6b303801bee271db81b5598338c5
Submitter: "Zuul (22348)"
Branch: r/stx.5.0

commit 1d637ea6273f6b303801bee271db81b5598338c5
Author: Thiago Brito <email address hidden>
Date: Wed May 19 19:23:43 2021 -0300

    Update armada image version to 7ef4b86

    After the merge of [1] we are still seeing some instances of the armada
    waiting indefinitely on Evicted pods until stx-openstack fails to apply.
    Investigating this further, I figured that the armada pods were me
    still using the old armada version due to the version of the images
    declared on these ansible playbooks.

    This commit upversion the Armada image to 7ef4b86 and Tiller to 2.16.9.

    Tested in a custom build in conjunction with [2], I was able to
    correctly deploy StarlingX and apply stx-openstack.

    [1] https://review.opendev.org/c/starlingx/integ/+/783083
    [2] https://review.opendev.org/c/starlingx/platform-armada-app/+/793972

    Depends-On: I8532036cd70b3e20a143dceba19119779413eb39
    Closes-bug: #1928994
    Signed-off-by: Thiago Brito <email address hidden>
    Change-Id: Id06d805229b55a471a52c3173b2aa8b62957fcf9
    (cherry picked from commit fcde313c6b8c91ae76eb71f2963c7928db6cbec2)

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.