Tox linters job randomly failing bashate in sysinv

Bug #1890287 reported by Al Bailey on 2020-08-04
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
StarlingX
Medium
Al Bailey

Bug Description

Brief Description
-----------------
When code is submitted for review in stx/config it may randomly fail zuul.
This is because bashate only aborts on some errors, or returns the last files success/failure for others.
Since bashate can process the files in random order, if the last file is a failure, zuul will fail, and if it is success, it returns success.

Severity
--------
Minor
(blocked submissions but there is a workaround)

Steps to Reproduce
------------------
Submit a code change in sysinv. It may randomly fail zuul.

Expected Behavior
------------------
Zuul should pass linters job consistently

Actual Behavior
----------------
Zuul is failing the linters job and therefore the review cannot merge.

Reproducibility
---------------
Seen Once

System Configuration
--------------------
N/A

Branch/Pull Time/Commit
-----------------------
master Aug 4, 2020

Last Pass
---------
Anytime. The bad file merged 12 days ago, but zuul could randomly pass anyways.

Timestamp/Logs
--------------
https://zuul.opendev.org/t/openstack/build/5f25f2200dec4b3b9aaf169437523375

...
Running bashate on /home/zuul/src/opendev.org/starlingx/config/sysinv/sysinv/sysinv/scripts/partition_info.sh
Running bashate on /home/zuul/src/opendev.org/starlingx/config/sysinv/sysinv/sysinv/scripts/kube-cert-rotation.sh
Running bashate on /home/zuul/src/opendev.org/starlingx/config/sysinv/sysinv/sysinv/scripts/validate-platform-backup.sh
/home/zuul/src/opendev.org/starlingx/config/sysinv/sysinv/sysinv/scripts/validate-platform-backup.sh:52:1: E004 File did not end with a newline
1 bashate error(s) found
ERROR: InvocationError for command /bin/bash -c 'find /home/zuul/src/opendev.org/starlingx/config -not \( -type d -name .?\* -prune \) -type f -not -name \*~ -not -name \*.md -name \*.sh -print0 | xargs -0 bashate -v -e E* -i E006,E010' (exited with code 123)

Test Activity
-------------
Submission to Zuul

Workaround
----------
Use 'recheck' and the next time it runs, the job may process the files in a different order and succeed

Ghada Khalil (gkhalil) on 2020-08-04
tags: added: stx.zuul
Changed in starlingx:
assignee: nobody → Al Bailey (albailey1974)
tags: added: stx.5.0
Changed in starlingx:
importance: Undecided → Medium

Reviewed: https://review.opendev.org/744712
Committed: https://git.openstack.org/cgit/starlingx/config/commit/?id=fc8b109ef0598534f1f137ee3d259cef396bfa03
Submitter: Zuul
Branch: master

commit fc8b109ef0598534f1f137ee3d259cef396bfa03
Author: albailey <email address hidden>
Date: Tue Aug 4 08:45:26 2020 -0500

    Fix a random tox linters failure in bashate

    There was a file that was missing a newline which would
    cause bashate to fail.
    However the bashate command was being run as a list of inputs
    rather than as individual files, so that error was being ignored
    unless that bad file was the last file processed.
    This submission fixes the file, and ensures that tox linters
    job will fail is any file has an error, regardless of what is
    processed afterwards.

    Change-Id: I7587b7017b6855aad63dfaeaea5ecca0bcfc21ea
    Closes-Bug: 1890287
    Signed-off-by: albailey <email address hidden>

Changed in starlingx:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers