InvocationError with no reason

[Eric Brown]

I created a profile with the following:

    gate:
        include:
# - any_other_function_with_shell_equals_true
# - assert_used
# - blacklist_calls
# - blacklist_import_func
# - blacklist_imports
# - exec_used
# - execute_with_run_as_root_equals_true
# - hardcoded_bind_all_interfaces
# - hardcoded_password
# - hardcoded_sql_expressions
# - hardcoded_tmp_directory
# - jinja2_autoescape_false
# - linux_commands_wildcard_injection
            - paramiko_calls
            - password_config_option_not_marked_secret
            - request_with_no_cert_validation
            - set_bad_file_permissions
# - subprocess_popen_with_shell_equals_true
# - subprocess_without_shell_equals_true
            - start_process_with_a_shell
            - start_process_with_no_shell
            - start_process_with_partial_path
            - ssl_with_bad_defaults
            - ssl_with_bad_version
            - ssl_with_no_version
            - try_except_pass
            - use_of_mako_templates

When bandit is run in tox with: bandit -c bandit.yaml -r nova -n 5 -p gate -ll

I get this error:

Files skipped (0):

Test results:

ERROR: InvocationError: '/home/ericwb/nova/.tox/bandit/bin/bandit -v -c bandit.yaml -r nova -n 5 -p gate -ll'
______________________________________________ summary _______________________________________________
ERROR: bandit: commands failed

Why did it not report the file and check that was performed?

If I take away the -ll, I get:

Run started:
 2015-07-29 05:37:14.447666

Files skipped (0):

Test results:

>> Issue: Starting a process without a shell.
   Severity: Low Confidence: Medium
   Location: nova/cmd/manage.py:1396
1395 try:
1396 os.execvp('sudo', ['sudo', '-u', '#%s' % st.st_uid] + sys.argv)
1397 except Exception:
1398 print(_('sudo failed, continuing as if nothing happened'))

ERROR: InvocationError: '/home/ericwb/nova/.tox/bandit/bin/bandit -c bandit.yaml -r nova -n 5 -p gate'
______________________________________________ summary _______________________________________________
ERROR: bandit: commands failed

[Ian Cordasco (icordasc)]

That is ... very odd. The InvocationError makes me think something in Bandit itself is failing and raising that, rather than being an error due to a problem in a checked file.

[Ian Cordasco (icordasc)]

So, the InvocationError comes from tox. This is likely because bandit is returning with a non-zero exit status and tox sees that as a Failure.

[Eric Brown (ericwb)]

No error when running bandit outside of tox:
http://paste.openstack.org/show/406243/

Error when running within tox:
http://paste.openstack.org/show/406245/

[Eric Brown (ericwb)]

I also tried modifying tox to use the latest bandit from git. Still occurs

http://paste.openstack.org/show/406251/

[Eric Brown (ericwb)]

To recreate:
- git clone https://github.com/openstack/nova.git
- modify the bandit.yaml with a new profile called 'gate' as indicated earlier in this bug
- modify tox.ini to run the new profile 'gate' (-p gate) in addition to -ll
- run tox -e bandit

Then remove the -ll from the bandit entry in tox.ini. Notice you'll get the output of a check.

[Travis McPeak (travis-mcpeak)]

This looks like it is caused by this bug: https://review.openstack.org/208629 . Basically even though you are filtering results and they aren't displayed the exit code still reflects that there were findings.

https://bugs.launchpad.net/bandit/+bug/1479216 <-- As soon as the fix for this lands, this should be fixed.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/208629
Committed: https://git.openstack.org/cgit/openstack/bandit/commit/?id=c6a7f7948b7b6a8d0368c82227191640427011ff
Submitter: Jenkins
Branch: master

commit c6a7f7948b7b6a8d0368c82227191640427011ff
Author: Travis McPeak <email address hidden>
Date: Mon Aug 3 13:57:00 2015 -0600

    Fixes exit code for filtered results

    This commit fixes the return code when using a severity filter.
    Prior to this commit the exit code would be '1' - indicating
    results even when there weren't results displayed. This was
    because the exit code is based on total results whereas the
    display is based on a filtered subset of the results.

    Rather than filtering the results earlier on, this commit leaves
    the full set of results available for formatters that may want it.
    It also adds filter functionality to the results counter that the
    main module uses to determine what the exit code should be.

    Change-Id: I0b29ccab31c0898ebe84121d37cb0478ac3e2da9
    Closes-Bug: #1480014
    Closes-Bug: #1479216

[Eric Brown (ericwb)]

Fix released in 0.13.0