novaclient stats all files in /usr/bin

Bug #1509500 reported by Eric Larese on 2015-10-23
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
neutron
Medium
Eric Larese
python-novaclient
Low
Andrey Kurilin

Bug Description

It appears that novaclient is searching Python's sys.path to find novaclient's own executable, and a side effect of this is an operating system security package will log hundreds of errors each time this happens. For example, this stack trace:

  /usr/lib/python2.7/site-packages/neutron/manager.py(244)get_plugin()
-> return weakref.proxy(cls.get_instance().plugin)
  /usr/lib/python2.7/site-packages/neutron/manager.py(238)get_instance()
-> cls._create_instance()
  /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py(252)inner()
-> return f(*args, **kwargs)
  /usr/lib/python2.7/site-packages/neutron/manager.py(224)_create_instance()
-> cls._instance = cls()
  /usr/lib/python2.7/site-packages/neutron/manager.py(120)__init__()
-> plugin_provider)
  /usr/lib/python2.7/site-packages/neutron/manager.py(157)_get_plugin_instance()
-> return plugin_class()
  /usr/lib/python2.7/site-packages/neutron/quota/resource_registry.py(121)wrapper()
-> return f(*args, **kwargs)
  /usr/lib/python2.7/site-packages/neutron/plugins/ml2/plugin.py(145)__init__()
-> super(Ml2Plugin, self).__init__()
  /usr/lib/python2.7/site-packages/neutron/db/db_base_plugin_v2.py(103)__init__()
-> self.nova_notifier = nova.Notifier()
  /usr/lib/python2.7/site-packages/neutron/notifiers/nova.py(98)__init__()
-> ext for ext in nova_client.discover_extensions(NOVA_API_VERSION)
> /usr/lib/python2.7/site-packages/novaclient/client.py(724)discover_extensions()
-> _discover_via_contrib_path(version)

This stack trace is during neutron server startup, a novaclient call is made which results in _discover_via_python_path() being invoked here: https://github.com/openstack/python-novaclient/blob/master/novaclient/client.py#L723

This method uses pkgutil.iter_modules() which will search all of /usr/bin (among many other places). An operating system security package such as SELinux on RedHat will log hundreds of errors like this to /var/log/audit/audit.log:

type=AVC msg=audit(10/23/2015 15:41:08.766:368903) : avc: denied { getattr } for pid=13716 comm=neutron-server path=/usr/bin/virsh dev="dm-5" ino=138258059 scontext=system_u:system_r:neutron_t:s0 tcontext=system_u:object_r:virsh_exec_t:s0 tclass=file

One error is logged for every searched file in /usr/bin, about 1,300 messages each time neutron-server restarts on my test system. This generates a huge amount of noise in audit.log. I have not attempted to reproduce this with Ubuntu / AppArmor to verify if the issue is the same.

Is this something the novaclient code would worry about? Is there some way I could submit a patch to fix this?

Changed in python-novaclient:
status: New → In Progress
importance: Undecided → Low
assignee: nobody → Andrey Kurilin (andreykurilin)

Reviewed: https://review.openstack.org/280725
Committed: https://git.openstack.org/cgit/openstack/python-novaclient/commit/?id=c18ccb1bfae574b4b496c138e9192fc737ed9c20
Submitter: Jenkins
Branch: master

commit c18ccb1bfae574b4b496c138e9192fc737ed9c20
Author: Andrey Kurilin <email address hidden>
Date: Tue Feb 16 15:35:57 2016 +0200

    Add a way to discover only contrib extensions

    Several OS projects(cinder, neutron, osc...) use
    `novaclient.discover_extensions` for initialization novaclient.client.Client
    with novaclient.v2.contrib extensions. In this case, it would be nice to
    provide a way to not discover extension via python path an entry-point.

    Change-Id: I030f4c55c2795c7f7973f5f12e54b9819c4a5578
    Closes-Bug: #1509500

Changed in python-novaclient:
status: In Progress → Fix Released

This issue was fixed in the openstack/python-novaclient 3.3.0 release.

Fix proposed to branch: master
Review: https://review.openstack.org/287449

Changed in neutron:
assignee: nobody → Eric Larese (erlarese)
status: New → In Progress

This bug is > 180 days without activity. We are unsetting assignee and milestone and setting status to Incomplete in order to allow its expiry in 60 days.

If the bug is still valid, then update the bug status.

Changed in neutron:
assignee: Eric Larese (erlarese) → nobody
status: In Progress → Incomplete
Changed in neutron:
status: Incomplete → In Progress
assignee: nobody → Eric Larese (erlarese)
Changed in neutron:
milestone: none → ocata-rc1
tags: added: ocata-rc-potential
Changed in neutron:
importance: Undecided → Medium

Reviewed: https://review.openstack.org/287449
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=1db573ff12172f0a278b7318fe8094527e2bc72a
Submitter: Jenkins
Branch: master

commit 1db573ff12172f0a278b7318fe8094527e2bc72a
Author: Eric Larese <email address hidden>
Date: Wed Mar 2 15:41:36 2016 -0500

    Use only_contrib option for nova_client calls

    Use the only_contrib option that was added by
    I030f4c55c2795c7f7973f5f12e54b9819c4a5578 to speed up nova_client calls
    and skip the search for nova_client extensions to reduce
    /var/log/audit noise.

    Change-Id: Ic97b342a3633ffdf05b02ddd81baad88e1605a75
    Closes-Bug: #1509500

Changed in neutron:
status: In Progress → Fix Released

Reviewed: https://review.openstack.org/430317
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=0fb35ea59068b58be7e922314e66b530ffdea2ee
Submitter: Jenkins
Branch: stable/ocata

commit 0fb35ea59068b58be7e922314e66b530ffdea2ee
Author: Eric Larese <email address hidden>
Date: Wed Mar 2 15:41:36 2016 -0500

    Use only_contrib option for nova_client calls

    Use the only_contrib option that was added by
    I030f4c55c2795c7f7973f5f12e54b9819c4a5578 to speed up nova_client calls
    and skip the search for nova_client extensions to reduce
    /var/log/audit noise.

    Change-Id: Ic97b342a3633ffdf05b02ddd81baad88e1605a75
    Closes-Bug: #1509500
    (cherry picked from commit 1db573ff12172f0a278b7318fe8094527e2bc72a)

tags: added: in-stable-ocata

This issue was fixed in the openstack/neutron 10.0.0.0rc2 release candidate.

This issue was fixed in the openstack/neutron 11.0.0.0b1 development milestone.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers