AttributeError: 'module' object has no attribute 'convert_to_boolean'

[Tong Liu]

With latest neutron master code, neutron service q-svc could start due to the following error:
2016-11-08 21:54:39.435 DEBUG oslo_concurrency.lockutils [-] Lock "manager" released by "neutron.manager._create_instance" :: held 1.467s from (pid=18534) inner /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:282
2016-11-08 21:54:39.435 ERROR neutron.service [-] Unrecoverable error: please check log for details.
2016-11-08 21:54:39.435 TRACE neutron.service Traceback (most recent call last):
2016-11-08 21:54:39.435 TRACE neutron.service File "/opt/stack/neutron/neutron/service.py", line 87, in serve_wsgi
2016-11-08 21:54:39.435 TRACE neutron.service service.start()
2016-11-08 21:54:39.435 TRACE neutron.service File "/opt/stack/neutron/neutron/service.py", line 63, in start
2016-11-08 21:54:39.435 TRACE neutron.service self.wsgi_app = _run_wsgi(self.app_name)
2016-11-08 21:54:39.435 TRACE neutron.service File "/opt/stack/neutron/neutron/service.py", line 289, in _run_wsgi
2016-11-08 21:54:39.435 TRACE neutron.service app = config.load_paste_app(app_name)
2016-11-08 21:54:39.435 TRACE neutron.service File "/opt/stack/neutron/neutron/common/config.py", line 125, in load_paste_app
2016-11-08 21:54:39.435 TRACE neutron.service app = loader.load_app(app_name)
2016-11-08 21:54:39.435 TRACE neutron.service File "/usr/local/lib/python2.7/dist-packages/oslo_service/wsgi.py", line 353, in load_app
2016-11-08 21:54:39.435 TRACE neutron.service return deploy.loadapp("config:%s" % self.config_path, name=name)
2016-11-08 21:54:39.435 TRACE neutron.service File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
2016-11-08 21:54:39.435 TRACE neutron.service return loadobj(APP, uri, name=name, **kw)
2016-11-08 21:54:39.435 TRACE neutron.service File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 272, in loadobj
2016-11-08 21:54:39.435 TRACE neutron.service return context.create()
2016-11-08 21:54:39.435 TRACE neutron.service File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 710, in create
2016-11-08 21:54:39.435 TRACE neutron.service return self.object_type.invoke(self)
2016-11-08 21:54:39.435 TRACE neutron.service File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 144, in invoke
2016-11-08 21:54:39.435 TRACE neutron.service **context.local_conf)
2016-11-08 21:54:39.435 TRACE neutron.service File "/usr/local/lib/python2.7/dist-packages/paste/deploy/util.py", line 55, in fix_call
2016-11-08 21:54:39.435 TRACE neutron.service val = callable(*args, **kw)
2016-11-08 21:54:39.435 TRACE neutron.service File "/usr/local/lib/python2.7/dist-packages/paste/urlmap.py", line 31, in urlmap_factory
2016-11-08 21:54:39.435 TRACE neutron.service app = loader.get_app(app_name, global_conf=global_conf)
2016-11-08 21:54:39.435 TRACE neutron.service File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 350, in get_app
2016-11-08 21:54:39.435 TRACE neutron.service name=name, global_conf=global_conf).create()
2016-11-08 21:54:39.435 TRACE neutron.service File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 710, in create
2016-11-08 21:54:39.435 TRACE neutron.service return self.object_type.invoke(self)
2016-11-08 21:54:39.435 TRACE neutron.service File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 144, in invoke
2016-11-08 21:54:39.435 TRACE neutron.service **context.local_conf)
2016-11-08 21:54:39.435 TRACE neutron.service File "/usr/local/lib/python2.7/dist-packages/paste/deploy/util.py", line 55, in fix_call
2016-11-08 21:54:39.435 TRACE neutron.service val = callable(*args, **kw)
2016-11-08 21:54:39.435 TRACE neutron.service File "/opt/stack/neutron/neutron/auth.py", line 71, in pipeline_factory
2016-11-08 21:54:39.435 TRACE neutron.service app = loader.get_app(pipeline[-1])
2016-11-08 21:54:39.435 TRACE neutron.service File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 350, in get_app
2016-11-08 21:54:39.435 TRACE neutron.service name=name, global_conf=global_conf).create()
2016-11-08 21:54:39.435 TRACE neutron.service File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 710, in create
2016-11-08 21:54:39.435 TRACE neutron.service return self.object_type.invoke(self)
2016-11-08 21:54:39.435 TRACE neutron.service File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 146, in invoke
2016-11-08 21:54:39.435 TRACE neutron.service return fix_call(context.object, context.global_conf, **context.local_conf)
2016-11-08 21:54:39.435 TRACE neutron.service File "/usr/local/lib/python2.7/dist-packages/paste/deploy/util.py", line 55, in fix_call
2016-11-08 21:54:39.435 TRACE neutron.service val = callable(*args, **kw)
2016-11-08 21:54:39.435 TRACE neutron.service File "/opt/stack/neutron/neutron/api/v2/router.py", line 72, in factory
2016-11-08 21:54:39.435 TRACE neutron.service return cls(**local_config)
2016-11-08 21:54:39.435 TRACE neutron.service File "/opt/stack/neutron/neutron/api/v2/router.py", line 76, in __init__
2016-11-08 21:54:39.435 TRACE neutron.service plugin = manager.NeutronManager.get_plugin()
2016-11-08 21:54:39.435 TRACE neutron.service File "/opt/stack/neutron/neutron/manager.py", line 246, in get_plugin
2016-11-08 21:54:39.435 TRACE neutron.service return weakref.proxy(cls.get_instance().plugin)
2016-11-08 21:54:39.435 TRACE neutron.service File "/opt/stack/neutron/neutron/manager.py", line 240, in get_instance
2016-11-08 21:54:39.435 TRACE neutron.service cls._create_instance()
2016-11-08 21:54:39.435 TRACE neutron.service File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 271, in inner
2016-11-08 21:54:39.435 TRACE neutron.service return f(*args, **kwargs)
2016-11-08 21:54:39.435 TRACE neutron.service File "/opt/stack/neutron/neutron/manager.py", line 226, in _create_instance
2016-11-08 21:54:39.435 TRACE neutron.service cls._instance = cls()
2016-11-08 21:54:39.435 TRACE neutron.service File "/opt/stack/neutron/neutron/manager.py", line 127, in __init__
2016-11-08 21:54:39.435 TRACE neutron.service plugin_provider)
2016-11-08 21:54:39.435 TRACE neutron.service File "/opt/stack/neutron/neutron/manager.py", line 161, in _get_plugin_instance
2016-11-08 21:54:39.435 TRACE neutron.service plugin_class = self.load_class_for_provider(namespace, plugin_provider)
2016-11-08 21:54:39.435 TRACE neutron.service File "/opt/stack/neutron/neutron/manager.py", line 155, in load_class_for_provider
2016-11-08 21:54:39.435 TRACE neutron.service plugin_provider)
2016-11-08 21:54:39.435 TRACE neutron.service File "/opt/stack/neutron/neutron/manager.py", line 161, in _get_plugin_instance
2016-11-08 21:54:39.435 TRACE neutron.service plugin_class = self.load_class_for_provider(namespace, plugin_provider)
2016-11-08 21:54:39.435 TRACE neutron.service File "/opt/stack/neutron/neutron/manager.py", line 155, in load_class_for_provider
2016-11-08 21:54:39.435 TRACE neutron.service plugin_provider)
2016-11-08 21:54:39.435 TRACE neutron.service File "/opt/stack/neutron/neutron/common/utils.py", line 368, in load_class_by_alias_or_classname
2016-11-08 21:54:39.435 TRACE neutron.service class_to_load = importutils.import_class(name)
2016-11-08 21:54:39.435 TRACE neutron.service File "/usr/local/lib/python2.7/dist-packages/oslo_utils/importutils.py", line 30, in import_class
2016-11-08 21:54:39.435 TRACE neutron.service __import__(mod_str)
2016-11-08 21:54:39.435 TRACE neutron.service File "/opt/stack/vmware-nsx/vmware_nsx/plugin.py", line 20, in <module>
2016-11-08 21:54:39.435 TRACE neutron.service from vmware_nsx.plugins.nsx_v import plugin as nsx_v
2016-11-08 21:54:39.435 TRACE neutron.service File "/opt/stack/vmware-nsx/vmware_nsx/plugins/nsx_v/plugin.py", line 91, in <module>
2016-11-08 21:54:39.435 TRACE neutron.service from vmware_nsx.db import extended_security_group as extended_secgroup
2016-11-08 21:54:39.435 TRACE neutron.service File "/opt/stack/vmware-nsx/vmware_nsx/db/extended_security_group.py", line 35, in <module>
2016-11-08 21:54:39.435 TRACE neutron.service from vmware_nsx.extensions import securitygrouplogging as sg_logging
2016-11-08 21:54:39.435 TRACE neutron.service File "/opt/stack/vmware-nsx/vmware_nsx/extensions/securitygrouplogging.py", line 25, in <module>
2016-11-08 21:54:39.435 TRACE neutron.service 'convert_to': attributes.convert_to_boolean,
2016-11-08 21:54:39.435 TRACE neutron.service File "/opt/stack/neutron/neutron/common/_deprecate.py", line 126, in __getattr__
2016-11-08 21:54:39.435 TRACE neutron.service _("'module' object has no attribute '%s'") % name)
2016-11-08 21:54:39.435 TRACE neutron.service AttributeError: 'module' object has no attribute 'convert_to_boolean'
2016-11-08 21:54:39.435 TRACE neutron.service
2016-11-08 21:54:39.440 CRITICAL neutron [-] AttributeError: 'module' object has no attribute 'convert_to_boolean'

[Tong Liu (liutong)]

It seems this issue is caused by the following two patches from neutron repo:
commit 04bee5befc785cb44eb8c0284776867e4eba518b
Merge: 7c59522 6d85a51
Author: Jenkins <email address hidden>
Date: Tue Nov 8 20:19:54 2016 +0000

    Merge "Removed deprecation warning for converters and validators"

commit 7c59522790d83d04c4bfe96508e784b9a809a8e9
Merge: 4752418 e43eb80
Author: Jenkins <email address hidden>
Date: Tue Nov 8 20:16:40 2016 +0000

    Merge "Removed deprecation warning for waiting_until_true"

After I reset to commit 475241834dae0d9e252e693dfca35fbff1513a57, then neutron server can start.

[Brian Haley (brian-haley)]

Those methods had been marked for deprecation and were finally removed, the NSX plugin needs to be updated to use the neutron-lib code instead.

[Kostiantyn Volenbovskyi (kostiantyn-volenbovskyi)]

Hi,
It sounds that fault is specific to VMware NSX agent that uses attributes module for validation instead of using of using neutron-lib

See
https://github.com/openstack/vmware-nsx/blob/24be12ff27b8142de735cbd7c09ea4546c1cd5b4/vmware_nsx/extensions/securitygrouplogging.py#L25
while now it is supposed to use converters from neutron-lib, see for example

https://github.com/openstack/vmware-nsx/blob/24be12ff27b8142de735cbd7c09ea4546c1cd5b4/vmware_nsx/extensions/providersecuritygroup.py#L32

So the commit mentioned above
commit 04bee5befc785cb44eb8c0284776867e4eba518b
Merge: 7c59522 6d85a51
Author: Jenkins <email address hidden>
Date: Tue Nov 8 20:19:54 2016 +0000

    Merge "Removed deprecation warning for converters and validators"
removes deprecation and forces to use neutron-lib

[John Davidge (john-davidge)]

This is also affecting neutron-dynamic-routing, which I believe tracks bugs via neutron.

[Jean-Philippe Evrard (jean-philippe-evrard)]

neutron-dynamic-routing is installed by default in our neutron venvs, so we are impacted by this issue too.

[Jean-Philippe Evrard (jean-philippe-evrard)]

(another log linked to this issue: http://paste.openstack.org/show/588529/ )

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/395556

[YAMAMOTO Takashi (yamamoto)]

https://review.openstack.org/#/c/395450/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/395556
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible-os_neutron/commit/?id=d4dd9df137877756507a5fcfa033d94f0f743429
Submitter: Jenkins
Branch: master

commit d4dd9df137877756507a5fcfa033d94f0f743429
Author: Jesse Pretorius <email address hidden>
Date: Wed Nov 9 11:51:07 2016 +0000

    Move optional plugin installs out of the venv

    The optionally enabled plugins do not need to be included in the
    neutron venv, but should be added to the venv when enabled.

    This prevents clashes between plugins and unnecessary db
    migrations, including migrations which may not be working due to
    changes in neutron core which haven't been applied to the plugin
    yet.

    Additionally, to unblock the CentOS-7 check, this patch includes
    adding a task to remove any problem packages on the target host.

    Related-Bug: #1640319
    Co-Authored-By: Donovan Francesco <email address hidden>
    Change-Id: I7dcab3f3d56118b93b9a5d74065234acb620ae22

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/newton
Review: https://review.openstack.org/395718

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/395718
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible-os_neutron/commit/?id=3d5740a7ff2fb3a52fc240f0eea76ea8086c6ac9
Submitter: Jenkins
Branch: stable/newton

commit 3d5740a7ff2fb3a52fc240f0eea76ea8086c6ac9
Author: Jesse Pretorius <email address hidden>
Date: Wed Nov 9 11:51:07 2016 +0000

    Move optional plugin installs out of the venv

    The optionally enabled plugins do not need to be included in the
    neutron venv, but should be added to the venv when enabled.

    This prevents clashes between plugins and unnecessary db
    migrations, including migrations which may not be working due to
    changes in neutron core which haven't been applied to the plugin
    yet.

    Combined backport of:
    - https://review.openstack.org/395556
    - https://review.openstack.org/395570

    Related-Bug: #1640319
    Change-Id: I7dcab3f3d56118b93b9a5d74065234acb620ae22
    (cherry picked from commit d4dd9df137877756507a5fcfa033d94f0f743429)

[Kostiantyn Volenbovskyi (kostiantyn-volenbovskyi)]

For VMware NSX, the fix was done in https://review.openstack.org/#/c/395414/. I think that he could assign bug to himself?

And I think that tag like Closes-Bug: #1640319 should have been present in that commit

[Adit Sarfaty (asarfaty)]

Fixed by https://review.openstack.org/#/c/395414/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/395405
Committed: https://git.openstack.org/cgit/openstack/neutron-dynamic-routing/commit/?id=635a4df5887e72fdc5b27b0c9e054ab6cb6b0e93
Submitter: Jenkins
Branch: master

commit 635a4df5887e72fdc5b27b0c9e054ab6cb6b0e93
Author: fumihiko kakuma <email address hidden>
Date: Wed Nov 9 13:51:02 2016 +0900

    Fix no attribute error for 'convert_to_boolean'

    convert_to_boolean has been moved from neutron.api.v2.attributes to
    neutron_lib.api.converters.

    Partial-Bug: #1640319
    Change-Id: Id894814c8fbdabd80b708add01c2d99be1e10d6a

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/395450
Committed: https://git.openstack.org/cgit/openstack/networking-midonet/commit/?id=3dfc497daf26f5b8ebce6569e72e40371138b0cc
Submitter: Jenkins
Branch: master

commit 3dfc497daf26f5b8ebce6569e72e40371138b0cc
Author: YAMAMOTO Takashi <email address hidden>
Date: Wed Nov 9 16:51:22 2016 +0900

    Use some definitions from neutron-lib

    The following two unrelated changes are combined
    into this single commit to pass the gate.

    -------------------------------

    Use some definitions from neutron-lib

    Including:
        - convert_to_boolean
        - PORT_STATUS_ACTIVE
        - TAP_DEVICE_PREFIX
        - db_api.get_engine

    Closes-Bug: #1640319
    Closes-Bug: #1641481
    Closes-Bug: #1641494
    Partial-Bug: #1581361

    -------------------------------

    l3_db_midonet: Update after the recent Neutron change. [1]

    [1] I93e4d847f96707b17c4a7dfdb3bbf81d062fe3fb

    Closes-Bug: #1643747
    Related-Bug: #1540844

    -------------------------------

    Change-Id: Ide96124c12c993a3ee0f2353c8201da9a6b5890c

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/networking-midonet 4.0.0 release.