Faiing tests when building Rocky rc1 Debian package

[Thomas Goirand]

Hi,

While building the congress Debian package (rocky rc1), I get the below failing tests. Could you please help me to fix them?

======================================================================
FAIL: congress.tests.api.test_datasource_model.TestDatasourceModel.test_add_datasource_with_custom_driver
congress.tests.api.test_datasource_model.TestDatasourceModel.test_add_datasource_with_custom_driver
----------------------------------------------------------------------
_StringException: pythonlogging:'': {{{
Heartbeat support requested but it is not supported by the kombu driver or the broker
Heartbeat support requested but it is not supported by the kombu driver or the broker
Heartbeat support requested but it is not supported by the kombu driver or the broker
Failed to load driver endpoint test=congress.tests.test_custom_driver:TestCustomDriver
Traceback (most recent call last):
  File "/<<BUILDDIR>>/congress-8.0.0~rc1+dfsg1/congress/dse2/dse_node.py", line 534, in load_custom_drivers
    ep_plugin = ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2317, in load
    self.require(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2340, in require
    items = working_set.resolve(reqs, env, installer, extras=self.extras)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 779, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (Babel 2.4.0 (/usr/lib/python3/dist-packages), Requirement.parse('Babel!=2.4.0,>=2.3.4'))
Registering congress datasource services on node testnode
Heartbeat support requested but it is not supported by the kombu driver or the broker
started datasource synchronizer on node testnode
Registering congress PolicyEngine service on node testnode
Heartbeat support requested but it is not supported by the kombu driver or the broker
started policy-rule synchronizer on node testnode
engine policies synchronized, added 0 removed 0
completed synchronization of policies
sync classification policy with DB
sync action policy with DB
Registering congress policy library service on node testnode
Heartbeat support requested but it is not supported by the kombu driver or the broker
Registering congress API service on node testnode
Registered API handler: VersionV1Handler(/v1[/]?$)
Registered API handler: CollectionHandler(/v1/policies$)
Registered API handler: ElementHandler(/v1/policies/(?P<policy_id>[^/]+)$)
Registered API handler: CollectionHandler(/v1/librarypolicies$)
Registered API handler: ElementHandler(/v1/librarypolicies/(?P<policy_id>[^/]+)$)
Registered API handler: CollectionHandler(/v1/policies/(?P<policy_id>[^/]+)/rules$)
Registered API handler: ElementHandler(/v1/policies/(?P<policy_id>[^/]+)/rules/(?P<rule_id>[^/]+)$)
Registered API handler: CollectionHandler(/v1/data-sources$)
Registered API handler: ElementHandler(/v1/data-sources/(?P<ds_id>[^/]+)$)
Registered API handler: ElementHandler(/v1/data-sources/(?P<ds_id>[^/]+)/schema$)
Registered API handler: ElementHandler(/v1/data-sources/(?P<ds_id>[^/]+)/tables/(?P<table_id>[^/]+)/spec$)
Registered API handler: CollectionHandler(/v1/data-sources/(?P<ds_id>[^/]+)/actions$)
Registered API handler: ElementHandler(/v1/data-sources/(?P<ds_id>[^/]+)/status$)
Registered API handler: ElementHandler(/v1/policies/(?P<policy_id>[^/]+)/status$)
Registered API handler: ElementHandler(/v1/policies/(?P<policy_id>[^/]+)/rules/(?P<rule_id>[^/]+)/status$)
Registered API handler: CollectionHandler((/v1/data-sources/(?P<ds_id>[^/]+)|/v1/policies/(?P<policy_id>[^/]+))/tables$)
Registered API handler: ElementHandler((/v1/data-sources/(?P<ds_id>[^/]+)|/v1/policies/(?P<policy_id>[^/]+))/tables/(?P<table_id>[^/]+)$)
Registered API handler: CollectionHandler((/v1/data-sources/(?P<ds_id>[^/]+)|/v1/policies/(?P<policy_id>[^/]+))/tables/(?P<table_id>[^/]+)/rows$)
Registered API handler: ElementHandler((/v1/data-sources/(?P<ds_id>[^/]+)|/v1/policies/(?P<policy_id>[^/]+))/tables/(?P<table_id>[^/]+)/rows/(?P<row_id>[^/]+)$)
Registered API handler: CollectionHandler(/v1/data-sources/(?P<ds_id>[^/]+)/webhook$)
Registered API handler: CollectionHandler(/v1/system/drivers$)
Registered API handler: ElementHandler(/v1/system/drivers/(?P<driver_id>[^/]+)$)
Heartbeat support requested but it is not supported by the kombu driver or the broker
tenant_name is deprecated, use project_name instead
creating service datasource1 with class <class 'congress.tests.fake_datasource.FakeDataSource'> and args {'name': 'datasource1', 'args': {'auth_url': 'foo', 'username': 'armax', 'password': '****', 'tenant_name': 'armax', 'ds_id': 'c34651d0-9228-470f-bdf5-ac48ca732803'}}
Heartbeat support requested but it is not supported by the kombu driver or the broker
sync datasource1 policy with DB
datasource1:: polling
fake:: update_from_datasource
datasource1:: finished polling
Heartbeat support requested but it is not supported by the kombu driver or the broker
Exception during message handling
Traceback (most recent call last):
  File "/<<BUILDDIR>>/congress-8.0.0~rc1+dfsg1/congress/dse2/dse_node.py", line 635, in validate_create_datasource
    loaded_driver = self.get_driver_info(driver)
  File "/<<BUILDDIR>>/congress-8.0.0~rc1+dfsg1/congress/dse2/dse_node.py", line 546, in get_driver_info
    raise exception.DriverNotFound(id=driver_name)
congress.exception.DriverNotFound: Driver not found test. Either not supported or disabled by deployer in Congress config

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming
    res = self.dispatcher.dispatch(message)
  File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/dispatcher.py", line 265, in dispatch
    return self._do_dispatch(endpoint, method, ctxt, args)
  File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch
    result = func(ctxt, **new_args)
  File "/<<BUILDDIR>>/congress-8.0.0~rc1+dfsg1/congress/dse2/datasource_manager.py", line 134, in add_datasource
    return self.ds_manager.add_datasource(items)
  File "/<<BUILDDIR>>/congress-8.0.0~rc1+dfsg1/congress/dse2/datasource_manager.py", line 53, in add_datasource
    self.node.validate_create_datasource(req)
  File "/<<BUILDDIR>>/congress-8.0.0~rc1+dfsg1/congress/dse2/dse_node.py", line 637, in validate_create_datasource
    raise exception.InvalidDriver(driver=req)
congress.exception.InvalidDriver: Invalid driver: {'id': '851347c1-f035-43c8-a491-26a9bd86887f', 'name': 'datasource_test_4', 'driver': 'test', 'description': 'hello world!', 'type': None, 'enabled': True, 'config': {'auth_url': 'foo', 'username': 'armax', 'password': '<hidden>', 'tenant_name': 'armax'}}
Stopping DSE node 'testnode'
killed datasource1 polling worker thread
}}}

raceback (most recent call last):
  File "/<<BUILDDIR>>/congress-8.0.0~rc1+dfsg1/congress/api/datasource_model.py", line 87, in add_item
    timeout=self.dse_long_timeout)
  File "/<<BUILDDIR>>/congress-8.0.0~rc1+dfsg1/congress/api/base.py", line 42, in invoke_rpc
    caller, name, kwargs, timeout=timeout, local=local)
  File "/<<BUILDDIR>>/congress-8.0.0~rc1+dfsg1/congress/dse2/data_service.py", line 192, in rpc
    service, action, kwargs, timeout=timeout, local=local, retry=retry)
  File "/<<BUILDDIR>>/congress-8.0.0~rc1+dfsg1/congress/dse2/dse_node.py", line 329, in invoke_service_rpc
    result = client.call(self.context, method, **kwargs)
  File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/client.py", line 505, in call
    return self.prepare().call(ctxt, method, **kwargs)
  File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/client.py", line 179, in call
    retry=self.retry)
  File "/usr/lib/python3/dist-packages/oslo_messaging/transport.py", line 133, in _send
    retry=retry)
  File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 584, in send
    call_monitor_timeout, retry=retry)
  File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 575, in _send
    raise result
congress.exception_Remote.InvalidDriver_Remote: Invalid driver: {'id': '851347c1-f035-43c8-a491-26a9bd86887f', 'name': 'datasource_test_4', 'driver': 'test', 'description': 'hello world!', 'type': None, 'enabled': True, 'config': {'auth_url': 'foo', 'username': 'armax', 'password': '<hidden>', 'tenant_name': 'armax'}}
Traceback (most recent call last):

  File "/<<BUILDDIR>>/congress-8.0.0~rc1+dfsg1/congress/dse2/dse_node.py", line 635, in validate_create_datasource
    loaded_driver = self.get_driver_info(driver)

  File "/<<BUILDDIR>>/congress-8.0.0~rc1+dfsg1/congress/dse2/dse_node.py", line 546, in get_driver_info
    raise exception.DriverNotFound(id=driver_name)

congress.exception.DriverNotFound: Driver not found test. Either not supported or disabled by deployer in Congress config

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming
    res = self.dispatcher.dispatch(message)

  File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/dispatcher.py", line 265, in dispatch
    return self._do_dispatch(endpoint, method, ctxt, args)

  File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch
    result = func(ctxt, **new_args)

  File "/<<BUILDDIR>>/congress-8.0.0~rc1+dfsg1/congress/dse2/datasource_manager.py", line 134, in add_datasource
    return self.ds_manager.add_datasource(items)

  File "/<<BUILDDIR>>/congress-8.0.0~rc1+dfsg1/congress/dse2/datasource_manager.py", line 53, in add_datasource
    self.node.validate_create_datasource(req)

  File "/<<BUILDDIR>>/congress-8.0.0~rc1+dfsg1/congress/dse2/dse_node.py", line 637, in validate_create_datasource
    raise exception.InvalidDriver(driver=req)

congress.exception.InvalidDriver: Invalid driver: {'id': '851347c1-f035-43c8-a491-26a9bd86887f', 'name': 'datasource_test_4', 'driver': 'test', 'description': 'hello world!', 'type': None, 'enabled': True, 'config': {'auth_url': 'foo', 'username': 'armax', 'password': '<hidden>', 'tenant_name': 'armax'}}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/<<BUILDDIR>>/congress-8.0.0~rc1+dfsg1/congress/tests/api/test_datasource_model.py", line 88, in test_add_datasource_with_custom_driver
    self.datasource_model.add_item(datasource4, {})
  File "/<<BUILDDIR>>/congress-8.0.0~rc1+dfsg1/congress/api/datasource_model.py", line 95, in add_item
    e.code, webservice.original_msg(e), http_status_code=e.code)
congress.api.webservice.DataModelException: Invalid driver: {'id': '851347c1-f035-43c8-a491-26a9bd86887f', 'name': 'datasource_test_4', 'driver': 'test', 'description': 'hello world!', 'type': None, 'enabled': True, 'config': {'auth_url': 'foo', 'username': 'armax', 'password': '<hidden>', 'tenant_name': 'armax'}}

======================================================================
FAIL: congress.tests.api.test_driver_model.TestDriverModel.test_drivers_list_with_custom_drivers
congress.tests.api.test_driver_model.TestDriverModel.test_drivers_list_with_custom_drivers
----------------------------------------------------------------------
_StringException: pythonlogging:'': {{{
Heartbeat support requested but it is not supported by the kombu driver or the broker
Heartbeat support requested but it is not supported by the kombu driver or the broker
Heartbeat support requested but it is not supported by the kombu driver or the broker
Registering congress datasource services on node testnode
Heartbeat support requested but it is not supported by the kombu driver or the broker
started datasource synchronizer on node testnode
Registering congress PolicyEngine service on node testnode
Heartbeat support requested but it is not supported by the kombu driver or the broker
started policy-rule synchronizer on node testnode
engine policies synchronized, added 0 removed 0
completed synchronization of policies
sync classification policy with DB
sync action policy with DB
Registering congress policy library service on node testnode
Heartbeat support requested but it is not supported by the kombu driver or the broker
Registering congress API service on node testnode
Registered API handler: VersionV1Handler(/v1[/]?$)
Registered API handler: CollectionHandler(/v1/policies$)
Registered API handler: ElementHandler(/v1/policies/(?P<policy_id>[^/]+)$)
Registered API handler: CollectionHandler(/v1/librarypolicies$)
Registered API handler: ElementHandler(/v1/librarypolicies/(?P<policy_id>[^/]+)$)
Registered API handler: CollectionHandler(/v1/policies/(?P<policy_id>[^/]+)/rules$)
Registered API handler: ElementHandler(/v1/policies/(?P<policy_id>[^/]+)/rules/(?P<rule_id>[^/]+)$)
Registered API handler: CollectionHandler(/v1/data-sources$)
Registered API handler: ElementHandler(/v1/data-sources/(?P<ds_id>[^/]+)$)
Registered API handler: ElementHandler(/v1/data-sources/(?P<ds_id>[^/]+)/schema$)
Registered API handler: ElementHandler(/v1/data-sources/(?P<ds_id>[^/]+)/tables/(?P<table_id>[^/]+)/spec$)
Registered API handler: CollectionHandler(/v1/data-sources/(?P<ds_id>[^/]+)/actions$)
Registered API handler: ElementHandler(/v1/data-sources/(?P<ds_id>[^/]+)/status$)
Registered API handler: ElementHandler(/v1/policies/(?P<policy_id>[^/]+)/status$)
Registered API handler: ElementHandler(/v1/policies/(?P<policy_id>[^/]+)/rules/(?P<rule_id>[^/]+)/status$)
Registered API handler: CollectionHandler((/v1/data-sources/(?P<ds_id>[^/]+)|/v1/policies/(?P<policy_id>[^/]+))/tables$)
Registered API handler: ElementHandler((/v1/data-sources/(?P<ds_id>[^/]+)|/v1/policies/(?P<policy_id>[^/]+))/tables/(?P<table_id>[^/]+)$)
Registered API handler: CollectionHandler((/v1/data-sources/(?P<ds_id>[^/]+)|/v1/policies/(?P<policy_id>[^/]+))/tables/(?P<table_id>[^/]+)/rows$)
Registered API handler: ElementHandler((/v1/data-sources/(?P<ds_id>[^/]+)|/v1/policies/(?P<policy_id>[^/]+))/tables/(?P<table_id>[^/]+)/rows/(?P<row_id>[^/]+)$)
Registered API handler: CollectionHandler(/v1/data-sources/(?P<ds_id>[^/]+)/webhook$)
Registered API handler: CollectionHandler(/v1/system/drivers$)
Registered API handler: ElementHandler(/v1/system/drivers/(?P<driver_id>[^/]+)$)
Heartbeat support requested but it is not supported by the kombu driver or the broker
Heartbeat support requested but it is not supported by the kombu driver or the broker
Heartbeat support requested but it is not supported by the kombu driver or the broker
data:: polling
fake:: update_from_datasource
data:: finished polling
Heartbeat support requested but it is not supported by the kombu driver or the broker
Heartbeat support requested but it is not supported by the kombu driver or the broker
Failed to load driver endpoint test=congress.tests.test_custom_driver:TestCustomDriver
Traceback (most recent call last):
  File "/<<BUILDDIR>>/congress-8.0.0~rc1+dfsg1/congress/dse2/dse_node.py", line 534, in load_custom_drivers
    ep_plugin = ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2317, in load
    self.require(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2340, in require
    items = working_set.resolve(reqs, env, installer, extras=self.extras)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 779, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (Babel 2.4.0 (/usr/lib/python3/dist-packages), Requirement.parse('Babel!=2.4.0,>=2.3.4'))
Registering congress datasource services on node test-node-2
Heartbeat support requested but it is not supported by the kombu driver or the broker
started datasource synchronizer on node test-node-2
Registering congress PolicyEngine service on node test-node-2
Heartbeat support requested but it is not supported by the kombu driver or the broker
started policy-rule synchronizer on node test-node-2
engine policies synchronized, added 2 removed 0
completed synchronization of policies
Registering congress policy library service on node test-node-2
Heartbeat support requested but it is not supported by the kombu driver or the broker
Registering congress API service on node test-node-2
Registered API handler: VersionV1Handler(/v1[/]?$)
Registered API handler: CollectionHandler(/v1/policies$)
Registered API handler: ElementHandler(/v1/policies/(?P<policy_id>[^/]+)$)
Registered API handler: CollectionHandler(/v1/librarypolicies$)
Registered API handler: ElementHandler(/v1/librarypolicies/(?P<policy_id>[^/]+)$)
Registered API handler: CollectionHandler(/v1/policies/(?P<policy_id>[^/]+)/rules$)
Registered API handler: ElementHandler(/v1/policies/(?P<policy_id>[^/]+)/rules/(?P<rule_id>[^/]+)$)
Registered API handler: CollectionHandler(/v1/data-sources$)
Registered API handler: ElementHandler(/v1/data-sources/(?P<ds_id>[^/]+)$)
Registered API handler: ElementHandler(/v1/data-sources/(?P<ds_id>[^/]+)/schema$)
Registered API handler: ElementHandler(/v1/data-sources/(?P<ds_id>[^/]+)/tables/(?P<table_id>[^/]+)/spec$)
Registered API handler: CollectionHandler(/v1/data-sources/(?P<ds_id>[^/]+)/actions$)
Registered API handler: ElementHandler(/v1/data-sources/(?P<ds_id>[^/]+)/status$)
Registered API handler: ElementHandler(/v1/policies/(?P<policy_id>[^/]+)/status$)
Registered API handler: ElementHandler(/v1/policies/(?P<policy_id>[^/]+)/rules/(?P<rule_id>[^/]+)/status$)
Registered API handler: CollectionHandler((/v1/data-sources/(?P<ds_id>[^/]+)|/v1/policies/(?P<policy_id>[^/]+))/tables$)
Registered API handler: ElementHandler((/v1/data-sources/(?P<ds_id>[^/]+)|/v1/policies/(?P<policy_id>[^/]+))/tables/(?P<table_id>[^/]+)$)
Registered API handler: CollectionHandler((/v1/data-sources/(?P<ds_id>[^/]+)|/v1/policies/(?P<policy_id>[^/]+))/tables/(?P<table_id>[^/]+)/rows$)
Registered API handler: ElementHandler((/v1/data-sources/(?P<ds_id>[^/]+)|/v1/policies/(?P<policy_id>[^/]+))/tables/(?P<table_id>[^/]+)/rows/(?P<row_id>[^/]+)$)
Registered API handler: CollectionHandler(/v1/data-sources/(?P<ds_id>[^/]+)/webhook$)
Registered API handler: CollectionHandler(/v1/system/drivers$)
Registered API handler: ElementHandler(/v1/system/drivers/(?P<driver_id>[^/]+)$)
Heartbeat support requested but it is not supported by the kombu driver or the broker
Heartbeat support requested but it is not supported by the kombu driver or the broker
data:: polling
fake:: update_from_datasource
data:: finished polling
}}}

Traceback (most recent call last):
  File "/<<BUILDDIR>>/congress-8.0.0~rc1+dfsg1/congress/tests/api/test_driver_model.py", line 62, in test_drivers_list_with_custom_drivers
    self.assertIn('test', ret)
  File "/usr/lib/python3/dist-packages/testtools/testcase.py", line 417, in assertIn
    self.assertThat(haystack, Contains(needle), message)
  File "/usr/lib/python3/dist-packages/testtools/testcase.py", line 498, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: 'test' not in ['aodh', 'cinder', 'cloudfoundryv2', 'config', 'doctor', 'fake_datasource', 'glancev2', 'heat', 'ironic', 'keystone', 'keystonev3', 'mistral', 'monasca', 'monasca_webhook', 'murano', 'neutronv2', 'neutronv2_qos', 'nova', 'plexxi', 'swift', 'vcenter', 'vitrage']

[Eric K (ekcs)]

Hi Thomas thanks for the report!

Looks like both failures are caused by a Babel version (2.4.0) that conflicts with Requirements [1]. 2.4.0 is blocked in global-requirements [2].

This line is particularly picky about meeting all the requirements: https://github.com/openstack/congress/blob/8.0.0.0rc1/congress/dse2/dse_node.py#L534

Would you suggest that we modify the code to tolerate this requirement mismatch? Thanks again!

[1]
Failed to load driver endpoint test=congress.tests.test_custom_driver:TestCustomDriver
Traceback (most recent call last):
  File "/<<BUILDDIR>>/congress-8.0.0~rc1+dfsg1/congress/dse2/dse_node.py", line 534, in load_custom_drivers
    ep_plugin = ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2317, in load
    self.require(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2340, in require
    items = working_set.resolve(reqs, env, installer, extras=self.extras)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 779, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (Babel 2.4.0 (/usr/lib/python3/dist-packages), Requirement.parse('Babel!=2.4.0,>=2.3.4'))

[2] https://github.com/openstack/requirements/blob/stable/rocky/global-requirements.txt#L11

[Thomas Goirand (thomas-goirand)]

As much as Debian is concerned, I don't mind anymore, because Buster has version 2.6.0, which isn't conflicting anymore.

[Eric K (ekcs)]

Got it thanks for the update, Thomas!