Tempest tries to run watcher, blazar, murano, cyborg, and monasca tests

[Andrea Ieri]

When we created the Tempest snap we decide to bundle all available Tempest plugins, just in case a user could need them. In turn this caused the Tempest rock to contain Tempest plugins for services that are not part of the Sunbeam collection (at least not at the moment).

Since Tempest appears to consider service availability to default to True, having - for example - a murano plugin installed will cause murano tests to run, unless murano isn't explicitly marked to be unavailable in tempest.conf.

The Tempest charm relies on python-tempestconf to generate a suitable tempest.conf; that project is however unable to determine the absence of the services mentioned in this bug title (see https://storyboard.openstack.org/#!/story/2011096).

The net result of all of the above is that in a current Sunbeam deployment the validation plugin will attempt to run tests that will never succeed (for example when using the smoke profile).

A temporary workaround would be to provide tempestconf with a deployer file that explicitly sets those services to be absent. The downside would of course be that the deployer file would need to be amended if we ever add any of the mentioned services as part of the Sunbeam charms.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: main
Review: https://review.opendev.org/c/openstack/sunbeam-charms/+/915460