topology implementation assumes existence of swift topology file

[Jonathan Maron]

The topology helper assumes that a cluster has a swift implementation that requires rack awareness support. However, there are instances in which an openstack deployment does not have swift configured/deployed. The correct behavior would be to check for the existence of a swift data file and only then add that information to the generated topology mapping.

The stack trace seen when no swift mapping file is provided (again, for the situation in which swift in not available in the OpenStack deployment) is:

2013-12-04 17:17:27.100 11904 DEBUG savanna.utils.remote [-] [test2-slave-003] _execute_command took 1.9 seconds to complete _log_command /Users/jmaron/dev/workspaces/cinder/savanna/savanna/utils/remote.py:385
2013-12-04 17:17:27.964 11904 ERROR savanna.service.api [-] Can't configure cluster 'test2' (reason: Unable to find file etc/savanna/swift.topology with Swift topology)
2013-12-04 17:17:27.964 11904 TRACE savanna.service.api Traceback (most recent call last):
2013-12-04 17:17:27.964 11904 TRACE savanna.service.api File "/Users/jmaron/dev/workspaces/cinder/savanna/savanna/service/api.py", line 159, in _provision_cluster
2013-12-04 17:17:27.964 11904 TRACE savanna.service.api plugin.configure_cluster(cluster)
2013-12-04 17:17:27.964 11904 TRACE savanna.service.api File "/Users/jmaron/dev/workspaces/cinder/savanna/savanna/plugins/hdp/ambariplugin.py", line 238, in configure_cluster
2013-12-04 17:17:27.964 11904 TRACE savanna.service.api self.create_cluster(cluster)
2013-12-04 17:17:27.964 11904 TRACE savanna.service.api File "/Users/jmaron/dev/workspaces/cinder/savanna/savanna/plugins/hdp/ambariplugin.py", line 72, in create_cluster
2013-12-04 17:17:27.964 11904 TRACE savanna.service.api self._configure_topology_for_cluster(cluster, servers)
2013-12-04 17:17:27.964 11904 TRACE savanna.service.api File "/Users/jmaron/dev/workspaces/cinder/savanna/savanna/plugins/hdp/ambariplugin.py", line 43, in _configure_topology_for_cluster
2013-12-04 17:17:27.964 11904 TRACE savanna.service.api cluster, CONF.enable_hypervisor_awareness)
2013-12-04 17:17:27.964 11904 TRACE savanna.service.api File "/Users/jmaron/dev/workspaces/cinder/savanna/savanna/topology/topology_helper.py", line 126, in generate_topology_map
2013-12-04 17:17:27.964 11904 TRACE savanna.service.api topology_mapping.update(_read_swift_topology())
2013-12-04 17:17:27.964 11904 TRACE savanna.service.api File "/Users/jmaron/dev/workspaces/cinder/savanna/savanna/topology/topology_helper.py", line 80, in _read_swift_topology
2013-12-04 17:17:27.964 11904 TRACE savanna.service.api "Unable to find file %s with Swift topology")
2013-12-04 17:17:27.964 11904 TRACE savanna.service.api NotFoundException: Unable to find file etc/savanna/swift.topology with Swift topology

[Alexander Kuznetsov (akuznetsov)]

You can leave empty etc/savanna/swift.topology file, in case if you have no Swift or don't know topology for it.

[Jonathan Maron (jmaron)]

That's a workaround, but you still agree that this is a bug, correct?

[Matthew Farrellee (mattf)]

Why would we want to disable Swift?

How about we stop making Swift optional and require all plugins to have it enabled by default?

[Andrew Lazarev (alazarev)]

Matt, we can want to enable data locality for nodes only, but not for swift.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/97376

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/97376
Committed: https://git.openstack.org/cgit/openstack/sahara/commit/?id=85707cda6b3f182a287148770b6ba365ea895852
Submitter: Jenkins
Branch: master

commit 85707cda6b3f182a287148770b6ba365ea895852
Author: Andrew Lazarev <email address hidden>
Date: Mon Jun 2 17:21:08 2014 -0700

    Made Swift topology optional for data locality

    Change-Id: I80d33543cc2173dd62beab17f811f0f8e162a078
    Closes-Bug: #1257945