Validation for security groups can fail for valid groups

[Chad Roberts]

From validations/base.py

def check_security_groups_exist(security_groups):
    security_group_list = nova.client().security_groups.list()
    allowed_groups = set(reduce(
        operator.add, [[sg.id, sg.name] for sg in security_group_list], []))
    for sg in security_groups:
        if sg not in allowed_groups:
            raise ex.InvalidException(_("Security group '%s' not found") % sg)

The above fails with Security group '2' not found.

The input is security_groups = [u'2'] STRING 2
But allowed groups is [u'default', 2] INT 2
So, '2' is never found in the allowed_groups.

Based on an IRC conversation, it should be acceptable to cast the ids to str() so that we don't improperly report an invalid security group.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/130576

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/130576
Committed: https://git.openstack.org/cgit/openstack/sahara/commit/?id=fc1b5c8463248b8d142f46f310afae0645312b62
Submitter: Jenkins
Branch: master

commit fc1b5c8463248b8d142f46f310afae0645312b62
Author: Chad Roberts <email address hidden>
Date: Thu Oct 23 11:38:27 2014 -0400

    Fixing validation exception for valid security group

    Now casting the id of the allowed security group to a
    six.text_type() to avoid type mismatching when using ids.

    Change-Id: Iff2e442501d9c85c2b73c9a36114728830b04701
    Closes-Bug: #1384743

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/juno
Review: https://review.openstack.org/132265

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/132265
Committed: https://git.openstack.org/cgit/openstack/sahara/commit/?id=35b881f9336a8e57fd2204e83b6c58c3b23e9d74
Submitter: Jenkins
Branch: stable/juno

commit 35b881f9336a8e57fd2204e83b6c58c3b23e9d74
Author: Chad Roberts <email address hidden>
Date: Thu Oct 23 11:38:27 2014 -0400

    Fixing validation exception for valid security group

    Now casting the id of the allowed security group to a
    six.text_type() to avoid type mismatching when using ids.

    Change-Id: Iff2e442501d9c85c2b73c9a36114728830b04701
    Closes-Bug: #1384743
    (cherry picked from commit fc1b5c8463248b8d142f46f310afae0645312b62)

[Nastooh (navesta)]

Did “pip install sahara –upgrade” to get the latest Sahara, and while I don’t see any problems during node template creation; however, during cluster instantiation, similar problem is observed:

2014-12-08 22:50:26.473 4724 INFO sahara.cli.sahara_all [-] 10.0.0.86 - - [08/Dec/2014 22:50:26] "GET /v1.1/926c31c887f441f6a4e4b8031b8cc528/clusters/1fa6aa6b-52ab-4836-8771-accb4d183fdb HTTP/1.1" 200 2338 0.318082
2014-12-08 22:50:26.643 4724 INFO sahara.utils.general [-] Cluster status has been changed: id=1fa6aa6b-52ab-4836-8771-accb4d183fdb, New status=Spawning
2014-12-08 22:50:27.936 4724 ERROR sahara.service.ops [-] Error during operating cluster 'tst' (reason: Security group 2 not found for project 926c31c887f441f6a4e4b8031b8cc528. (HTTP 400))
2014-12-08 22:50:27.936 4724 TRACE sahara.service.ops Traceback (most recent call last):
2014-12-08 22:50:27.936 4724 TRACE sahara.service.ops File "/usr/local/lib/python2.7/dist-packages/sahara/service/ops.py", line 113, in wrapper
2014-12-08 22:50:27.936 4724 TRACE sahara.service.ops f(cluster_id, *args, **kwds)
2014-12-08 22:50:27.936 4724 TRACE sahara.service.ops File "/usr/local/lib/python2.7/dist-packages/sahara/service/ops.py", line 198, in _provision_cluster
2014-12-08 22:50:27.936 4724 TRACE sahara.service.ops INFRA.create_cluster(cluster)
2014-12-08 22:50:27.936 4724 TRACE sahara.service.ops File "/usr/local/lib/python2.7/dist-packages/sahara/service/direct_engine.py", line 51, in create_cluster
2014-12-08 22:50:27.936 4724 TRACE sahara.service.ops self._create_instances(cluster)
2014-12-08 22:50:27.936 4724 TRACE sahara.service.ops File "/usr/local/lib/python2.7/dist-packages/sahara/service/direct_engine.py", line 168, in _create_instances
2014-12-08 22:50:27.936 4724 TRACE sahara.service.ops self._run_instance(cluster, node_group, idx, aa_group=aa_group)
2014-12-08 22:50:27.936 4724 TRACE sahara.service.ops File "/usr/local/lib/python2.7/dist-packages/sahara/service/direct_engine.py", line 319, in _run_instance
2014-12-08 22:50:27.936 4724 TRACE sahara.service.ops **nova_kwargs)
2014-12-08 22:50:27.936 4724 TRACE sahara.service.ops File "/usr/lib/python2.7/dist-packages/novaclient/v1_1/servers.py", line 883, in create
2014-12-08 22:50:27.936 4724 TRACE sahara.service.ops **boot_kwargs)
2014-12-08 22:50:27.936 4724 TRACE sahara.service.ops File "/usr/lib/python2.7/dist-packages/novaclient/v1_1/servers.py", line 546, in _boot
2014-12-08 22:50:27.936 4724 TRACE sahara.service.ops return_raw=return_raw, **kwargs)
2014-12-08 22:50:27.936 4724 TRACE sahara.service.ops File "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 100, in _create
2014-12-08 22:50:27.936 4724 TRACE sahara.service.ops _resp, body = self.api.client.post(url, body=body)
2014-12-08 22:50:27.936 4724 TRACE sahara.service.ops File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 490, in post
2014-12-08 22:50:27.936 4724 TRACE sahara.service.ops return self._cs_request(url, 'POST', **kwargs)
2014-12-08 22:50:27.936 4724 TRACE sahara.service.ops File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 465, in _cs_request
2014-12-08 22:50:27.936 4...

[Dmitry (i-like-pie)]

Hello,

I am afraid the proposed correction is not valid. The validation you have corrected is the one that is done during template creation from the dashboard, however in the template the security group ID is still saved as an integer. Later, during cluster creation, when the template is used another validation is pefrormed that fails with the same mistake.

My suggestion is to either not use the ID at all and just stick with the name, or save the ID as a string value in the template.
(first option used successfuly at https://ask.openstack.org/en/question/55161/juno-sahara-spark-100-security-group-error/)

[Andrew Lazarev (alazarev)]

There is also https://bugs.launchpad.net/sahara/+bug/1381035