Refstack client fails with SSL Error for defcore URL of tests even with insecure provided

[Imran Hayder]

While trying to run defcore list of tests URL with refstack , i get insecure error even i have provided insecure to refstack CLI.

(.venv)CI-2:~/workspace/refstack_cloud_compute/refstack-client # ./refstack-client test --insecure -c '.tempest/etc/tempest.conf' -v --test-list "https://refstack.openstack.org/api/v1/guidelines/next/tests?target=object&type=required&alias=true&flag=false"
  InsecurePlatformWarning
/var/lib/jenkins/workspace/refstack_cloud_compute/refstack-client/.venv/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:838: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/security.html
  InsecureRequestWarning)
2016-10-13 09:24:03,283 refstack_client:403 INFO Starting Tempest test...
2016-10-13 09:24:03,283 refstack_client:413 INFO Normalizing test list...
Option "verbose" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future.
/var/lib/jenkins/workspace/refstack_cloud_compute/refstack-client/.venv/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
2016-10-13 09:24:17,558 refstack_client.list_parser:109 ERROR Error reading the passed in test list file.
Traceback (most recent call last):
  File "./refstack-client", line 31, in <module>
    raise SystemExit(getattr(test, args.func)())
  File "/var/lib/jenkins/workspace/refstack_cloud_compute/refstack-client/refstack_client/refstack_client.py", line 417, in test
    list_file = parser.create_whitelist(self.args.test_list)
  File "/var/lib/jenkins/workspace/refstack_cloud_compute/refstack-client/refstack_client/list_parser.py", line 219, in create_whitelist
    normalized_list = open(self.get_normalized_test_list(list_location),
  File "/var/lib/jenkins/workspace/refstack_cloud_compute/refstack-client/refstack_client/list_parser.py", line 203, in get_normalized_test_list
    base_test_ids = self._get_base_test_ids_from_list_file(list_location)
  File "/var/lib/jenkins/workspace/refstack_cloud_compute/refstack-client/refstack_client/list_parser.py", line 95, in _get_base_test_ids_from_list_file
    response = requests.get(list_location)
  File "/var/lib/jenkins/workspace/refstack_cloud_compute/refstack-client/.venv/lib/python2.7/site-packages/requests/api.py", line 70, in get
    return request('get', url, params=params, **kwargs)
  File "/var/lib/jenkins/workspace/refstack_cloud_compute/refstack-client/.venv/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/var/lib/jenkins/workspace/refstack_cloud_compute/refstack-client/.venv/lib/python2.7/site-packages/requests/sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "/var/lib/jenkins/workspace/refstack_cloud_compute/refstack-client/.venv/lib/python2.7/site-packages/requests/sessions.py", line 596, in send
    r = adapter.send(request, **kwargs)
  File "/var/lib/jenkins/workspace/refstack_cloud_compute/refstack-client/.venv/lib/python2.7/site-packages/requests/adapters.py", line 497, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: [Errno 1] _ssl.c:523: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

It should respect "insecure" even when trying to retrieve the tests via a HTTPS based URL

[Catherine Diep (cdiep)]

The RefStack team is investigating the failure condition. Thanks for opening thus big!

[Catherine Diep (cdiep)]

Imran, Thanks for reporting the error. We are investigating the error. Meanwhile, please help with the following questions:

1) Does this error stop your test ?
2) What is the setting of validation.disable_ssl_certificate_validation in your tempest.conf file?

Thanks!

[Imran Hayder (hayderimran7)]

Catherine
thanks for followup.
Yes its stopping me from running tests if i provide URL to --test-list arg.
however if i provide a file location then it runs fine.
I checked the code, its nothing to do with tempest validation.disable_ssl_certificate_validation value, it actual happens here when it makes call to python requests
https://github.com/openstack/refstack-client/blob/master/refstack_client/list_parser.py#L95

Now the default behavior of python requests is to set verify=True, so if URL has https, it will always check for SSL cert,
this should be fixed by making it similar to this https://github.com/openstack/refstack-client/blob/master/refstack_client/refstack_client.py#L264
i.e.

            response = requests.post(auth_url,
                                     data=json.dumps(content),
                                     headers=headers,
                                     verify=not self.args.insecure)

so verify=not self.args.insecure should be set same way for getting the test lists URL in list_parser.py https://github.com/openstack/refstack-client/blob/master/refstack_client/list_parser.py#L95

thanks :)

[Catherine Diep (cdiep)]

Hi Imran,
Thanks for the information. This helps a lot! Would you please test and confirm that https://review.openstack.org/#/c/388250 will fix your issue?

[Imran Hayder (hayderimran7)]

Hi Catherine. Thank you so much for quick fix.
I will test it today and verify it works
Thanks

[Imran Hayder (hayderimran7)]

Hello, i verified this patch works :)
https://review.openstack.org/#/c/388250

its good to merge now- thanks for the quick fix :)

(.venv)CI-2:~/workspace/refstack_cloud_compute/refstack-client # ./refstack-client test --insecure -c '.tempest/etc/tempest.conf' -v --test-list "https://refstack.openstack.org/api/v1/guidelines/2016.08/tests?target=object&type=required&alias=true&flag=false"
2016-10-18 17:48:53,508 refstack_client:184 WARNING Using identity section of tempest config file to specify user credentials is deprecated and won't be supported soon. User credentials should be defined in the accounts file as described in the Tempest configuration guide (http://docs.openstack.org/developer/tempest/configuration.html).
/var/lib/jenkins/workspace/refstack_cloud_compute/refstack-client/.venv/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
  SNIMissingWarning
/var/lib/jenkins/workspace/refstack_cloud_compute/refstack-client/.venv/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
/var/lib/jenkins/workspace/refstack_cloud_compute/refstack-client/.venv/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:838: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/security.html
  InsecureRequestWarning)
2016-10-18 17:48:54,377 refstack_client:403 INFO Starting Tempest test...
2016-10-18 17:48:54,377 refstack_client:413 INFO Normalizing test list...
Option "verbose" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future.
/var/lib/jenkins/workspace/refstack_cloud_compute/refstack-client/.venv/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
/var/lib/jenkins/workspace/refstack_cloud_compute/refstack-client/.venv/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:838: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. Se...

[Catherine Diep (cdiep)]

Hi Imran,
Thank you so much for submitting the bug and verifying the fix.

[Catherine Diep (cdiep)]

Fixed by https://review.openstack.org/#/c/388250/ merged Oct 19, 2016.