pip-installed python-request breaks SSL/TLS mode

[Rob Crittenden]

The SSL/TLS code relies on the system-wide CA bundle to include the CA that issued the certificates used by the various services. This was done to stop the proliferation of CA file options being added to every server and client, and sometimes doubly so when a server would talk to other servers.

The problem is that if a newer version of python-requests is required than is provided by the underlying OS provider then the upstream version is installed by pip. The upstream version of python-requests defaults to using its own CA bundle. The per-distro versions are modified to return their distro-specific path.

This will eventually cause installation to fail due to untrusted SSL server certificates.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/186545

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/186545
Committed: https://git.openstack.org/cgit/openstack-dev/devstack/commit/?id=7d350720fe5d25fece68c5d1625a33a6cad431ef
Submitter: Jenkins
Branch: master

commit 7d350720fe5d25fece68c5d1625a33a6cad431ef
Author: Rob Crittenden <email address hidden>
Date: Thu May 28 14:59:31 2015 -0400

    Replace pip-installed requests CA bundle with link

    If the version of python-requests required is higher than
    that provided by the operating system, pip will install
    it from upstream.

    The upstream version provides its own CA certificate bundle
    based on the Mozilla bundle, and defaults to that in case
    a CA certificate file is not specified for a request.

    The distribution-specific packages point to the system-wide
    CA bundle that can be managed by tools such as
    update-ca-trust (Fedora/RHEL) and update-ca-certificates
    (Debian/Ubuntu).

    When installing in SSL/TLS mode, either with SSL=True or by
    adding tls-proxy to ENABLED_SERVICES, if a non-systemwide
    CA bundle is used, then the CA generated by devstack will
    not be used causing the installation to fail.

    Replace the upstream-provided bundle with a link to the
    system bundle when possible.

    Change-Id: I349662ff8f851b4a7f879f89b8975a068f2d73dc
    Closes-Bug: #1459789

[Rob Crittenden (rcritten)]

The original patch broke the build if python-requests wasn't installed.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/190276

[Robert Collins (lifeless)]

So this sounds like a bug in the interface for this in requests in general. It should not require replacement of the code as a whole to use the thing here.

[Rob Crittenden (rcritten)]

I can sympathize with upstream python-requests: what are they supposed to return? I'd be glad if they did a distro-specific check and only defaulted to their own if one is not found.

This patch doesn't replace code, it replaces the package-provided bundle with a symlink to the system bundle instead.