Improve pip retrieval macro

Bug #1893204 reported by Gaël THEROND
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kolla
Fix Released
Wishlist
Gaël THEROND

Bug Description

What happened:

When performing an offline build of the kolla containers (mirroring everything internally PYPI/REPOS), the
get_pip() function from the macros.j2 template can't reach the internet and so can't install python-pip and fail.

What should happen:

Because major distribution provides a python-pip package, even if there is situations where this is not possible, we should retrieve pip from those distribution repositories using the available package manager and then fallback to this external retrieval if not applicable.

That would allow offline build based on package manager to easily and seamlessly retrieve pip without having to use an external URL or a HTTP(s) proxy.

Some offline installation don't allow any kind of HTTP proxy but yet provide either internal mirrors of a distribution repositories and/or a PYPI mirror/proxy that specifically can't be used as a HTTP(s) proxy (Nexus python proxy for instance).

How to reproduce:

Launch a build of the kolla containers in an offline mode, pip won't install as containers are not able to reach internet and consequently fail the build.

**Environment**
OS: CentOS-8.1.1911
Kernel:4.18.0-147
Docker version: 19.03
Kolla version: 9.2.0

Revision history for this message
Gaël THEROND (gtherond) wrote :

Gonna work on this patch.

Changed in kolla:
assignee: nobody → Gaël THEROND (gtherond)
Gaël THEROND (gtherond)
Changed in kolla:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla (master)

Fix proposed to branch: master
Review: https://review.opendev.org/751615

Changed in kolla:
importance: Undecided → Wishlist
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.opendev.org/751706

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on kolla (master)

Change abandoned by Gaël THEROND (<email address hidden>) on branch: master
Review: https://review.opendev.org/751706
Reason: conflict with already existing change: https://review.opendev.org/#/c/751615/

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla (master)

Fix proposed to branch: master
Review: https://review.opendev.org/751742

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.opendev.org/751765

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on kolla (master)

Change abandoned by Gaël THEROND (<email address hidden>) on branch: master
Review: https://review.opendev.org/751765
Reason: Will push again with a better commit tree.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Change abandoned by Gaël THEROND (<email address hidden>) on branch: master
Review: https://review.opendev.org/751742
Reason: Will push again with a better commit tree.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Change abandoned by Gaël THEROND (<email address hidden>) on branch: master
Review: https://review.opendev.org/751615
Reason: Will push again with a better commit tree

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla (master)

Fix proposed to branch: master
Review: https://review.opendev.org/751787

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.opendev.org/752455

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on kolla (master)

Change abandoned by Gaël THEROND (<email address hidden>) on branch: master
Review: https://review.opendev.org/752455
Reason: improper commit message.

Changed in kolla:
assignee: Gaël THEROND (gtherond) → Radosław Piliszek (yoctozepto)
Gaël THEROND (gtherond)
Changed in kolla:
assignee: Radosław Piliszek (yoctozepto) → Gaël THEROND (gtherond)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla (master)

Reviewed: https://review.opendev.org/751787
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=b93c40a3635fbce72e48dc29120e50484cba36ca
Submitter: Zuul
Branch: master

commit b93c40a3635fbce72e48dc29120e50484cba36ca
Author: Gaël THEROND (Fl1nt) <email address hidden>
Date: Thu Sep 17 14:38:21 2020 +0200

    Improve pip install process for offline deployment.

        * "Use distribution-provided pip"
        * "Use python's pip module invocation method"
        * "Install pip earlier in order to avoid multiple installation"
        * "Remove pip_version variable requirement and call"

    Change-Id: Id0e738044a1931f9d611a7281a48ea4a593f1cf1
    Closes-bug: #1893204

Changed in kolla:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/758054

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/758057

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla (stable/ussuri)

Reviewed: https://review.opendev.org/758054
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=943c03f2ea455dfa583f065bdc42168c8def448e
Submitter: Zuul
Branch: stable/ussuri

commit 943c03f2ea455dfa583f065bdc42168c8def448e
Author: Gaël THEROND (Fl1nt) <email address hidden>
Date: Thu Sep 17 14:38:21 2020 +0200

    Improve pip install process for offline deployment.

        * "Use distribution-provided pip"
        * "Use python's pip module invocation method"
        * "Install pip earlier in order to avoid multiple installation"
        * "Remove pip_version variable requirement and call"

    Change-Id: Id0e738044a1931f9d611a7281a48ea4a593f1cf1
    Closes-bug: #1893204
    (cherry picked from commit b93c40a3635fbce72e48dc29120e50484cba36ca)

tags: added: in-stable-ussuri
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on kolla (stable/train)

Change abandoned by Mark Goddard (<email address hidden>) on branch: stable/train
Review: https://review.opendev.org/758057

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla 10.2.0

This issue was fixed in the openstack/kolla 10.2.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla 9.3.1

This issue was fixed in the openstack/kolla 9.3.1 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.