Update build to produce list of docker images required for platform deployment

[Frank Miller]

From the starlingx-discuss mailing list the following request was made by Cheng Li but several others also requested this list.

Subject: [Starlingx-discuss] Docker image list

As you know many docker images are pulled during starlingx deployment. It may be fast to pull all these images in America, but it’s very slow in China.
To speed up starlingx deployment, I have set up a private docker registry for which I sync images every night from upstream registries by cron job.
I installed starlingx without using my private docker registry so that I can get the upstream docker image list. Every item in the list is synced every day.

This works fine except that the docker image list changes sometimes. In this case, I would have to collect the image list by deploying without using private docker registry, which is very slow.
So I wonder if it’s possible to publish the docker image list file together with ISO and tarball on CENGN?

--------------

This launchpad will be used to track an enhancement for the build to produce a list of docker images required by StarlingX to deploy the platform (up to the point where the platform-integ-apps is applied).

Note: the images required by the stx-openstack application can be obtained already from the CENGN build output.

[Bob Church (rchurch)]

We have the following:

Stage 1: After Ansible setup

REPOSITORY TAG
starlingx/k8s-cni-sriov master-centos-stable-latest
quay.io/airshipit/armada dd2e56c473549fd16f94212b553ed58c48d1f51b-ubuntu_bionic
quay.io/calico/node v3.6.2
quay.io/calico/cni v3.6.2
quay.io/calico/kube-controllers v3.6.2
k8s.gcr.io/kube-proxy v1.13.5
k8s.gcr.io/kube-apiserver v1.13.5
k8s.gcr.io/kube-controller-manager v1.13.5
k8s.gcr.io/kube-scheduler v1.13.5
nfvpe/multus v3.2
gcr.io/kubernetes-helm/tiller v2.13.1
k8s.gcr.io/coredns 1.2.6
k8s.gcr.io/pause 3.1

Stage 2: After first unlock. platform-integ-apps are applied.

REPOSITORY TAG
quay.io/external_storage/rbd-provisioner v2.1.1-k8s1.11
port/ceph-config-helper v1.10.3

I took a crack at trying to locate the information from my private build. We would basically need a set of greps to derive them. I think it can be done if someone wants to take on packaging it into a script.

# For these they all are derived from here:
# starlingx/k8s-cni-sriov master-centos-stable-latest
# quay.io/calico/node v3.6.2
# quay.io/calico/cni v3.6.2
# quay.io/calico/kube-controllers v3.6.2
[rchurch@soho-rchurch-lx2 cgcs-root]$ grep -ri "image:" $MY_REPO/stx/stx-config/puppet-manifests/src/modules/platform/templates | sort -u
/l/d/rchurch/starlingx-2/cgcs-root/stx/stx-config/puppet-manifests/src/modules/platform/templates/calico.yaml.erb: image: "<%= @quay_registry %>/calico/cni:v3.6.2"
/l/d/rchurch/starlingx-2/cgcs-root/stx/stx-config/puppet-manifests/src/modules/platform/templates/calico.yaml.erb: image: "<%= @quay_registry %>/calico/kube-controllers:v3.6.2"
/l/d/rchurch/starlingx-2/cgcs-root/stx/stx-config/puppet-manifests/src/modules/platform/templates/calico.yaml.erb: image: "<%= @quay_registry %>/calico/node:v3.6.2"
/l/d/rchurch/starlingx-2/cgcs-root/stx/stx-config/puppet-manifests/src/modules/platform/templates/multus.yaml.erb: image: <%= @docker_registry %>/nfvpe/multus:v3.2
/l/d/rchurch/starlingx-2/cgcs-root/stx/stx-config/puppet-manifests/src/modules/platform/templates/sriov-cni.yaml.erb: image: <%= @docker_registry %>/starlingx/k8s-cni-sriov:master-centos-stable-latest
/l/d/rchurch/starlingx-2/cgcs-root/stx/stx-config/puppet-manifests/src/modules/platform/templates/sriovdp-daemonset.yaml.erb: image: <%= @docker_registry %>/starlingx/k8s-plugins-sriov-network-device:master-centos-stable-latest

# For these we can derive it from the Kubernetes version:
# k8s.gcr.io/kube-proxy v1.13.5
# k8s.gcr.io/kube-apiserver v1.13.5
# k8s.gcr.io/kube-controller-manager v1.13.5
# k8s.gcr.io/kube-scheduler v1.13.5
[rchurch@soho-rchurch-lx2 cgcs-root]$ grep -ri "kubernetesVersion" $MY_REPO/stx/stx-config/puppet-...

[Ghada Khalil (gkhalil)]

Marking as not gating - this is more a build/tools enhancement

[Ghada Khalil (gkhalil)]

Assigning to the build/tools team

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/docs/+/884643
Committed: https://opendev.org/starlingx/docs/commit/1fc795a68f166607dfd7f75d4c74eebea8756b1f
Submitter: "Zuul (22348)"
Branch: master

commit 1fc795a68f166607dfd7f75d4c74eebea8756b1f
Author: Bruno Muniz <email address hidden>
Date: Mon May 29 17:23:16 2023 -0300

    Add a local registry guide for developers

    The guide is aimed at developers working in virtual environments that
    need to build or run several controller-0 setups (specifically, to
    developers in need of running the Ansible bootstrap playbook multiple
    times).

    By configuring a local registry and downloading the needed images
    beforehand, all subsequent installations can point to this local
    registry to download images from it to the controller-0. Not going to
    the internet everytime, devs are not subject to public registries' rate
    limits.

    Story: 2005051
    Task: 47895
    Partial-Bug: 1834504
    Partial-Bug: 1888610

    Change-Id: Ie3e3cbc09030a6491f7b573a4ea5fd57e3158d54
    Signed-off-by: Bruno Muniz <email address hidden>