Various results with --skip-parents and --skip-existing

[Mark Goddard]

Examples:

1. kolla-build --type source --skip-parents monasca-grafana

This does not try to build the monasca-grafana image.

2. kolla-build --type source --skip-parents

This will build all images. Expect it to skip parents.

3. kolla-build --type source --skip-existing

This will build all images. Expect it to skip existing images.

[Mark Goddard (mgoddard)]

Looks like the problem with 1 is when the image is a grandchild - the parent gets marked as unmatched rather than skipped.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/713221

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/713221
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=dfddcce3a5bc98cbd177c984fd9d0a8bab51d40c
Submitter: Zuul
Branch: master

commit dfddcce3a5bc98cbd177c984fd9d0a8bab51d40c
Author: Mark Goddard <email address hidden>
Date: Mon Mar 16 12:54:11 2020 +0000

    Fix --skip-existing and --skip-parents

    There are currently a few corner cases with the --skip-existing and
    --skip-parents flags. Examples:

    1. kolla-build --type source --skip-parents monasca-grafana

    This does not try to build the monasca-grafana image. This appears to be
    an issue when the image is a grandchild.

    2. kolla-build --type source --skip-parents

    This will build all images. Expect it to skip parents.

    3. kolla-build --type source --skip-existing

    This will build all images. Expect it to skip existing images.

    The filter_images method does quite a lot, including handling
    whether images are buildable, regex/profile matching, and which images
    to skip. The matching and skipping parts are done in a single pass,
    which can lead to some weird effects due to dependencies between the
    images and their statuses. Also, skipping is only currently applied when
    there is a regex/profile filter.

    This change splits out the matching and skipping into two separate
    passes. In the first pass, we mark all buildable images that match the
    filter as matched. In the second, we iterate over matched images,
    applying the skip existing and skip parents rules.

    Change-Id: I2f895aea0cc59d808129e9fc636af0890196af33
    Closes-Bug: #1867614
    Related-Bug: #1810979

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/717506

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/717506
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=c59dc07e9031eaf127b4222a66efc06db29454d2
Submitter: Zuul
Branch: stable/train

commit c59dc07e9031eaf127b4222a66efc06db29454d2
Author: Mark Goddard <email address hidden>
Date: Mon Mar 16 12:54:11 2020 +0000

    Fix --skip-existing and --skip-parents

    There are currently a few corner cases with the --skip-existing and
    --skip-parents flags. Examples:

    1. kolla-build --type source --skip-parents monasca-grafana

    This does not try to build the monasca-grafana image. This appears to be
    an issue when the image is a grandchild.

    2. kolla-build --type source --skip-parents

    This will build all images. Expect it to skip parents.

    3. kolla-build --type source --skip-existing

    This will build all images. Expect it to skip existing images.

    The filter_images method does quite a lot, including handling
    whether images are buildable, regex/profile matching, and which images
    to skip. The matching and skipping parts are done in a single pass,
    which can lead to some weird effects due to dependencies between the
    images and their statuses. Also, skipping is only currently applied when
    there is a regex/profile filter.

    This change splits out the matching and skipping into two separate
    passes. In the first pass, we mark all buildable images that match the
    filter as matched. In the second, we iterate over matched images,
    applying the skip existing and skip parents rules.

    Change-Id: I2f895aea0cc59d808129e9fc636af0890196af33
    Closes-Bug: #1867614
    Related-Bug: #1810979
    (cherry picked from commit dfddcce3a5bc98cbd177c984fd9d0a8bab51d40c)

[yuan.lu (luyuan)]

1.Version: 8.0.3 stein
2.Run Command：

kolla-build --type source --skip-parents --tag latest-2020-04-23 --config-file /tmp/kolla-build-glance.conf glance

3、Result：

DEBUG:kolla.common.utils:Image glance-base matched regex
DEBUG:kolla.common.utils:Image openstack-base matched regex
DEBUG:kolla.common.utils:Image base matched regex
INFO:kolla.common.utils:===========================
INFO:kolla.common.utils:Images that failed to build
INFO:kolla.common.utils:===========================
ERROR:kolla.common.utils:glance-api Failed with status: matched
ERROR:kolla.common.utils:glance-registry Failed with status: matched

skipped": [{"name": "base"}, {"name": "glance-base"}, {"name": "openstack-base"}]}

[Mark Goddard (mgoddard)]

Yuan Lu, the code had changed too much since Stein to do a clean backport.