Branch creation tool failures

[Scott Little]

Brief Description
-----------------
create_branches_and_tags.sh doesn't produce a manifest that uses
the default revision facility.

create_branches_and_tags.sh --lockdown is too aggressive, replacing tags with sha's

create_branches_and_tags.sh failes to update .gitreview files

push_branches_tags.sh fails to push to opendev.org

Severity
--------
Provide the severity of the defect.
Major

Steps to Reproduce
------------------
create_branches_and_tags.sh --branch=$NEW_BRANCH --remotes=$STX_REMOTES --manifest --lock-down
push_branches_tags.sh --branch=$NEW_BRANCH --remotes=$STX_REMOTES --manifest

Expected Behavior
------------------
- Repos are branched
- .gitreviews are updated.
- A manifest is produced that makes minimal changes, and is usable without further manual editing
- All tags/branches/updates are delivered upstream

Actual Behavior
----------------
- .gitreviews are not updates
- Manifest is heavily modified, and requires manual cleanup
- Unable to deliver changes to opendev.org

Reproducibility
---------------
100%

System Configuration
--------------------
N/A

Branch/Pull Time/Commit
-----------------------
April 15, 2021

Last Pass
---------
2019 ?

Timestamp/Logs
--------------
 ! [remote rejected] r/stx.5.0 -> r/stx.5.0 (prohibited by Gerrit: update for creating new commit object not permitted)

Test Activity
-------------
N/A

Workaround
----------
N/A

[Ghada Khalil (gkhalil)]

screening: marking for stx.6.0 given the r/stx.5.0 release branch is already created. If it makes sense to cherrypick the tool fixes to that branch, the developer can go proceed with that if the changes are ready before branch freeze/final compile.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: r/stx.5.0
Review: https://review.opendev.org/c/starlingx/root/+/790290

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/root/+/786570
Committed: https://opendev.org/starlingx/root/commit/f457bd15b9ce0be512ec96abcb9b858b0bab2ea8
Submitter: "Zuul (22348)"
Branch: master

commit f457bd15b9ce0be512ec96abcb9b858b0bab2ea8
Author: Scott Little <email address hidden>
Date: Fri Apr 16 01:39:17 2021 -0400

    Improved branching tools

    create_branches_and_tags.sh:
    - Update the .gitreview files in branched git repos.
    - When updating a manifest, add the ability to update and
      use the default revision field.
    - Create two levels of manifest lockdown, soft and hard.
      Soft lockdown only sets sha revisions on unbranched projects
      that lack a revision, or set the revision to master.
      Hard lockdown applies to all unbranched projects.

    push_branches_tags.sh:
    - opendev no longer accepts 'git push' for the delivery of
      new branches with updates. Instead we must now
      use separate commands to deliver the tag, the branch,
      and any updates.

    Closes-Bug: 1924762
    Signed-off-by: Scott Little <email address hidden>
    Change-Id: I6d669ddc80cc9b3cb9e72d65a64589dbccf43ae3

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: r/stx.5.0
Review: https://review.opendev.org/c/starlingx/root/+/791253

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/root/+/791253
Committed: https://opendev.org/starlingx/root/commit/3ea0fbe527a600811933e4ba36e87b4b580573ac
Submitter: "Zuul (22348)"
Branch: r/stx.5.0

commit 3ea0fbe527a600811933e4ba36e87b4b580573ac
Author: Scott Little <email address hidden>
Date: Fri Apr 16 01:39:17 2021 -0400

    Improved branching tools

    create_branches_and_tags.sh:
    - Update the .gitreview files in branched git repos.
    - When updating a manifest, add the ability to update and
      use the default revision field.
    - Create two levels of manifest lockdown, soft and hard.
      Soft lockdown only sets sha revisions on unbranched projects
      that lack a revision, or set the revision to master.
      Hard lockdown applies to all unbranched projects.

    push_branches_tags.sh:
    - opendev no longer accepts 'git push' for the delivery of
      new branches with updates. Instead we must now
      use separate commands to deliver the tag, the branch,
      and any updates.

    Closes-Bug: 1924762
    Signed-off-by: Scott Little <email address hidden>
    (cherry picked from commit f457bd15b9ce0be512ec96abcb9b858b0bab2ea8)
    Change-Id: If4ee1320e68edcdfafc6bb9c0aff4594e864352d

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Scott Little <email address hidden>" on branch: r/stx.5.0
Review: https://review.opendev.org/c/starlingx/root/+/790290

[Ghada Khalil (gkhalil)]

screening: added the stx.5.0 and the in-r-stx50 tags since the code was cherrypicked to the r/stx.5.0 release branch

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: f/centos8
Review: https://review.opendev.org/c/starlingx/root/+/792232

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/root/+/792232
Committed: https://opendev.org/starlingx/root/commit/7cb55d28111dbf7458ed3f01a5eda5c4dfdb124b
Submitter: "Zuul (22348)"
Branch: f/centos8

commit f457bd15b9ce0be512ec96abcb9b858b0bab2ea8
Author: Scott Little <email address hidden>
Date: Fri Apr 16 01:39:17 2021 -0400

    Improved branching tools

    create_branches_and_tags.sh:
    - Update the .gitreview files in branched git repos.
    - When updating a manifest, add the ability to update and
      use the default revision field.
    - Create two levels of manifest lockdown, soft and hard.
      Soft lockdown only sets sha revisions on unbranched projects
      that lack a revision, or set the revision to master.
      Hard lockdown applies to all unbranched projects.

    push_branches_tags.sh:
    - opendev no longer accepts 'git push' for the delivery of
      new branches with updates. Instead we must now
      use separate commands to deliver the tag, the branch,
      and any updates.

    Closes-Bug: 1924762
    Signed-off-by: Scott Little <email address hidden>
    Change-Id: I6d669ddc80cc9b3cb9e72d65a64589dbccf43ae3

commit 0babd33b6d851dd11492c26e92c8a6ac2c2557de
Author: Rafael Jardim <email address hidden>
Date: Wed May 12 11:25:54 2021 -0300

    Update stx-platformclients tag to stx.5.0-v1.4.3

    This commit updates the image with the updated clients.

    Test:
    Some normal commands
    Commands related with https dcmanager that wasn't working
    System application-upload that wasn't working when executed
    from remote cli

    Closes-Bug: 1928233
    Closes-Bug: 1928231
    Signed-off-by: Rafael Jardim <email address hidden>
    Change-Id: I8a0d12f699336a4412be5ff3c73cfb8d59038780

commit a163d7723e659e89c37ec933d1b0f9aa638a6a73
Author: Cole Walker <email address hidden>
Date: Wed May 5 09:39:17 2021 -0400

    Update image tag for notificationservice-base

    Update image tag to stx.5.0-v1.0.4 for notificationservice-base

    Closes-Bug: 1924201
    Closes-Bug: 1924197

    Signed-off-by: Cole Walker <email address hidden>
    Change-Id: Id863c4f154cc0b39e30ee8986fc07f9856a22826

commit 84c45f5e3a241237887af9db89d8c0aa1f8923e0
Author: Charles Short <email address hidden>
Date: Sun May 2 12:04:04 2021 -0400

    Fix wheels tarball generation

    In d7c5a54ab94bce6635b83d91a807d28f97836a81, django was dropped in favor
    of rfc3986. However the wheel was not added to the build-wheels
    generation which breaks the docker images. Also add the migrate wheel
    since it was mising as well. Add the required wheels in order to
    build the docker image properly.

    Test:
    - Build new centos-stable-wheels tarball
    - Build stx-keystone-api-proxy container

    Closes-Bug: 1926795

    Signed-off-by: Charles Short <email address hidden>
    Change-Id: Ib6f0abfdcc82ca14f92ebc5b45fe8df961e804ee

commit 062353b6de7d4b0e017203c3e6086891bd6b9213
Author: Cole Walker <email address hidden>
Date: Mon May 3 09:48:54 2021 -0400

    Update image tag for notificationclient-base...

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/root/+/796699

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/root/+/796699
Committed: https://opendev.org/starlingx/root/commit/1398925730db8eb0e95b8562ff23776037463314
Submitter: "Zuul (22348)"
Branch: master

commit 1398925730db8eb0e95b8562ff23776037463314
Author: Scott Little <email address hidden>
Date: Thu Jun 3 14:38:56 2021 -0400

    Three more improvements to branch creation tools.

    1) Add default revision and soft/hard lockdown options to create_tags.sh.

    These options already appear in the create_branches_and_tags.sh script.

    Default revision set the default revision field of the manifest,
    removing the need to specify a revision for each project.

    Soft lockdown is content with tags already specified in the manifest.
    Hard lockdown forces the use of sha's.

    2) Improved determination of the local branch.

    Need to work around inconsistencies of various git versions.
    Also need to deal with 'repo' leaving us in detached head state and
    local updates on a detached state, in which case we try
    to find the 'best fit' local branch from which we diverged.

    3) Ability to ask the branch/tag push scripts to bypass gerrit.

    Successful use requires the git repositories and gerrit to
    to permit direct push. No true of the current opendev repo setup,
    but might be useful for mirrors.

    Closes-Bug: 1924762
    Signed-off-by: Scott Little <email address hidden>
    Change-Id: I2ba01e11ef27c1d10c816b5af6104afab440b3b3

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/root/+/798692

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/root/+/798692
Committed: https://opendev.org/starlingx/root/commit/214fdbf4ef6e7bd83d4bff249ae68e2ef197bc5e
Submitter: "Zuul (22348)"
Branch: master

commit 214fdbf4ef6e7bd83d4bff249ae68e2ef197bc5e
Author: Scott Little <email address hidden>
Date: Mon Jun 28 19:10:53 2021 -0400

    Improved branching tools

    This content was intended for inclusion vs LP 1924762.

    Improve create_branches_and_tags.sh. by adding options to alter
    .gitreview settings, including host, port, and project, as part
    of branch creation.

    Also add improved help/usage text for create_branches_and_tags.sh.

    Improve git_remote_branch by adding a fallback method for finding
    the remote branch. It finds the nearest branch on the current remote
    and reports that. I only expect this to apply when the local branch
    hasn't been set to track a remote branch.

    Closes-Bug: 1924762
    Signed-off-by: Scott Little <email address hidden>
    Change-Id: I790aae159df0a2360d5e50d930fa3d3be038bcc1