Recognize symbolic link for snapcraft.yaml in Git source

[John Neffenger]

The Launchpad build service fails to recognize a snapcraft.yaml file that is a symbolic link in a Git source tree and reports the error:

  The top level of snapcraft.yaml from ~jgneff/+git/openjdk:beta is not a mapping

For example, see the snapcraft.yaml symbolic link below:

index : ~jgneff/+git/openjdkf
https://git.launchpad.net/~jgneff/+git/openjdk/tree/snap?h=beta&id=ad1a2188f8fd6ad7aca0812b76a65f7b864277cd

The failure occurs only when building a snap from a Git branch on Launchpad. All other builds work fine, such as when building automatically from the main branch on GitHub to the edge channel and when running remote builds with the 'snapcraft remote-build' command. It seems to occur only when trying to build the snap directly from a Git repository branch on Launchpad.

I want to use a symbolic link for the snapcraft.yaml file to solve a problem with the Git workflow. I have to maintain three permanent branches for each of the three channels, with the main branch building into the edge channel, the beta branch building into the beta channel, and the candidate branch building into the candidate channel, which can then be promoted to the stable channel. Each of these channels track a different upstream repository and branch for the development, early access, and release versions of the software being packaged.

The problem is that almost all of the important changes are in the snapcraft.yaml file itself, so when you change the file on main and merge into the other two branches, you always get merge conflicts and have to re-do the correct repositories, branches, and version numbers manually over and over again.

By making it a symbolic link, I can put the snapcraft.yaml file for all three branches under the 'snap/local' directory, and each branch can have its own symbolic link pointing to the correct actual file:

  Under the 'snap' directory:
    snapcraft.yaml -> local/edge.yaml (for the main branch)
    snapcraft.yaml -> local/beta.yaml (for the beta branch)
    snapcraft.yaml -> local/candidate.yaml (for the candidate branch)

With that setup, I can test all three releases on the main branch by temporarily changing the symbolic link, and when I merge the main branch into beta or candidate, there are no merge conflicts, and I don't wipe out any important source, source-branch, or version fields in the YAML file.

[John Neffenger (jgneff)]

I believe the bug lies in the Launchpad build service, but I included the Snapcraft project for opinions on whether I'm going about this the right way. Feel free to delete Snapcraft from this bug report if that was inappropriate.

[John Neffenger (jgneff)]

Here's how I fixed my Git workflow so that I can have a similar strategy for the branches without using any symbolic links.

Follow the workflow recommended by Junio Hamano, the core maintainer of Git, for maintaining permanent parallel branches. Put common code on one branch, create a separate branch for each custom deployment, and stick to the following two rules:

- You make edits to common files only on the common branch.
- You merge from common to deployment, never the other way.

See:

  Re: Is there a way to exclude user-specified files
  or directories from participating in merges?
  https://www.spinics.net/linux/lists/git/msg94767.html

Make the main branch the branch with the common code and without a snapcraft.yaml file. Create branches named after the candidate, beta, and edge channels that contain their specific snapcraft.yaml files.

Don't build from the main branch on GitHub to the edge channel. Create a new edge branch for the channel and build it on Launchpad. That will allow updates to the main branch without triggering an unnecessary build, such as updating the README file or adding an issue template.

Don't merge as if it pulls over diffs. While 'git rebase' recreates commits by replaying diffs onto the target branch, the 'git merge' command pulls over entire files from the commit snapshots. That's why all the snapcraft.yaml changes for the channel need to be added back again after merging from one branch into another. See:

  Commits are snapshots, not diffs
  https://github.blog/2020-12-17-commits-are-snapshots-not-diffs/

See also:

  Git: how to maintain permanent parallel branches
  https://stackoverflow.com/a/24978991

  customer branches
  https://gitolite.com/archived/special-branches.html

  fixing-up common and customer branches
  https://gitolite.com/archived/special-branch-fixups.html

[Sergio Schvezov (sergiusens)]

Marking invalid for Snapcraft as this is just applicable to launchpad itself.