Ability to easily cherry-pick git commits

[Stéphane Graber]

It'd sometimes be very useful if I could specify a number of git commits (or any VCS with a similar concept) that I want to see cherry-picked on top of the listed source for my part.

This would be most useful for hotfixes that are already merged upstream but not part of a release yet and would align pretty well with what we're doing for our deb packages.

For example, I'd like to be able to do:
  lxd:
    source: https://github.com/lxc/lxd
    source-type: git
    source-tag: lxd-2.14
    source-cherry-picks:
     - 0ffe2b2ef14b1c6fc66df9f04f6c23a761183a0e
     - 679aafe1fd65fed20fa3aa85359e6b053cecf276
     - b1eb7453d3911644a9b6d894de7e6dedd241b17d
     - 17de1869158f22d7a77e6e33f46fa51100af9d1d
     - 4263ec402833cd45b0705e5448a560f610756691
     - 865ec4ee242c6b6e1c44f440f686d1880e48afc1

Is there already some way to do so today which I missed?
If not, would the approach above be the right way to go or do you have something else in mind which would achieve the same result?

The proposed approach above should be easy enough to implement that I may do it myself, having already some experience with messing with the VCS code in snapcraft.

[Kyle Fazzari (kyrofa)]

I think this sort of falls under patching versus forking discussion, where we have historically leaned toward saying fork instead of patch. In this case, apply those commits to another branch and use that instead of lxd-2.14.

That said, the winds may be changing here. Curious to hear Sergio's thoughts. Would this be a git-specific feature? I'm not sure the other VCS's have similar capabilities (bzr?).

[Stéphane Graber (stgraber)]

As the upstream, it feels pretty weird to fork our own code just to include some cherry-picks that are already in our tree.

[Kyle Fazzari (kyrofa)]

Another option (for some projects) might be to use the `prepare` script, something like so:

lxd:
  plugin: go
  source: https://github.com/lxc/lxd
  source-type: git
  source-tag: lxd-2.14
  prepare: |
    git cherry-pick 0ffe2b2ef14b1c6fc66df9f04f6c23a761183a0e
    git cherry-pick 679aafe1fd65fed20fa3aa85359e6b053cecf276
    # [...]

That will pull lxd-2.14, and before building, apply the requested cherry-picks. That won't REALLY work for go, though, since you probably need that done in the pull step before `go get` is called. However, we have been discussing adding pre/post scripts to each lifecycle step (including pull), which would add a nicer, generic solution to this.

[Stéphane Graber (stgraber)]

Ah yeah, something like that would be fine.
It may still be fine for go. We don't introduce new dependencies in our cherry-picks, so having it run after "go get" but before "go build" should be fine.

[Stéphane Graber (stgraber)]

Ok, so I just changed our snap to use the prepare step and indeed, for Go it's not good enough, the patches are applied before "go install" but I'm guess snapcraft must copy the git tree to the gopath prior to that, so "go install" doesn't rebuild the modified code and we end up with an unpatched binary.

[Stéphane Graber (stgraber)]

Just found a workaround for that, having "prepare" do "cd ../src" so that it updates the tree that's actually used by "go install".

[Kyle Fazzari (kyrofa)]

A follow-up on this: we're planning on adding before/after scripts for each lifecycle step. This is something that would be best done in an `after-pull` script. As soon as that lands this will become less hacky.

[Stéphane Graber (stgraber)]

Just a quick update on this one, the current support for overrides hasn't really helped all that much.

The issue we ran into is that `override-pull` runs before all pull actions and `override-build` runs after all pull actions.

A git cherry-pick would happen in between the git pull action but before the plugin's own pull.
This is particularly important for things like the Go plugin where a missing cherry-pick can cause its own pull to fail.

Currently our options are to either re-implement the entirety of "pull" for both the source and plugin in use or to ship a modified version of the plugin (which is what we're doing currently).

Also, we like things being declarative so having a "source-cherrypick" or something along those lines which can handle this cleanly would allow for additional validation and a cleaner workflow.

[Chris Patterson (cjp256)]

Core20 should limit the pull step to "source" pulling. There is no longer any plugin-specific pull steps.

After switching to core20, it should be straight forward to do:

```
override-pull:
  snapcraftctl pull
  git cherry-pick ...
```

If that this doesn't satisfy your use case (without explicit yaml support), please let us know! :)