Python plugin: file conflicts when using the plugin with a custom version of Python from another part

[Tamás Nepusz]

My use-case is as follows: I need to snap a package that supports Python 3.7 only, so I added the compilation of Python 3.7 as a separate part to my snap, similarly to the following snippet:

parts:
  python37:
    source: https://www.python.org/ftp/python/3.7.3/Python-3.7.3.tar.xz
    source-type: tar
    source-checksum: md5/93df27aec0cd18d6d42173e601ffbbfd
    plugin: autotools
    configflags:
      - --prefix=/usr
    build-packages: ***omitted***
    stage-packages: ***omitted***
    stage:
      - -usr/lib/python3.7/test
      - -var

  my-package:
    plugin: python
    source: .
    requirements: ./requirements.txt
    after:
      - python37
    override-pull: |
      ...some custom script that is probably not relevant...
      snapcraftctl pull
    override-prime: |
      snapcraftctl prime
      ...some custom script that is probably not relevant...

During the first build, the Python plugin fails to detect that another version of Python has already been built and staged in another part so it includes the default Python version (3.5) in the "my-package" part, leading to conflicts between "my-package" and "python37"; in particular, the following files are present in the install area of both parts:

/usr/bin/2to3
/usr/bin/pydoc3
/usr/bin/python3

If my understanding is correct, the Python plugin should be able to detect that I have built Python 3.7 in another part and should simply use that instead of bundling Python 3.5.

Cleaning the `my-package` part and rebuiling everything from there seems to resolve the problem, but it's not an ideal experience.

It is also possible to work around the issue by not staging the conflicting files from `my-package`, but it still means that there are two versions of Python in my snap, even though one would be enough.

I managed to track the bug down to the part that the `stage_packages` property in the Python plugin is evaluated right at the beginning of the build (when there is no `python37` part built yet), and that's why the plugin infers that it should bundle Python 3.5 with the `my-package` part. If the `stage_packages` property were evaluated _later_, right before starting to build `my-package`, it should be able to infer that my custom Python is already compiled.

Snapcraft version: 2.44 (I would use Snapcraft 3.x if I could, but I'm preparing a snap for a Raspberry Pi so I'm using classic on Ubuntu Server 18.04, which provides only Snapcraft 2.44 within the classic environment).