Python plugin: paths for from the snapcraft snap in sys.path break dependencies

Bug #1860884 reported by Alberto Donato
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Snapcraft
High
Sergio Schvezov

Bug Description

Tested with Snapcraft 3.9.8 (3970) from stable channel

In a python part for a snap, if python3 is called inside a build step, the interpreter installed from the part is used. However, this has some paths from the python from the snapcraft snap in sys.path.

He're a minimal reproducer:

name: python-path-test
version: "0.0.1"
summary: python-path-test
description: python-path-test
grade: stable
confinement: strict
base: core18

apps:
  python-path-test:
    command: python-path-test

parts:
  python-path-test:
    plugin: python
    source: .
    source-type: local
    requirements:
      - requirements.txt
    override-build: |
      set -e
      snapcraftctl build
      echo -------------
      which python3
      echo -------------
      python3 -c "import sys; import pprint; pprint.pprint(sys.path)"
      echo -------------
      python-path-test

The "python-path-test" script does exactly the same as the oneliner above.

The output during build is the following:

-------------
/home/ack/Desktop/python-path-test/parts/python-path-test/install/usr/bin/python3
-------------
['',
 '/home/ack/Desktop/python-path-test/parts/python-path-test/install/usr/lib/python36.zip',
 '/home/ack/Desktop/python-path-test/parts/python-path-test/install/usr/lib/python3.6',
 '/home/ack/Desktop/python-path-test/parts/python-path-test/install/usr/lib/python3.6/lib-dynload',
 '/home/ack/Desktop/python-path-test/parts/python-path-test/install/usr/lib/python3/dist-packages',
 '/snap/snapcraft/3970/lib/python3.6/site-packages',
 '/home/ack/Desktop/python-path-test/stage/lib/python3.6/site-packages',
 '/home/ack/Desktop/python-path-test/parts/python-path-test/install/lib/python3.6/site-packages']
-------------
['/home/ack/Desktop/python-path-test/parts/python-path-test/install/bin',
 '/home/ack/Desktop/python-path-test/parts/python-path-test/install/usr/lib/python36.zip',
 '/home/ack/Desktop/python-path-test/parts/python-path-test/install/usr/lib/python3.6',
 '/home/ack/Desktop/python-path-test/parts/python-path-test/install/usr/lib/python3.6/lib-dynload',
 '/home/ack/Desktop/python-path-test/parts/python-path-test/install/usr/lib/python3/dist-packages',
 '/snap/snapcraft/3970/lib/python3.6/site-packages',
 '/home/ack/Desktop/python-path-test/stage/lib/python3.6/site-packages',
 '/home/ack/Desktop/python-path-test/parts/python-path-test/install/lib/python3.6/site-packages']

In both cases '/snap/snapcraft/3970/lib/python3.6/site-packages' ia included. It seems like only paths under $SNAPCRAFT_INSTALL_PART should be there.

This can cause issues when the application has dependency requirements that conflict with the ones from snapcraft itself, since in that case pkg_resources breaks.

Alberto Donato (ack)
summary: - Python plugin: paths for from the snapcraft snap in sys.pat break
+ Python plugin: paths for from the snapcraft snap in sys.path break
dependencies
Revision history for this message
Chris Patterson (cjp256) wrote :

This is an effect of running `snapcraftctl build`. It modifies sitecustomize.py to include snap dir, stage dir, and part install site-packages.

I believe stage dir is required (if you use stage-packages to pull in dependencies).

sergiusens will be able to weigh in with a more expert opinion. In the meantime, you could update sitecustomize.py to work as you want it after `snapcraftctl build`.

e.g.
    override-build: |
      set -e
      snapcraftctl build
      sed -i "s|snap_dir, ||" $SNAPCRAFT_PART_INSTALL/usr/lib/python3.6/sitecustomize.py
      python3 -c "import sys; import pprint; pprint.pprint(sys.path)"
      python-path-test

Revision history for this message
Alberto Donato (ack) wrote :

Thanks for the explaination

Yeah I think stage dir should be fine. The issue I'm seeing (that I derived this example from) is with having the snap dir there.

Specifically, an application that uses pymacaroons needs a version that conflicts with snapcraft's own requirements, making pkg_resources choke during the build stage.

Changed in snapcraft:
status: New → Triaged
importance: Undecided → High
assignee: nobody → Sergio Schvezov (sergiusens)
Changed in snapcraft:
status: Triaged → In Progress
Revision history for this message
Sergio Schvezov (sergiusens) wrote :
Changed in snapcraft:
status: In Progress → Fix Committed
Revision history for this message
Sergio Schvezov (sergiusens) wrote :

Part of 3.10

Changed in snapcraft:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers