it is no longer possible to have a plainbox part with a python stage-package

Bug #1663792 reported by Leo Arias
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Snapcraft
Fix Released
Medium
Sergio Schvezov

Bug Description

It has never been possible to combine a python part that installs packages using the python install script, with a part that installs python packages from the ubuntu archive. This causes many conflicts.

However, on the plainbox plugin the install script is called with a prefix (python3 manage.py install --prefix=/providers/name), so this used to be doable without any conflicts. You could have a plainbox snap installing python dependencies from the archive.

This week, in order to support python snaps in classic [1], we introduced to the python plugin a rewrite to /etc/python3.5/sitecustomize.py. So now there is a new conflict that does affect the plainbox plugin. If it tries to install a python deb from the archive it will bring ubuntu's default /etc/python3.5/sitecustomize.py and the snap will fail.

To reproduce:

$ git clone https://github.com/snapcore/snapcraft /tmp/snapcraft
$ cd /tmp/snapcraft/integration_tests/snaps/plainbox-provider
Modify the snapcraft.yaml adding at the end:
stage-packages: [python3-petname]
$ /tmp/snapcraft/bin/snapcraft

Parts 'plainbox-local' and 'simple-plainbox-provider' have the followig file paths in common which have different contents:
    etc/python3.5/sitecustomize.py

[1] https://github.com/snapcore/snapcraft/pull/1093/files

Revision history for this message
Barry Warsaw (barry) wrote : Re: [Bug 1663792] [NEW] it is no longer possible to have a plainbox part with a python stage-package

On Feb 11, 2017, at 03:45 AM, Leo Arias wrote:

>It has never been possible to combine a python part that installs
>packages using the python install script, with a part that installs
>python packages from the ubuntu archive. This causes many conflicts.

You might consider using the dirtbike package to turn Python packages from the
Ubuntu archive into wheels (i.e. "rewheel"). Then you could install the
wheels from the archive and the normal packages otherwise.

We use dirtbike (albeit sparingly) in order to support Built-Using
dependencies for pip and friends, so I know it works fairly well, at least for
the relatively simple pure-Python packages that pip depends on.

Revision history for this message
Sergio Schvezov (sergiusens) wrote :

Thanks Barry.

Leo, notice that changing /etc/python3.5/sitecustomize.py was never the intention as is not even read by the interpreter on load as it is in a totally foreign path to the interpreter when run.

Revision history for this message
Sergio Schvezov (sergiusens) wrote :

Ah, it is a symlink! usr/lib/python3.5/sitecustomize.py -> ../../../etc/python3.5/sitecustomize.py

Filtering is our only option here and the team will need to be notified by it.

Changed in snapcraft:
importance: Undecided → Medium
assignee: nobody → Sergio Schvezov (sergiusens)
milestone: none → 2.27
status: New → In Progress
Revision history for this message
Sergio Schvezov (sergiusens) wrote :
Changed in snapcraft:
status: In Progress → Fix Committed
Revision history for this message
Leo Arias (elopio) wrote :

@Barry, if we can install them with pip, what do we win with dirtbike? Are there many packages in the ubuntu archive that are not in pypi?

Changed in snapcraft:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.