broken symlinks when snapping a python project using requirements.txt

[XiaoGuo, Liu]

The problem happens on my vivid 15.04 desktop. I have a python project at the location:

https://github.com/liu-xiao-guo/restapi

Basically, it is a python django webserver project. it works pretty well on my desktop environment. I have tried to put the needed python component into the requirements.txt file, and it looks like:

Django==1.9
argparse==1.2.1
djangorestframework==3.3.1
wsgiref==0.1.2

When I tried to compile the project, and it compiles successfully. However, after I deploy it onto the KVM environment, it does not work any more. After some debugging, I found the following broken link inside the snap installation.

(amd64)ubuntu@localhost:/apps/restapi.sideload/current/usr/lib/python2.7$ ls -al site-packages
lrwxrwxrwx 1 clickpkg clickpkg 89 Dec 7 04:13 site-packages -> /home/liuxg/snappy/tutorials/restapi/parts/django/install/usr/lib/python2.7/dist-packages

It is in fact a symbolic link in the file system. http://paste.ubuntu.com/13780734/

Thanks & best regards,
XiaoGuo

[Daniel Holbach (dholbach)]

In this case it would have been nice, if the reviewers tools would have complained about the broken symlink.

[Daniel Holbach (dholbach)]

Suggested patch. We need to follow symlinks, so we can find offenders, like this one:

daniel@daydream:~$ dpkg -c /home/daniel/dev/snappy/snapcraft/examples/py3-project/spongeshaker*.snap | grep home
lrwxrwxrwx root/root 0 2015-12-07 11:45 ./usr/lib/python3.5/site-packages -> /home/daniel/dev/snappy/snapcraft/examples/py3-project/parts/spongeshaker/install/usr/lib/python3/dist-packages
daniel@daydream:~$

[Daniel Holbach (dholbach)]

Jamie: does the patch look all right to you? The test suite still passes.

[Kyle Fazzari (kyrofa)]

Snapcraft bugfix here: https://github.com/ubuntu-core/snapcraft/pull/169

[Kyle Fazzari (kyrofa)]

Note that, while the broken symlink was a bug (thank you!), I'm not convinced that it's your problem, since your webserver is only running on loopback:

$ sudo netstat -tulpn
<snip>
tcp 0 0 127.0.0.1:8090 0.0.0.0:* LISTEN 2059/python

[XiaoGuo, Liu (liu-xiao-guo)]

Hi Kyle,

Thanks for your investigation. Yes, you are right. My server is running on the loopback. How can I fix it? Is this another issue?

Thanks & best regards,
XiaoGuo

[Kyle Fazzari (kyrofa)]

I'm not a Django pro, but perhaps try changing the service definition in your snapcraft.yaml to `start: python bin/manage.py runserver 0.0.0.0:8090` ?

[Kyle Fazzari (kyrofa)]

Note that this bug also affected Python3 projects. The fix includes both.

[XiaoGuo, Liu (liu-xiao-guo)]

Hi Kyle,

Thanks! This morning, I tried to change to run the service the same as what you suggested, and it worked. Sorry, I did not feedback it since I was on a trip back to my home.

I just re-pulled the latest snapcraft, and I rebuilt the whole project, and now it worked as expected. However, there is one thing which is confusing. I used the snapcraft in the bin directory of repository:

liuxg@liuxg:~/snappy/tutorials/restapi$ ../../snapcraft/bin/snapcraft

liuxg@liuxg:~/snappy/tutorials/restapi$ ../../snapcraft/bin/snapcraft version
snapcraft (0.5).
Run "snapcraft help" to get started.

 After installation, i still see:

(amd64)ubuntu@localhost:/apps/restapi.sideload/current/usr/lib/python2.7$ ls -al site-packages
lrwxrwxrwx 1 clickpkg clickpkg 89 Dec 17 15:27 site-packages -> /home/liuxg/snappy/tutorials/restapi/parts/django/install/usr/lib/python2.7/dist-packages

Is there anything wrong with it?

Thanks & best regards,
XiaoGuo

[Kyle Fazzari (kyrofa)]

Ah, the bugfix hasn't been merged into master just yet, so you're still going to get broken symlinks until that lands. We already knew that wasn't your problem-- you just confirmed it!

If you want to try out the fixed version, go to the pull request linked above and try THAT branch.

[Jamie Strandboge (jdstrand)]

@Daniel, I'm uncomfortable with the change to the review tools to follow symlinks. They are always run on the non-target system so following the symlink would have succeeded on the developers machine, possibly in the store's environment but likely not on the target machine. The review tools do already have a test to see if symlinks are pointing outside of the app's area (check_external_symlinks() in cr_lint.py). Perhaps there is a bug in there?

[Daniel Holbach (dholbach)]

Jamie, the external symlink does not end up in self.pkg_files.

[Jamie Strandboge (jdstrand)]

Closing due to inactivity.