/usr/local/lib/python/sitecustomize.py broken due to overriding sitecustomize in /etc

[Carey Underwood]

sitecustomize.py files have been added to /etc/python2.5/ nad /usr/lib/python2.5/ in hardy, which breaks the use of /usr/local/lib/site-python/sitecustomize.py, silently causing it to not be imported.

My impression has always been that sitecustomize.py is to be reserved for the user (i.e., it's the python equivilent of debian's /usr/local/, thou shalt not touch it).

site.py already allows for distribution specific additions; if they are to be placed outside site.py, then they should be placed in a third file such that the import of sitecustomize.py still proceeds as expected by the user.

[Carey Underwood (cwillu)]

Actually, it looks like /usr/local/ isn't even being looked at for pythonpath anymore.

[Matthias Klose (doko)]

upstream did have some concerns modifying site.py directly. Plus a missing site.py (you can remove the config file /etc/python2.5/site.py) is not handled very well, while a missing sitecustomize is. There is no problem adding your site stuff in /etc/python2.5/site.py.

[T. Middleton (timtoo)]

This slightly broke a development box of mine a few months ago. I didn't look into it until now because it just caused some weirdness occassionally (my sitecustomize.py does some fiddling with encoding & locales to ensure cross-platform homogeneity which only usually affects corner cases in my application).

I'm not sure how it could be ever considered a good idea to effectively disable sitecustomize.py from site-python for everyone, by putting this in python's main lib directory. And even worse than a bad idea, it's non-standard.

I'm sad to see how old this bug report is that that it seems nothing has been done about it.

Luckily my production servers run on FreeBSD so this did not effect them.

It seems there is little to no official documentation about sitecustomize.py that I can find to provide any official guidelines. My personal understanding of sitecustomize.py is similar to the original reporter of this bug. The main references on the internet to it is the very popular http://diveintopython.org/xml_processing/unicode.html which suggests (somewhat controversially) using it in site-python: he'll need to make a footnote that this technique will work everywhere except *ubuntu.

[Matthias Klose (doko)]

I don't agree with the statement "broken". just because our current behaviour doesn't match the behaviour of a guidebook it's not just broken. I cannot find any hint in the standard configuration that this file has to be placed in /usr/local.

the file is a config file, and there is just one version which can be imported. IMO it is far better to have one file explicitely editable, which survives changes during upgrades, than having N copies laying around where you are not sure which one gets imported.

we should update our diveintopython package to reflect this behaviour on ubuntu. closing the python task.

[T. Middleton (timtoo)]

Listen to yourself, Mattias. Now not only are you justifying a python behavior (forcing a single sitecustomize.py to a particular directory, effectively crippling any other software from having their own sitecustomize.py), which no other operating system or linux distribution does that I am aware of, but then you are saying you think ubuntu should edit one of the most popular freely available books on the subject so that ubuntu's version is different from all other versions (including the one i presume most people, even on ubuntu, use: online) to reflect this arbitrary decision.

Just something to think about.

It seems the whole reason for this issue is ubuntu's apparent need to inject (and maintain injected) "apport_python_hook" into every single python process that runs on ubuntu. This seems a somewhat dubious practice to me, but that's another issue. If ubuntu really feels the need to do this globally, it seems to me better practice that ubuntu should patch it's python package at a lower level than hijacking the general sitecustomize.py mechanism for itself globally.

I understand that this has been decided and there's nothing I can do about it, except find some way to work around the issue on specifically ubuntu, which I don't have to do on any other platform, if I want to support this platform (which I do -- it just makes a bit sad that ubuntu resorts to this sort of arbitrary stuff in some areas).

I also understand this is a fairly rare thing that anyone's even going to even know about sitecustomize.py, but still I hope that in the future maybe a better solution can be worked out.

[jtniehof (jtniehof)]

Explicitly setting PYTHONPATH to /usr/local/lib/site-python does the trick OMM (16.04). Whether it's best to then copy-paste /etc/sitecustomize.py into the local one, get into imp/importlib machinery, or decide one doesn't care about apport, is left as an exercise to the reader.

[Thomas Güttler (hv-tbz-pariv)]

Ubuntu 18.04 does come with a sitecustomize.py file.

I think this is not a good solution.

I would really like to be able to execute some python code during interpreter start.

AFAIK sitecustomize.py is the way to got. But if ubuntu provides one, then this does not work.

Related question: https://stackoverflow.com/questions/52449602/how-to-set-environment-variables-in-virtualenv-started-via-python

root@1b3dffd4d058:/# python
Python 2.7.15rc1 (default, Apr 15 2018, 21:51:34)
>>> import sitecustomize
>>> sitecustomize.__file__
'/usr/lib/python2.7/sitecustomize.pyc'

root@1b3dffd4d058:/# cat /etc/os-release
NAME="Ubuntu"
VERSION="18.04 LTS (Bionic Beaver)"