Graphite fails to install in a virtualenv

[Kamil Kisiel]

I created a virtualenv and attempted to install graphite, but it seems to have some hard-coded paths to /opt:

copying webapp/graphite/render/graphTemplates.conf -> build/lib/graphite/render

running build_scripts

creating build/scripts-2.7

copying and adjusting bin/run-graphite-devel-server.py -> build/scripts-2.7

changing mode of build/scripts-2.7/run-graphite-devel-server.py from 644 to 755

running install_lib

creating /opt/graphite

error: could not create '/opt/graphite': Permission denied

[Kamil Kisiel (kisielk)]

Steps I took to attempt install it:

virtualenv graphite
source graphite/bin/activate
pip install graphite

Ideally this should get graphite and all its dependencies from PyPi and install them in to the virtualenv.

[Nicholas Leskiw (nleskiw)]

Graphite does in fact use hardcoded paths.

Someone tried to make everything relative once, but it's trickier than you'd
think.
Chris can speak to this more, but I'd recommend installing it in
/opt/graphite

-Nick

On Fri, Jun 10, 2011 at 3:18 PM, Kamil Kisiel <email address hidden> wrote:

> Steps I took to attempt install it:
>
> virtualenv graphite
> source graphite/bin/activate
> pip install graphite
>
> Ideally this should get graphite and all its dependencies from PyPi and
> install them in to the virtualenv.
>
> --
> You received this bug notification because you are subscribed to
> Graphite.
> https://bugs.launchpad.net/bugs/795713
>
> Title:
> Graphite fails to install in a virtualenv
>
> Status in Graphite - Enterprise scalable realtime graphing:
> New
>
> Bug description:
> I created a virtualenv and attempted to install graphite, but it seems
> to have some hard-coded paths to /opt:
>
> copying webapp/graphite/render/graphTemplates.conf ->
> build/lib/graphite/render
>
> running build_scripts
>
> creating build/scripts-2.7
>
> copying and adjusting bin/run-graphite-devel-server.py ->
> build/scripts-2.7
>
> changing mode of build/scripts-2.7/run-graphite-devel-server.py from
> 644 to 755
>
> running install_lib
>
> creating /opt/graphite
>
> error: could not create '/opt/graphite': Permission denied
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/graphite/+bug/795713/+subscriptions
>

[chrismd (chrismd)]

Nick I think you're thinking of the URL paths, an attempt was made to make them relative instead of absolute and that ended up getting reverted. The hard-coded filesystem paths however are actually relative, you can install Graphite anywhere and the installation prefix (which is typically "/opt/graphite") gets figured out dynamically.

The reason Graphite doesn't install cleanly in a virtualenv is because there is some ugly stuff done in the webapp's setup.py, including non-code alongside code (like graphTemplates.conf), etc. All of that can be cleaned up and once its done I imagine things should work fine in virtualenv.

First off, graphTemplates.conf should go away, or really just be re-implemented cleanly (it wasn't a bad idea, just poorly implemented). Also it needs to live in $GRAPHITE_ROOT/conf/, not under webapp/.

[Mikhail Korobov (kmike84)]

The paths are also hardcoded in setup.cfg. This file is included in source distribution and I think this prevents 'pip install graphite-web' because there is no way to override values defined in this file (or at least I can't find one).

I was able to install graphite to virtualenv this way:

1. Download the source tarball from pypi;
2. remove the setup.cfg file;
3. run setup.py with --prefix and --install-lib params:

python setup.py install --prefix=/Users/kmike/envs/zina/graphite --install-lib=/Users/kmike/envs/zina/graphite/webapp

After that,

run-graphite-devel-server.py /Users/kmike/envs/zina/graphite

seems to work but I haven't dig deeper.

So I think it will be possible to install graphite-web to virtualenv using pip by just removing setup.cfg from the source distribution:

pip install graphite-web --install-option="--prefix=<env>/graphite" --install-option="--install-lib=<env_path>/graphite/webapp"

[Jason Moiron (jmoiron)]

You can install using those options directly from the tgz without modifying the setup.cfg or removing it. For instance:

pip install http://launchpad.net/graphite/1.0/0.9.8/+download/carbon-0.9.8.tar.gz --install-option="--install-scripts=~/.ve/<env>/bin/" --install-option="--prefix=~/.ve/<env>/graphite"

This works.

It would still be nice for it to play nicely with virtualenv by default.

[brainstorm (brainstorm)]

Jason, I cannot reproduce your command, it fails for me:

creating /opt/graphite

error: could not create '/opt/graphite': Permission denied

----------------------------------------
Command /home/romanvg/.virtualenvs/devel/bin/python -c "import setuptools;__file__='/tmp/pip-xs6a85-build/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-managed --record /tmp/pip-mK1VfA-record/install-record.txt --install-headers /home/romanvg/.virtualenvs/devel/include/site/python2.6 --install-scripts=/home/romanvg/.virtualenvs/devel/bin/ --prefix=/home/romanvg/.virtualenvs/devel/graphite failed with error code 1

[Dan Colish (dcolish-deactivatedaccount)]

I've found that the best way to get around this is to create a graphite user and make its home dir /opt/graphite. You can then easily create a virtualenv in the users home dir and install graphite into it. Obviously there are other ways via pip commands, but then you will have to reconfigure graphite to use the new paths internally as well. With my approach, none of that complexity exists.

[Dave Rawks (drawks)]

This behavior is still present and the work arounds suggested don't actually address the root problem.

[Marshall Pierce (marshallpierce)]

I had the same problem as comment #6 with the approach in #5. I got things working by tweaking the prefix in setup.cfg for carbon and graphite-web to be my virtualenv.

[Michael Leinartas (mleinartas)]

This is going to remain a pain in the ass as long as the /opt/graphite path is used (at least until/unless setup.py is revamped) but I believe graphite is usable in virtualenv now with some effort.

I've attempted to document this here: http://readthedocs.org/docs/graphite/en/0.9.x/install-virtualenv.html

Would love if someone using 0.9.10 could verify this

[Kamil Kisiel (kisielk)]

I got carbon to work from source by simply editing setup.cfg and commenting out the lines referring to the install prefix. It then installed in the virtualenv just fine. Without those lines there, it should be possible to even install it directly from PyPI via pip which I think would be a huge win :)

After that I just had to go to $VIRTUAL_ENV/conf and run:

for f in *.example; do cp $f ${f/.example/}; done
carbon-aggregator.py start

and I was up and running. No need for root access or a system-wide install.

The --prefix and --install-lib switches could be used when building RPMs or DEBs to preserve the existing behaviour there, but I don't think they should be the default in order to facilitate virtualenv installation.

I haven't tried anything with graphite-web yet.