twistd scripts don't work when installed

[Matt Conway]

I installed the latest graphite from bazaar (v426):

          mkdir -p /tmp/graphite_install
          cd /tmp/graphite_install
          bzr branch lp:graphite

          pushd graphite/whisper
          python setup.py install
          popd

          pushd graphite/carbon
          python setup.py install
          popd

          pushd graphite
          python setup.py install
          popd

But when I try to rub carbon, I get an exception:

# cd /opt/graphite
# ./bin/carbon-cache.py start
Traceback (most recent call last):
  File "/opt/graphite/bin/carbon-cache.py", line 30, in <module>
    run_twistd_plugin(__file__)
  File "/opt/graphite/lib/carbon/util.py", line 118, in run_twistd_plugin
    config.parseOptions(twistd_options)
  File "/usr/lib/python2.7/dist-packages/twisted/application/app.py", line 606, in parseOptions
    usage.Options.parseOptions(self, options)
  File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 234, in parseOptions
    raise UsageError("Unknown command: %s" % sub)
twisted.python.usage.UsageError: Unknown command: carbon-cache

[Matt Conway (wr0ngway)]

Looks like the twisted and twisted.plugins packages are missing from carbon/setup.py (ant he associated __init__.py in the associated directories)

I tried adding them in, but then util.py can't find the system installed twisted package - I don't know enough about python to say how to fix this.

root@matt:/opt/graphite# ./bin/carbon-cache.py start
Traceback (most recent call last):
  File "./bin/carbon-cache.py", line 28, in <module>
    from carbon.util import run_twistd_plugin
  File "/opt/graphite/lib/carbon/util.py", line 8, in <module>
    from twisted.python.util import initgroups
ImportError: No module named python.util

[Nicholas Leskiw (nleskiw)]

I'll take a look.

-Nick

On Jul 18, 2011, at 10:11 AM, wr0ngway <email address hidden> wrote:

> Looks like the twisted and twisted.plugins packages are missing from
> carbon/setup.py (ant he associated __init__.py in the associated
> directories)
>
> I tried adding them in, but then util.py can't find the system installed
> twisted package - I don't know enough about python to say how to fix
> this.
>
>
> root@matt:/opt/graphite# ./bin/carbon-cache.py start
> Traceback (most recent call last):
> File "./bin/carbon-cache.py", line 28, in <module>
> from carbon.util import run_twistd_plugin
> File "/opt/graphite/lib/carbon/util.py", line 8, in <module>
> from twisted.python.util import initgroups
> ImportError: No module named python.util
>
> --
> You received this bug notification because you are subscribed to
> Graphite.
> https://bugs.launchpad.net/bugs/812279
>
> Title:
> twistd scripts don't work when installed
>
> Status in Graphite - Enterprise scalable realtime graphing:
> New
>
> Bug description:
> I installed the latest graphite from bazaar (v426):
>
> mkdir -p /tmp/graphite_install
> cd /tmp/graphite_install
> bzr branch lp:graphite
>
> pushd graphite/whisper
> python setup.py install
> popd
>
> pushd graphite/carbon
> python setup.py install
> popd
>
> pushd graphite
> python setup.py install
> popd
>
> But when I try to rub carbon, I get an exception:
>
> # cd /opt/graphite
> # ./bin/carbon-cache.py start
> Traceback (most recent call last):
> File "/opt/graphite/bin/carbon-cache.py", line 30, in <module>
> run_twistd_plugin(__file__)
> File "/opt/graphite/lib/carbon/util.py", line 118, in run_twistd_plugin
> config.parseOptions(twistd_options)
> File "/usr/lib/python2.7/dist-packages/twisted/application/app.py", line 606, in parseOptions
> usage.Options.parseOptions(self, options)
> File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 234, in parseOptions
> raise UsageError("Unknown command: %s" % sub)
> twisted.python.usage.UsageError: Unknown command: carbon-cache
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/graphite/+bug/812279/+subscriptions

[Matt Conway (wr0ngway)]

As a test, I tried removing the __init__.py from the install dir /opt/graphite/lib/twisted/[plugins], and the script then seems to run. However, setup.py complains if they aren't there in the source directory, and then adds them to the install directory, so not sure what the solution is. Some way to get setup.py to copy directory trees without treating them as packages?

Basically, I think graphite (twisted?) relies on the plugin files being in a known package so that they get automatically loaded (my original problem). However, having them in the source tree as "packages" causes them to shadow the system package.

[Andreas Hasenack (ahasenack)]

Same happens here, carbon doesn't run, same backtrace as the OP. I'm on lucid, python 2.6.

[Nicholas Leskiw (nleskiw)]

I'm pretty sure this has been fixed. Please confirm.
If I don't hear back I'll close this bug...

[Jon Schedler - IMVU (jschedler)]

I pulled the latest head this morning and I get this error (slightly different traceback though)

root@jschedler-sandbox:/opt/graphite# sudo python bin/carbon-cache.py --debug --config=/opt/graphite/conf/carbon.conf start
Traceback (most recent call last):
  File "bin/carbon-cache.py", line 30, in <module>
    run_twistd_plugin(__file__)
  File "/opt/graphite/lib/carbon/util.py", line 79, in run_twistd_plugin
    config.parseOptions(twistd_options)
  File "/usr/lib/python2.6/dist-packages/twisted/application/app.py", line 668, in parseOptions
    usage.Options.parseOptions(self, options)
  File "/usr/lib/python2.6/dist-packages/twisted/python/usage.py", line 234, in parseOptions
    raise UsageError("Unknown command: %s" % sub)
twisted.python.usage.UsageError: Unknown command: carbon-cache

[Nicholas Leskiw (nleskiw)]

Hmmm I thought the current revision was in the high 570's are you using the 1.1 branch?

Jon Schedler - IMVU <email address hidden> wrote:

>I pulled the latest head this morning and I get this error (slightly
>different traceback though)
>
>root@jschedler-sandbox:/opt/graphite# sudo python bin/carbon-cache.py --debug --config=/opt/graphite/conf/carbon.conf start
>Traceback (most recent call last):
> File "bin/carbon-cache.py", line 30, in <module>
> run_twistd_plugin(__file__)
> File "/opt/graphite/lib/carbon/util.py", line 79, in run_twistd_plugin
> config.parseOptions(twistd_options)
> File "/usr/lib/python2.6/dist-packages/twisted/application/app.py", line 668, in parseOptions
> usage.Options.parseOptions(self, options)
> File "/usr/lib/python2.6/dist-packages/twisted/python/usage.py", line 234, in parseOptions
> raise UsageError("Unknown command: %s" % sub)
>twisted.python.usage.UsageError: Unknown command: carbon-cache
>
>--
>You received this bug notification because you are subscribed to
>Graphite.
>https://bugs.launchpad.net/bugs/812279
>
>Title:
> twistd scripts don't work when installed
>
>Status in Graphite - Enterprise scalable realtime graphing:
> New
>
>Bug description:
> I installed the latest graphite from bazaar (v426):
>
> mkdir -p /tmp/graphite_install
> cd /tmp/graphite_install
> bzr branch lp:graphite
>
> pushd graphite/whisper
> python setup.py install
> popd
>
> pushd graphite/carbon
> python setup.py install
> popd
>
> pushd graphite
> python setup.py install
> popd
>
> But when I try to rub carbon, I get an exception:
>
> # cd /opt/graphite
> # ./bin/carbon-cache.py start
> Traceback (most recent call last):
> File "/opt/graphite/bin/carbon-cache.py", line 30, in <module>
> run_twistd_plugin(__file__)
> File "/opt/graphite/lib/carbon/util.py", line 118, in run_twistd_plugin
> config.parseOptions(twistd_options)
> File "/usr/lib/python2.7/dist-packages/twisted/application/app.py", line 606, in parseOptions
> usage.Options.parseOptions(self, options)
> File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 234, in parseOptions
> raise UsageError("Unknown command: %s" % sub)
> twisted.python.usage.UsageError: Unknown command: carbon-cache
>
>To manage notifications about this bug go to:
>https://bugs.launchpad.net/graphite/+bug/812279/+subscriptions

[chrismd (chrismd)]

Also what version of twisted are you using?

[chrismd (chrismd)]

This was a problem with the way twisted caches modules and conflicting with the way we were importing modules. It's been fixed in trunk.

[Kuba Berlinski (berlinski-jakub)]

I have the same problem with v. 599.

 I use Centos 6.0 image thoughtpolice:
http://www.thoughtpolice.co.uk/vmware/#centos6.0

I install python (2.6.5) and other required items:
********************************
dhclient eth0
yum install -y python bzr pycairo wget unzip
mkdir -p /tmp/graphite_install
cd /tmp/graphite_install
wget http://www.djangoproject.com/download/1.3.1/tarball/
wget http://django-tagging.googlecode.com/files/django-tagging-0.3.1.zip
tar -zxvf Django-1.3.1.tar.gz
cd Django-1.3.1
python setup.py install
cd ..
unzip django-tagging-0.3.1.zip
cd django-tagging-0.3.1/
python setup.py install
cd ..
********************************

... then i get the latest revision (599) from bazar:

********************************
bzr branch lp:graphite
pushd graphite/whisper
python setup.py install
popd
pushd graphite/carbon
python setup.py install
popd
pushd graphite
python setup.py install
popd
********************************

I try to configure and run the application:

********************************
pushd /opt/graphite/conf
cp carbon.conf.example carbon.conf
cp storage-schemas.conf.example storage-schemas.conf
popd
cd /opt/graphite/webapp/graphite
python manage.py syncdb
cp local_settings.py.example local_settings.py
cd /opt/graphite/
./bin/carbon-cache.py start
********************************

... and I get the same error message:

[root@localhost graphite]# ./bin/carbon-cache.py start
Traceback (most recent call last):
  File "./bin/carbon-cache.py", line 28, in <module>
    from carbon.util import run_twistd_plugin
  File "/opt/graphite/lib/carbon/util.py", line 17, in <module>
    from twisted.python.util import initgroups
ImportError: No module named twisted.python.util

What am I missing?

[Kuba Berlinski (berlinski-jakub)]

Well, I managed to move a step further after installing Twisted and zope.interface manually...

yum install python-devel
wget http://twistedmatrix.com/Releases/Twisted/11.0/Twisted-11.0.0.tar.bz2
bunzip2 Twisted-11.0.0.tar.bz2
tar -xvf Twisted-11.0.0.tar
python setup.py install
wget http://pypi.python.org/packages/source/z/zope.interface/zope.interface-3.8.0.tar.gz#md5=8ab837320b4532774c9c89f030d2a389
tar -zxvf zope.interface-3.8.0.tar.gz
cd zope.interface-3.8.0
python setup.py install
cd /opt/graphite/

[root@localhost graphite]# bin/carbon-cache.py start
Starting carbon-cache (instance a)
[root@localhost graphite]# bin/carbon-cache.py status
carbon-cache (instance a) is running with pid 1467

looks much better now...

[Michael Leinartas (mleinartas)]

Released in 0.9.9

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Ross Gammon (rosco2)]

This was apparently fixed by the Graphite project in 0.9.9. As the currently supported Ubuntu versions all have at least 0.9.12, could you please confirm whether this issue has been fixed for you?