auto backend discovery at start time

[Bill Moyers]

Binary package hint: python-matplotlib

I am using Ubuntu 8.10. I installed the package python-matplotlib (0.98.3-4ubuntu1 (intrepid)). In my __init__.py I have:

import pylab

I am using Python 2.5.2. When I run the program I got the error:

Traceback (most recent call last):
  File "__init__.py", line 4, in <module>
    import pylab
  File "/usr/lib/python2.5/site-packages/pylab.py", line 1, in <module>
    from matplotlib.pylab import *
  File "/usr/lib/python2.5/site-packages/matplotlib/pylab.py", line 249, in <module>
    from matplotlib.pyplot import *
  File "/usr/lib/python2.5/site-packages/matplotlib/pyplot.py", line 75, in <module>
    new_figure_manager, draw_if_interactive, show = pylab_setup()
  File "/usr/lib/python2.5/site-packages/matplotlib/backends/__init__.py", line 23, in pylab_setup
    globals(),locals(),[backend_name])
  File "/usr/lib/python2.5/site-packages/matplotlib/backends/backend_tkagg.py", line 7, in <module>
    import Tkinter as Tk, FileDialog
  File "/usr/lib/python2.5/lib-tk/Tkinter.py", line 41, in <module>
    raise ImportError, str(msg) + ', please install the python-tk package'
ImportError: No module named _tkinter, please install the python-tk package

I installed the python-tk Ubuntu package and it cleared up. I guess a package dependency is missing?

[Benjamin Drung (bdrung)]

python-matplotlib 0.98.3-4ubuntu1 depends on "python-tk | python-gtk2 | python-wxgtk2.8 | python-qt3 | python-qt4". This means, that you need to have one of those packages installed. If you have none of those packages installed and you install python-matplotlib, python-tk will be installed. If you have one of those packages installed (except python-tk) and you install python-matplotlib, you have to install python-tk manually or change the default backend (in /etc/matplotlibrc).
You always need the python package installed wich correspond to the selected backend. I do not know how to assure that the correct python package is installed. The other way is to select the backend based on the installed packages. A plan for the future is to have a "auto" backend which searches for the installed packages and select the first he finds.

[Dimitri John Ledkov (xnox)]

An auto backend support within matplotlib would be great. I'll try to find bug tracker for matplotlib and see if there is a request already for this, if not I'll file one and help to add code to support that.

In the mean time we still have the all mightly debian packaging =DDDD

I think that it will be possible to write a post install script which will do the following:

1) check dependencies for all possible backends
2) Ignore backends for which dependencies are missing
3) Out of the working backends pick the pretties one
4) modify /etc/matplotlibrc to default to one that is installed

By bash-foo is not perfect but I will work on this in the next week or so. So for now I'll assign this bug to myself. If I succeed I'll work with debian packaging team to incorporate it =D

[Sandro Tosi (morph-debian)]

On Fri, Jan 23, 2009 at 03:56, Дмитрий Ледков <email address hidden> wrote:
> An auto backend support within matplotlib would be great. I'll try to
> find bug tracker for matplotlib and see if there is a request already
> for this, if not I'll file one and help to add code to support that.

it's on their SF website[1] even if -devel (or -user sometimes)
mailing-list is much more used than tracker.

[1] http://sourceforge.net/tracker/?group_id=80706

> I think that it will be possible to write a post install script which
> will do the following:
>
> 1) check dependencies for all possible backends
> 2) Ignore backends for which dependencies are missing
> 3) Out of the working backends pick the pretties one
> 4) modify /etc/matplotlibrc to default to one that is installed

what if install-time is weeks before the usage, so that packages gets
installed and removed in between?

> By bash-foo is not perfect but I will work on this in the next week or
> so. So for now I'll assign this bug to myself. If I succeed I'll work
> with debian packaging team to incorporate it =D

The best (only real?) solution is to let upstream provide an "auto"
mode, like they already said they're working on it.

Regards,
--
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

[Chris Samuel (chris-csamuel)]

Just ran into this problem in Jaunty when testing smem (mentioned on this weeks LWN front page), and also described in the smem list archive here by someone else: http://selenic.com/pipermail/smem/2009-April/000014.html

Apparently the Debian package does have python-tk as a requirement, as can be seen here - http://packages.debian.org/stable/python-matplotlib

[John Haitas (jhaitas)]

I can confirm this bug on Jaunty amd64

[John Haitas (jhaitas)]

here is a patch

[LaserJock (laserjock)]

John, that patch will make python-gtk a dependency, which is what we were trying to avoid. Perhaps we should using python-gd as a placeholder on the | so that we ensure python-tk is always installed without bringing in gtk/qt dependencies. It looks like that's what Debian did in the stable package as Chris pointed out.

[John Haitas (jhaitas)]

I am already working on a patch that moves python-gtk2 etc... to the
Suggests line...

would that produce the desired results?

--
John Haitas
<email address hidden>

[Dimitri John Ledkov (xnox)]

2009/5/5 John Haitas <email address hidden>:
> I am already working on a patch that moves python-gtk2 etc... to the
> Suggests line...
>
> would that produce the desired results?
>

Dunno... The idea here is that matplolib GUI has many types depending
on different backends.

A KDE user _can_ run the qt gui.
A Gnome user _can_ run the GTK gui.
etc.

At runtime the _available_ backends should be checked and one should
be picked which is curently available, ie. installed.

Due to python / apt nature what is avaiable at install time is
different from what is there at the runtime.

Ideally we should be able to detect that.

I got this idea to use Python-Apt bindings to write a small function
that will check with apt what's installed and pick the pretties one.
I'll work on this, should be easy. Hopefully I will have something
ready (patch, debdiff, PPA) in a few hours.

This will be Debian/Ubuntu solution. Forward looking when PackageKit
will catch on this solution can be ported to use PackageKit to be used
cross-distro.

--
With best regards

Dmitrijs Ledkovs (for short Dima),
Ледков Дмитрий Юрьевич

[John Haitas (jhaitas)]

That's why I went with the method I did. A guy in #ubuntu-motu thought my
method was wrong. I still think it is right to depend on python-tk and then
do the OR for the rest of the packages on the line. I know it will fix the
bug. I will try to have a PPA tomorrow. Please try my patch for now. This is
the first patch I have submitted. I would feel a huge sense of
accomplishment if I can get it right and receive credit for it.

[Sandro Tosi (morph-debian)]

2009/5/5 Дмитрий Ледков <email address hidden>:
> 2009/5/5 John Haitas <email address hidden>:
>> I am already working on a patch that moves python-gtk2 etc... to the
>> Suggests line...
>>
>> would that produce the desired results?
>>
>
> Dunno... The idea here is that matplolib GUI has many types depending
> on different backends.
>
> A KDE user _can_ run the qt gui.
> A Gnome user _can_ run the GTK gui.
> etc.
>
> At runtime the _available_ backends should be checked and one should
> be picked which is curently available, ie. installed.
>
> Due to python / apt nature what is avaiable at install time is
> different from what is there at the runtime.
>
> Ideally we should be able to detect that.

And we already talked with upstream authors to implement such functionality.

> I got this idea to use Python-Apt bindings to write a small function
> that will check with apt what's installed and pick the pretties one.
> I'll work on this, should be easy. Hopefully I will have something
> ready (patch, debdiff, PPA) in a few hours.

No, that should be done inside matplotlib: check what's binding is
available, choose the best.

> This will be Debian/Ubuntu solution.

The solution descrive above will enter Debian, not anything else.

Regards,
--
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

[Sandro Tosi (morph-debian)]

On Tue, May 5, 2009 at 06:09, John Haitas <email address hidden> wrote:
> That's why I went with the method I did.

If the patch is this [1] then it's wrong.

[1] http://launchpadlibrarian.net/26327836/matplotlib_0.98.5.2-1ubuntu4.debdiff

> A guy in #ubuntu-motu thought my
> method was wrong.

I agree with that guy

> I still think it is right to depend on python-tk and then
> do the OR for the rest of the packages on the line. I know it will fix the
> bug. I will try to have a PPA tomorrow. Please try my patch for now. This is
> the first patch I have submitted. I would feel a huge sense of
> accomplishment if I can get it right and receive credit for it.

No, read documentation we added in Debian packages about how to fix
this problem.

This is a configuration that needs to be done by the system
administrator, the one that installs the package and setup the
environment.

Regards,
--
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

[John Haitas (jhaitas)]

Given that there is not yet an upstream fix for this.

Can we just patch the current code so the packaging will work?

We can revisit the issue when a new version is released.

In the meantime, our users are going to struggle to understand what's going on.

Can we provide a working solution even if it is not the preferred long term solution?

[Sandro Tosi (morph-debian)]

On Fri, May 8, 2009 at 21:20, John Haitas <email address hidden> wrote:
> Given that there is not yet an upstream fix for this.
>
> Can we just patch the current code so the packaging will work?
>
> We can revisit the issue when a new version is released.
>
> In the meantime, our users are going to struggle to understand what's
> going on.
>
> Can we provide a working solution even if it is not the preferred long
> term solution?

The working solution is here:

/usr/share/doc/python-matplotlib/NEWS.Debian.gz
/usr/share/doc/python-matplotlib/README.Debian

I'm pretty sure Benjamin kept this file from Debian to Ubuntu. There
is documentation written ad-hoc for this change.

Regards,
--
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

[John Haitas (jhaitas)]

How bout we do an ubuntu4 that depends on python-tk

We can work on an ubuntu5 that selects the best option.

[Sandro Tosi (morph-debian)]

On Fri, May 8, 2009 at 22:14, John Haitas <email address hidden> wrote:
> How bout we do an ubuntu4 that depends on python-tk

Did you actually read the files I've mentioned above? that *is* part the change.

> We can work on an ubuntu5 that selects the best option.

Sure, please do it if you feel it's required; please give it back to
Debian then.

Regards,
--
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

[John Haitas (jhaitas)]

For now I will provide a patch that depends exclusively on 'python-tk' since this is the default configuration for the package as it is provided by upstream. An exclusive install depend on 'python-tk' will resolve THIS bug.

Users who are aware of other options should have the ability to create a ~/.matplotlib/matplotlibrc file that requests the 'backend' of their choice. Anyone with this knowledge should have the ability to install the appropriate packages to realize this.

We can open a new bug regarding automatic selection of the 'best' available backend.

[John Haitas (jhaitas)]

Here is the aforementioned ubuntu4.debdiff

[John Haitas (jhaitas)]

You can test the built package on your system using my PPA

https://launchpad.net/~jhaitas/+archive/python-matplotlib

deb http://ppa.launchpad.net/jhaitas/python-matplotlib/ubuntu jaunty main
deb-src http://ppa.launchpad.net/jhaitas/python-matplotlib/ubuntu jaunty main

This repository is signed with 1024R/3F4D6BEE OpenPGP key.

Dmitrijs, I am awaiting your feedback in particular.

[John Haitas (jhaitas)]

Sandro,
Is there a bug filed for this upstream at Debian?

[Benjamin Drung (bdrung)]

There is no bug filed in the Debian bug tracking system. The auto backend discovery at start time wish was discussed on the matplotlib-devel mailing list.

[Dimitri John Ledkov (xnox)]

LinkedIn
------------

Bug,

I'd like to add you to my professional network on LinkedIn.

- Dmitrijs Ledkovs

Confirm that you know Dmitrijs Ledkovs
https://www.linkedin.com/e/isd/1067430357/hDBkHOu-/

------
(c) 2010, LinkedIn Corporation