Please merge matplotlib 1.0.1-2 (universe) from debian unstable main

[Ben Gamari]

Version 1.0.1 was released earlier this month with several important API additions and bugfixes. It would be nice to see this in next ubuntu release.

See comment #40 and #41 for the new debdiff vs 1.0.1-2 from unstable

[Ben Gamari (bgamari)]

Please see my PPA[1] for a building, working package. Timely integration would be greatly appreciated, as I have some software that depends on 1.0.0 API.

[1] https://launchpad.net/~bgamari/+archive/matplotlib-unofficial

[Sandro Tosi (morph-debian)]

On Wed, Jul 21, 2010 at 19:15, Ben Gamari <email address hidden> wrote:
> Please see my PPA[1] for a building, working package. Timely integration
> would be greatly appreciated, as I have some software that depends on
> 1.0.0 API.
>
> [1] https://launchpad.net/~bgamari/+archive/matplotlib-unofficial

Sure update the package is quite easy, make it right it's another story.

The main blocker for the upload into Debian (and hence the sync with
ubuntu) is due to the get_sample_date() calls inside the examples in
doc generation. It's forbidden (at least in Debian, I don't know in
ubuntu) to acess (and download) anything at build time.

Follow up [1] and propose a patch: that would help us both Debian and
ubuntu (oh, and upstream too).

[1] http://thread.gmane.org/gmane.comp.python.matplotlib.devel/8865

Regards,
--
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

[Ben Gamari (bgamari)]

Thanks for your note! I'll look into it.

[Jouni K. Seppänen (jks)]

Please see

http://thread.gmane.org/gmane.comp.python.matplotlib.devel/9377
http://article.gmane.org/gmane.comp.python.matplotlib.scm/5157

for a way to disable the downloading mechanism. Does this help with the packaging?

[Ben Gamari (bgamari)]

See my PPA[1] for a first-cut at bringing matplotlib up-to-date. Unfortunately, more working is necessary[2].

[1] https://launchpad.net/~bgamari/+archive/matplotlib-unofficial
[2] http://thread.gmane.org/gmane.comp.python.matplotlib.devel/

[Achim Bohnet (allee)]

FWIW: 1.0.1 was released 10. Jan 2011: http://sourceforge.net/projects/matplotlib/files/matplotlib/matplotlib-1.0.1/
and today the 3rd scientist asked for the latest version as he/she miss features not in 0.9.9

[Achim Bohnet (allee)]

Solved! From Changelog of 1.0.1 (http://matplotlib.sourceforge.net/_static/CHANGELOG

2010-11-07 New rc parameters examples.download and examples.directory
           allow bypassing the download mechanism in get_sample_data.
           - JKS

This looks like 1.0.1 the prinzipal problem is fixed. Source size indicates this too:

-rw-r--r-- 1 ach ach 13285166 2011-01-06 14:59 matplotlib_1.0.1.orig.tar.gz
-rw-r--r-- 1 ach ach 61753323 2011-01-02 05:25 matplotlib_1.0.1-rc1-1ubuntu5.tar.gz

@Ben: A bit problematic is that you did use 1.0.1-rc1 instead of 1.0.1~rc1. An official natty upload would use 1.0.1-0ubuntu1 which is smaller than your version.

Any MOTU reading this? ;)

[Ben Gamari (bgamari)]

I believe that maintainer of the debian package has been working on bumping matplotlib in the debian repository. I'll poke him to see what the status of this is.

[Ben Gamari (bgamari)]

As the title now reflects, matplotlib 1.0.1 is now in debian experimental. Please pull into maverick updates. Thanks!

[Achim Bohnet (allee)]

As maverick is released it needs an SRU request. But bevor this can happen it required to be in natty:

For natty: DebianImportFreeze was 30th December ( https://wiki.ubuntu.com/NattyReleaseSchedule ) so we need to follow

  https://wiki.ubuntu.com/SyncRequestProcess

As there are ubuntu spec. changes they need to be merged etc. Any python MOTU subscribed that is familar to the task?

[Ben Gamari (bgamari)]

Ping.

[Alex Valavanis (valavanisalex)]

I have merged the packages. Unfortunately, python2.7-dbg crashes when debian/rules clean runs the line:

python2.7-dbg ./setup.py clean --all

This prevents me from creating the source package. The command is still run using the non-debug version of python2.7, so hopefully this isn't an issue. I have commented out the python2.7-dbg line from debian/rules.

Attaching debdiffs

[Sandro Tosi (morph-debian)]

On Tue, Mar 1, 2011 at 11:38, Alex Valavanis <email address hidden> wrote:
> I have merged the packages.  Unfortunately, python2.7-dbg crashes when
> debian/rules clean runs the line:
>
> python2.7-dbg ./setup.py clean --all
>
> This prevents me from creating the source package.  The command is still
> run using the non-debug version of python2.7, so hopefully this isn't an
> issue.  I have commented out the python2.7-dbg line from debian/rules.

What about trying to triage the problem, identify the culprit and fix
it, instead of work around it?

--
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

[Alex Valavanis (valavanisalex)]

Yes, that's the idea. I'll leave the report as "In progress" until I have finished investigating this. Any help would be appreciated!

[Alex Valavanis (valavanisalex)]

Curious... the source package is created fine on Maverick. Maybe a problem with python2.7-dbg in Natty?

New debdiffs attached.

[Alex Valavanis (valavanisalex)]

The crash happens when setupext.check_for_gtk attempts to import gtk. This only happens when the script is run with python2.7-dbg... the script runs fine with python2.6, python2.7 and python2.6-dbg.

[Sandro Tosi (morph-debian)]

On Tue, Mar 1, 2011 at 19:29, Alex Valavanis <email address hidden> wrote:
> The crash happens when setupext.check_for_gtk attempts to import gtk.
> This only happens when the script is run with python2.7-dbg... the
> script runs fine with python2.6, python2.7 and python2.6-dbg.

GTK+ bindings not properly rebuilt/working on 2.7 debug interpreter?
if you run python2.7-dbg -c "import gtk" what do you got?

--
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

[Alex Valavanis (valavanisalex)]

Actually, it goes a bit deeper...

$ python2.7-dbg
>>> import glib
Segmentation fault (core dumped)

[Alex Valavanis (valavanisalex)]

@Sandro - I get a simple segfault message, just as in comment #20

[Sandro Tosi (morph-debian)]

On Tue, Mar 1, 2011 at 20:09, Alex Valavanis <email address hidden> wrote:
> @Sandro - I get a simple segfault message, just as in comment #20

seen that, so it's a bug in glib

--
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

[Ben Gamari (bgamari)]

On Tue, 01 Mar 2011 19:20:26 -0000, Sandro Tosi <email address hidden> wrote:
> On Tue, Mar 1, 2011 at 20:09, Alex Valavanis <email address hidden> wrote:
> > @Sandro - I get a simple segfault message, just as in comment #20
>
> seen that, so it's a bug in glib
>
Today I took the debian package and compiled it on my Maverick
machine. The packages worked fine on my machine but they segfaulted on
another Maverick machine upon importing pyplot. I tried putting a
debugger on this but even with debugging symbols didn't get anything of
much help. The crash appears to happen in init_path
(src/_path.cpp). Regardless upon recompiling on the second machine it
worked and I've been unable to reproduce on another machine.

[Alex Valavanis (valavanisalex)]

@Ben - the trouble is that the package will need to be accepted into Natty before it can be backported to Maverick. Unless we can get it to build in a clean Natty environment, we're stuck!

[Ben Gamari (bgamari)]

Sure, I was just remarking about the crash for others' reference.

[Alex Valavanis (valavanisalex)]

It turns out that the python2.7-dbg crash goes away as long as the python-gobject-dbg package is installed. Hopefully, the package will now build in natty if we add "Build-Depends: python-gobject-dbg". I have created LP: #727649 to deal with the bug in python, but it should no longer hold us back. I'll try rebuilding the package and let you know.

[Alex Valavanis (valavanisalex)]

Updating debdiffs to include python-gobject-dbg build-dependency

[Alex Valavanis (valavanisalex)]

Successful PPA build on i386 & amd64: https://launchpad.net/~valavanisalex/+archive/matplotlib

[Alex Valavanis (valavanisalex)]

(now testing locally with natty amd64 pbuilder)

[Alex Valavanis (valavanisalex)]

Local build failed in debian/rules binary-indep:

ln -sf /usr/share/javascript/jquery/jquery.js \
  /tmp/buildd/matplotlib-1.0.1/debian/python-matplotlib-doc/usr/share/doc/python-matplotlib-doc/html/_static/jquery.js
ln: creating symbolic link `/tmp/buildd/matplotlib-1.0.1/debian/python-matplotlib-doc/usr/share/doc/python-matplotlib-doc/html/_static/jquery.js': No such file or directory
make: *** [binary-indep] Error 1
dpkg-buildpackage: error: fakeroot debian/rules binary gave error exit status 2
E: Failed autobuilding of package

...so the binary-arch rule copes fine in the PPA build, but the binary-indep rule is broken. Build log attached.

Any suggestions why this might be happening?

[WeatherGod (ben-v-root)]

I haven't figured out your particular error, but browsing through the build log revealed that the local copy of the sample data is incomplete:

/tmp/buildd/matplotlib-1.0.1/build/lib.linux-x86_64-2.7/matplotlib/sphinxext/plot_directive.py:322: PlotWarning: Exception running plot /tmp/buildd/matplotlib-1.0.1/doc/mpl_examples/pylab_examples/image_demo2.py
Traceback (most recent call last):
  File "/tmp/buildd/matplotlib-1.0.1/build/lib.linux-x86_64-2.7/matplotlib/sphinxext/plot_directive.py", line 319, in render_figures
    run_code(plot_path, function_name, plot_code, context=context)
  File "/tmp/buildd/matplotlib-1.0.1/build/lib.linux-x86_64-2.7/matplotlib/sphinxext/plot_directive.py", line 230, in run_code
    "__plot__", fd, fname, ('py', 'r', imp.PY_SOURCE))
  File "image_demo2.py", line 9, in <module>
IOError: [Errno 2] No such file or directory: '/tmp/buildd/matplotlib-1.0.1/sampledata/ct.raw'

In addition, the log also revealed a slight logical error on the part of upstream matplotlib, because it attempts to generate a basemap image, but matplotlib is not dependent on basemap -- minor issue, though. There are a few other build errors, but I believe they were fixed in subsequent revisions to matplotlib 1.0.x branch.

[Sandro Tosi (morph-debian)]

On Wed, Mar 2, 2011 at 16:08, WeatherGod <email address hidden> wrote:
> I haven't figured out your particular error, but browsing through the
> build log revealed that the local copy of the sample data is incomplete:
>
> /tmp/buildd/matplotlib-1.0.1/build/lib.linux-x86_64-2.7/matplotlib/sphinxext/plot_directive.py:322: PlotWarning: Exception running plot /tmp/buildd/matplotlib-1.0.1/doc/mpl_examples/pylab_examples/image_demo2.py
> Traceback (most recent call last):
>  File "/tmp/buildd/matplotlib-1.0.1/build/lib.linux-x86_64-2.7/matplotlib/sphinxext/plot_directive.py", line 319, in render_figures
>    run_code(plot_path, function_name, plot_code, context=context)
>  File "/tmp/buildd/matplotlib-1.0.1/build/lib.linux-x86_64-2.7/matplotlib/sphinxext/plot_directive.py", line 230, in run_code
>    "__plot__", fd, fname, ('py', 'r', imp.PY_SOURCE))
>  File "image_demo2.py", line 9, in <module>
> IOError: [Errno 2] No such file or directory: '/tmp/buildd/matplotlib-1.0.1/sampledata/ct.raw'

known error, that example will simple be skipped, irrelevant to this discussion.

>
> In addition, the log also revealed a slight logical error on the part of upstream matplotlib, because it attempts to generate a basemap image, but matplotlib is not dependent on basemap -- minor issue, though.

known too, irrelevant for this discussion.

>  There are a few other build errors, but I believe they were fixed in subsequent revisions to matplotlib 1.0.x branch.

why do you believe that? did you report them upstream? did you check
upstream VCS?

--
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

[WeatherGod (ben-v-root)]

I believe that because I am one of the upstream maintainers. Please see the commit log for 1.0.x branch

https://github.com/matplotlib/matplotlib/commits/v1.0.x

In particular, the commit for 2011-02-23. I don't know what you have included for this package though.

[Dave Walker (davewalker)]

Just a note, that if this sync is going to happen this cycle - it requires an FFe from the release team, as we are quite far into the cycle. Someone needs to complete:
https://wiki.ubuntu.com/FreezeExceptionProcess

Thanks.

[Ben Gamari (bgamari)]

Has anyone submitted an FFe? Alex, I'm assuming the package in your PPA is the package that will be merged?

[Alex Valavanis (valavanisalex)]

Hi Ben,

I'm afraid I haven't been able to finish working on this yet, so I will unassign myself for now. Unfortunately, the version in my PPA is not suitable for release because binary-independent builds fail (see comments 32 and 33). Something funny is going on with the html documentation build, meaning that the binary-indep rule can't link files into the documentation folder. I guess it's a directory-naming problem, but I haven't had chance to track it down.

[Julian Taylor (jtaylor)]

attached new debdiff vs 1.0.1-2 from unstable main
to build it in pbuilder you need to have /dev/shm bindmounted

[Julian Taylor (jtaylor)]

attached the buildlog for above debdiff

[Julian Taylor (jtaylor)]

ubuntu delta dropped:
40_UnicodeEncodeError.patch was applied upstream, included in 1.0.1
70_search_new_tkdir.patch was applied in the debian package 1.0.1-2

only the qt3-drop delta remaining

[Bhavani Shankar (bhavi)]

Any news as to whether this package can be merged?

Regards

[Ben Gamari (bgamari)]

I'm afraid no progress has been made on the documentation build issue. I haven't been able to reproduce the issue, so my efforts have been fruitless.

[Julian Taylor (jtaylor)]

the documentation issue is solved. Above debdiff builds in a oneiric chroot with /dev/shm mounted
There is also a patch for 0.99 here: https://github.com/matplotlib/matplotlib/pull/104

[Launchpad Janitor (janitor)]

This bug was fixed in the package matplotlib - 1.0.1-2ubuntu1

---------------
matplotlib (1.0.1-2ubuntu1) oneiric; urgency=low

  * Merge from debian unstable (LP: #607395). Remaining changes:
    - Drop python-qt-dev from build-depends and python-qt3 from
      python-matplotlib suggests to support python-qt3 removal. Reflect
      this in README.Debian.

matplotlib (1.0.1-2) unstable; urgency=low

  * Upload to unstable
    - fixed a FTBFS since the failing code (unicode vs byte-string) is rewritten
      in 1.0.x branch; thanks to Lucas Nussbaum for the report and to Stefano
      Rivera for the analysis; Closes: #625150
  * debian/rules
    - remove verbose logging for debhelper
    - remove compiled python code from documentation
  * debian/patches/70_search_new_tkdir.patch
    - added patch to look for the updated location of 'tkConfig.sh' script;
      thanks to Julian Taylor for the report and patch; Closes: #621700
  * debian/control
    - bump Standards-Version to 3.9.2 (no changes needed)
    - removed XB-P-V/Conflicts/Replaces/Provides, no more needed

matplotlib (1.0.1-1) experimental; urgency=low

  * New upstream release; Closes: #606116
  * debian/patches/{10_build_fix.patch, 20_matplotlibrc_path_search_fix.patch}
    - updated to new upstream code
  * debian/patches/30_doc_pass_dpi.patch
    - removed, merged upstream
  * debian/rules
    - don't compress objects.inv; thanks to Michael Fladischer for the report;
      Closes: #608760
  * debian/rules, debian/patches/30_disable_sample_downloads.patch
    - bypass examples data download from internet, but use local copy instead
  * debian/python-matplotlib-data.install
    - install examples sample data
  * debian/control
    - bump Standards-Version: to 3.9.1 (no changes needed)
    - removed DM-U-A flag, no more needed
    - added python-xlwt to b-d, needed to build doc examples
    - Benjamin Drung removed himself from the Uploaders list
  * debian/copyright
    - updated debian packaging copyright
  * debian/patches/40_bts608939_draw_markers_description.patch
    - fix a glitch (missing reference to documentation page) in draw_markers()
      description; thanks to Jakub Wilk for report and patch; Closes: #608939
  * debian/patches/50_bts608942_spaces_in_param_args.patch
    - don't separate param and its argument with a space, new sphinx consider
      the first argument as the type; thanks to Jakub Wilk for the report;
      Closes: #608942
  * debian/patches/60_doc_output_format.patch
    - obtain the documentation output format even if the value is a unicode (as
      returned by recent sphinx); thanks to Jakub Wilk for the tip on IRC
 -- Julian Taylor <email address hidden> Sun, 08 May 2011 07:25:16 +0100

[Ben Gamari (bgamari)]

Can we please merge this back into Natty?

[Julian Taylor (jtaylor)]

I have requested a backport for natty:
https://bugs.launchpad.net/natty-backports/+bug/798191