doctests fail with numpy 1.14

[Dan Watkins]

======================================================================
FAIL: saveDataTxt (yade.plot)
Doctest: yade.plot.saveDataTxt
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/doctest.py", line 2226, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for yade.plot.saveDataTxt
  File "/usr/lib/x86_64-linux-gnu/yade/py/yade/plot.py", line 614, in saveDataTxt

----------------------------------------------------------------------
File "/usr/lib/x86_64-linux-gnu/yade/py/yade/plot.py", line 626, in yade.plot.saveDataTxt
Failed example:
    d=numpy.genfromtxt('/tmp/dataFile.txt.bz2',dtype=None,names=True)
Exception raised:
    Traceback (most recent call last):
      File "/usr/lib/python2.7/doctest.py", line 1315, in __run
        compileflags, 1) in test.globs
      File "<doctest yade.plot.saveDataTxt[8]>", line 1, in <module>
        d=numpy.genfromtxt('/tmp/dataFile.txt.bz2',dtype=None,names=True)
      File "/usr/lib/python2.7/dist-packages/numpy/lib/npyio.py", line 1689, in genfromtxt
        fhd = iter(np.lib._datasource.open(fname, 'rt', encoding=encoding))
      File "/usr/lib/python2.7/dist-packages/numpy/lib/_datasource.py", line 260, in open
        return ds.open(path, mode, encoding=encoding, newline=newline)
      File "/usr/lib/python2.7/dist-packages/numpy/lib/_datasource.py", line 614, in open
        encoding=encoding, newline=newline)
      File "/usr/lib/python2.7/dist-packages/numpy/lib/_datasource.py", line 88, in _python2_bz2open
        raise ValueError("bz2 text files not supported in python2")
    ValueError: bz2 text files not supported in python2
----------------------------------------------------------------------
File "/usr/lib/x86_64-linux-gnu/yade/py/yade/plot.py", line 627, in yade.plot.saveDataTxt
Failed example:
    d['a']
Exception raised:
    Traceback (most recent call last):
      File "/usr/lib/python2.7/doctest.py", line 1315, in __run
        compileflags, 1) in test.globs
      File "<doctest yade.plot.saveDataTxt[9]>", line 1, in <module>
        d['a']
    NameError: name 'd' is not defined
----------------------------------------------------------------------
File "/usr/lib/x86_64-linux-gnu/yade/py/yade/plot.py", line 629, in yade.plot.saveDataTxt
Failed example:
    d['b']
Exception raised:
    Traceback (most recent call last):
      File "/usr/lib/python2.7/doctest.py", line 1315, in __run
        compileflags, 1) in test.globs
      File "<doctest yade.plot.saveDataTxt[10]>", line 1, in <module>
        d['b']
    NameError: name 'd' is not defined

----------------------------------------------------------------------
Ran 58 tests in 1.063s

[Dan Watkins (oddbloke)]

This was found in Ubuntu autopkgtests (at https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-cosmic/cosmic/amd64/y/yade/20180529_175759_38e3f@/log.gz)

[Bruno Chareyre (bruno-chareyre)]

Thanks for reporting.
The strange part is that numpy 1.14 is still supporting bz2 according to the documentation.
Could someone confirm "ValueError: bz2 text files not supported in python2" on his system with the following code:

from yade import plot
plot.reset()
plot.addData(a=1,b=11,c=21,d=31) # add some data here
plot.saveDataTxt('/tmp/dataFile.txt.bz2',vars=('a','b','c'))
import numpy
d=numpy.genfromtxt('/tmp/dataFile.txt.bz2',dtype=None,names=True)

Bruno

[Jérôme Duriez (jduriez)]

I do not confirm it:

jerome.duriez@AXP17003:~$ yadedaily
Welcome to Yade 2018.02b-1270674828~xenial
TCP python prompt on localhost:9000, auth cookie `dckayu'
XMLRPC info provider on http://localhost:21000
[[ ^L clears screen, ^U kills line. F12 controller, F11 3d view (use h-key for showing help), F10 both, F9 generator, F8 plot. ]]

Yade [1]: from yade import plot

Yade [2]: plot.reset()

Yade [3]: plot.addData(a=1,b=11,c=21,d=31) # add some data here

Yade [4]: plot.saveDataTxt('/tmp/dataFile.txt.bz2',vars=('a','b','c'))

Yade [5]: import numpy

Yade [6]: d=numpy.genfromtxt('/tmp/dataFile.txt.bz2',dtype=None,names=True)

Yade [7]: d
 -> [7]:
array((1, 11, 21),
      dtype=[('a', '<i8'), ('b', '<i8'), ('c', '<i8')])

I have Python 2.7.12 on my machine (in case it matters)

[Bruno Chareyre (bruno-chareyre)]

@Jérôme
And you have numpy 1.14?

[Jérôme Duriez (jduriez)]

I have numpy 1.11.0 actually

[Anton Gladky (gladky-anton)]

Workaround is to use tar.gz instead of bz2.

[Bruno Chareyre (bruno-chareyre)]

Bug Watch released a fix?... how?
B

[Anton Gladky (gladky-anton)]

I workarounded it on Weekend [1]. I wanted to test it before the commit into the Yade's trunk. There are really problems with the new numpy 1.14, python 2 and bz2. tar.gz works.

Also there were some problems with the new sphinx [2]. It should also be backported into the Yade's trunk.

[1] https://salsa.debian.org/science-team/yade/blob/master/debian/patches/10_fix_bz2_pyhon2_problem.patch

[2] https://salsa.debian.org/science-team/yade/blob/master/debian/patches/12_fix_doc_generation.patch

[Janek Kozicki (cosurgi)]

moved from https://bugs.launchpad.net/yade/+bug/1774065 to https://gitlab.com/yade-dev/trunk/issues/41