scipy.io.loadmat() excessively slow (regression)

[Emanuele Olivetti]

Binary package hint: python-scipy

Recent ubuntus (at least from 9.04 till lucid) provide scipy v0.7.0 (python-scipy) which has an important regression problem for the function scipy.io.loadmat(). The loadmat() imports data from Matlab format (".mat") files into NumPy arrays, a task which is very popular among scipy users. Note that there is no alternative to using this function if you need to work with Matlab files.

Scipy v0.7.0 introduced a severe regression problem in that function which became excessively slow, like requiring tens of minutes instead of ten seconds to load files. The result is that the loadmat() function is unusable in SciPy v0.7.0.

This bug has been tracked and solved by SciPy developers nearly one year ago with very simple patch. Details here, with a test case and thorough discussion:
http://thread.gmane.org/gmane.comp.python.scientific.devel/9934/focus=9974

The very simple patch is here:
http://article.gmane.org/gmane.comp.python.scientific.devel/9944
it modifies few lines of gzipstreams.py which lies here:
/usr/share/pyshared/scipy/io/matlab/gzipstreams.py
Note that within the discussion's thread there more than one patch was proposed, but the only correct one is the last proposed, which the above link refers to.

I am attaching the same patch here for your convenience.

Please apply it or upgrade python-scipy package to a more recent SciPy version.

Thanks.

[Rick McBride (rmcbride)]

I have a branch of the UDD branch for this waiting to be proposed when Maverick Meerkat opens up.

[malev (marcosvanetta)]

Thanks for helping ubuntu.
Hi, could you provide us a link to the commit in the original project?
thanks

[Rick McBride (rmcbride)]

My branch is lp:~rmcbride/+junk/python-scipy

I guess I needed to push a branch to a different path in order to propose a merge. I'll fix that as soon as I remind myself where it should have been pushed.

[Rick McBride (rmcbride)]

Sorry about that. I did the fix branch prior to Maverick opening up. Correct branch is at
lp:~rmcbride/ubuntu/maverick/python-scipy/fix-538774

I'm about to propose the merge.

[James Westby (james-w)]

Isn't this fixed in the 0.7.2 in Debian unstable? If so then the fix will be in maverick in a
few days.

Thanks,

James

[Daniel Holbach (dholbach)]

Can you find out if it's closed in Debian and Maverick and close it if it is?

[Rick McBride (rmcbride)]

checking on current debian and maverick status.

[Rick McBride (rmcbride)]

Maverick is Candidate: 0.7.0-2, where Lucid is Candidate: 0.7.0-2ubuntu1, which would seem to indicate removal of ubuntu-specific patches.

Debian stands at 0.7.2-1 0.

Do I need to do a sync request? or is this something that would already be in work?

[James Westby (james-w)]

This has now been synced.

Thanks,

James

[Gael Varoquaux (gael-varoquaux)]

Any chance that this fix gets backported in the 10.04 LTS? For people using matlab, this bugs makes scipy unusable in the LTS