scilab: save, open and ATOMS system is broken in Scilab 6 on bionic, cosmic, disco and eoan

Bug #1765503 reported by Norbert on 2018-04-19
28
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Scilab
Confirmed
High
hdf5 (Ubuntu)
Undecided
Unassigned
scilab (Debian)
New
Unknown
scilab (Ubuntu)
Undecided
Unassigned

Bug Description

Steps to reproduce:
1. Install Ubuntu 18.04 LTS (x86_64 in my case)
2. Install `scilab` package
3. Launch Scilab CLI (as normal `scilab` does not start because of bug 1762999) from terminal with `scilab-cli`
4. Try to show information of a package from ATOMS system with
`atomsShow('coselica')`

Expected results:
* Scilab updates local ATOMS cache, then shows information about Coselica
toolbox

Actual results:
* Scilab shows error in its console:

--> atomsShow('coselica')
Scanning repository http://atoms.scilab.org/6.0 ... Done

HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) thread 140526662825856:
  #000: ../../../src/H5G.c line 553 in H5Gget_info(): invalid argument
    major: Invalid arguments to routine
    minor: Bad value
HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) thread 140526662825856:
  #000: ../../../src/H5G.c line 301 in H5Gcreate2(): not a location
    major: Invalid arguments to routine
    minor: Inappropriate type
  #001: ../../../src/H5Gloc.c line 253 in H5G_loc(): invalid object ID
    major: Invalid arguments to routine
    minor: Bad value
HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) thread 140526662825856:
  #000: ../../../src/H5A.c line 265 in H5Acreate2(): not a location
    major: Invalid arguments to routine
    minor: Inappropriate type
  #001: ../../../src/H5Gloc.c line 253 in H5G_loc(): invalid object ID
    major: Invalid arguments to routine
    minor: Bad value
HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) thread 140526662825856:
  #000: ../../../src/H5D.c line 121 in H5Dcreate2(): not a location ID
    major: Invalid arguments to routine
    minor: Inappropriate type
  #001: ../../../src/H5Gloc.c line 253 in H5G_loc(): invalid object ID
    major: Invalid arguments to routine
    minor: Bad value
HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) thread 140526662825856:
  #000: ../../../src/H5F.c line 749 in H5Fclose(): not a file ID
    major: Invalid arguments to routine
    minor: Inappropriate type
failed to close fileat line 265 of function atomsDESCRIPTIONget ( /usr/share/scilab/modules/atoms/macros/atoms_internals/atomsDESCRIPTIONget.sci line 284 )
at line 31 of function atomsIsPackage ( /usr/share/scilab/modules/atoms/macros/atoms_internals/atomsIsPackage.sci line 47 )
at line 43 of function atomsShow ( /usr/share/scilab/modules/atoms/macros/atomsShow.sci line 59 )

atomsDESCRIPTIONget: save ('/home/beaver/.Scilab/scilab-6.0.1/.atoms/packages') has failed.

ProblemType: Bug
DistroRelease: Ubuntu 18.04
Package: scilab-cli 6.0.1-1ubuntu1
ProcVersionSignature: Ubuntu 4.15.0-17.18-generic 4.15.17
Uname: Linux 4.15.0-17-generic x86_64
ApportVersion: 2.20.9-0ubuntu5
Architecture: amd64
CurrentDesktop: MATE
Date: Thu Apr 19 22:08:58 2018
PackageArchitecture: all
SourcePackage: scilab
UpgradeStatus: No upgrade log present (probably fresh install)
---
ProblemType: Bug
ApportVersion: 2.20.9-0ubuntu7.6
Architecture: amd64
CurrentDesktop: MATE
DistroRelease: Ubuntu 18.04
InstallationDate: Installed on 2018-04-26 (354 days ago)
InstallationMedia: Ubuntu-MATE 18.04 LTS "Bionic Beaver" - Release amd64 (20180426)
Package: scilab 6.0.1-7ubuntu1~18.04
PackageArchitecture: all
ProcVersionSignature: Ubuntu 4.15.0-47.50-generic 4.15.18
Tags: bionic
Uname: Linux 4.15.0-47-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
_MarkForUpload: True
---
ProblemType: Bug
ApportVersion: 2.20.10-0ubuntu13.2
Architecture: amd64
CurrentDesktop: MATE
DistroRelease: Ubuntu 18.10
InstallationDate: Installed on 2019-02-17 (58 days ago)
InstallationMedia: Ubuntu-MATE 18.10 "Cosmic Cuttlefish" - Release amd64 (20181017.2)
Package: scilab 6.0.1-7ubuntu1~18.10
PackageArchitecture: all
ProcVersionSignature: Ubuntu 4.18.0-16.17-generic 4.18.20
Tags: cosmic
Uname: Linux 4.18.0-16-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
_MarkForUpload: True

BUG DESCRIPTION:
----------------
Not able to save or load a variable

ERROR LOG:
----------
HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) thread 140383204820864:
  #000: ../../../src/H5G.c line 553 in H5Gget_info(): invalid argument
    major: Invalid arguments to routine
    minor: Bad value
HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) thread 140383204820864:
  #000: ../../../src/H5D.c line 121 in H5Dcreate2(): not a location ID
    major: Invalid arguments to routine
    minor: Inappropriate type
  #001: ../../../src/H5Gloc.c line 253 in H5G_loc(): invalid object ID
    major: Invalid arguments to routine
    minor: Bad value
HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) thread 140383204820864:
  #000: ../../../src/H5F.c line 749 in H5Fclose(): not a file ID
    major: Invalid arguments to routine
    minor: Inappropriate type
failed to close file

HOW TO REPRODUCE THE BUG:
-------------------------
 - install a brand new debian testing
 - install scilab 6.0.0 package and dependencies
 - create any variable (e.g. "stack")
 - save it with:
         save("stack.sod","stack")

OTHER INFORMATION:
------------------
Scilab also fails to load the ATOMS manager, and it seems to be bound to that problem.

Norbert (nrbrtx) wrote :
Changed in scilab (Debian):
status: Unknown → New
Norbert (nrbrtx) wrote :

Bug exists in Ubuntu 18.04 LTS with all updates.

Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in scilab (Ubuntu):
status: New → Confirmed
arian (sainiravi1) wrote :

Hi,
I am also facing the same problem with exact symptoms as brought out above. Need urgent fix / alternative method to use atoms libraries.

Any help/ suggestions are welcome.

Regards.

Norbert (nrbrtx) wrote :

Bug still exists on Ubuntu 18.04 LTS with all updates.

18.10 cosmic is affected too.

summary: - scilab: ATOMS system is broken in Scilab 6 on bionic
+ scilab: ATOMS system is broken in Scilab 6 on bionic and cosmic
tags: added: cosmic

*** Bug 15738 has been marked as a duplicate of this bug. ***

I confirm that this bug is also present under Ubuntu 18.04 64bits.

Antoine

Antoine, could you please confirm here that this bug is now fixed on the 6.0-branch and on the master one, for this OS, as you did on users@?
Thanks

PhiRG,
Could you please try on Debian with the nightly build: http://www.scilab.org/en/development/nightly_builds/branch60
Thanks

This bug is resolved scilab-branch-6.0-1534494461 and scilab-master-1534776009, at least for Ubuntu 18.04 64bits.

Antoine

Hello Samuel,

Well done, it works with scilab-branch-6.0-1534494461 now.

The atoms manager opens too now.

Thank you very much for your work!

Philippe

Norbert (nrbrtx) wrote :

Still broken on Cosmic.

Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in hdf5 (Ubuntu):
status: New → Confirmed
Norbert (nrbrtx) on 2019-03-23
summary: - scilab: ATOMS system is broken in Scilab 6 on bionic and cosmic
+ scilab: ATOMS system is broken in Scilab 6 on bionic, cosmic and disco
tags: added: disco

There are plenty of users who install distribution packages believing they are "official" packages that still fail with the same problem. We cannot always say them to install the .tar.gz

@Stéphane: I don't understand your rationale for reopening this report.
Do you mean that there are some distributions packaging the official Scilab 6.0.2, with which the reported problem still occur?

Currently, no mainstream dist (Ubuntu, Fedora, ...) packaged Scilab 6.0.2 because the packaged 6.0.1 version does not work already. Just try the following pair of keywords on Google:

Scilab Ubuntu
Scilab Fedora
...

In fact, Scilab 6.0.2 and the master can be built w.r.t hdf5 1.10.n, but at runtime the problem of bug #15394 still occurs.
The only version that is supported in Scilab is hdf5 1.8. The problem is that in recent distributions, the matio lib is too recent and absolutely needs hdf5 1.10.n (I have this problem also in OSX when I use brew as a packaging system).

The problem is that package maintainers are not informed of this strict dependency :-(

(In reply to Stéphane MOTTELET from comment #9)
> Currently, no mainstream dist (Ubuntu, Fedora, ...) packaged Scilab 6.0.2 because the packaged 6.0.1 version does not work already. Just try the
> following pair of keywords on Google:
>
> Scilab Ubuntu
> Scilab Fedora
> ...

Google is not always up to date :) ; "6.0.1" was mostly reported as buggy due to its startup and search results are also cluttered by it. Scilab 6.0.2 is available as a Fedora package and atomsSystemUpdate works (I just checked) cause Fedora did not update to hdf5 1.10 yet.

Mixing packaging issues with upstream bugs seems a bad idea to me, it will make us responsible for packaging Scilab in systems that are free to update/patch/deprecates/etc... If a distribution package wished to upgrade to hdf5 1.10 for some reason, that's ok but when issues arise we could only try to help them fix things on their side and work together to find the root cause (is it hdf5, is it Scilab, is it some obscure compilation mismatch)

> In fact, Scilab 6.0.2 and the master can be built w.r.t hdf5 1.10.n, but at runtime the problem of bug #15394 still occurs.
> The only version that is supported in Scilab is hdf5 1.8. The problem is that in recent distributions, the matio lib is too recent and absolutely
> needs hdf5 1.10.n (I have this problem also in OSX when I use brew as a packaging system).

And that's the whole point, hdf5 1.10 seems to have an issue at runtime and matio started to require it. I clarified the summary and let's work on that :)

apport information

tags: added: apport-collected
description: updated

apport information

apport information

Still a problem on bionic even with Scilab 6.0.1-7ubuntu1~18.04 .

apport information

description: updated

apport information

apport information

Still a problem on cosmic even with Scilab 6.0.1-7ubuntu1~18.10 .

Norbert (nrbrtx) on 2019-04-21
summary: - scilab: ATOMS system is broken in Scilab 6 on bionic, cosmic and disco
+ scilab: ATOMS system is broken in Scilab 6 on bionic, cosmic, disco and
+ eoan
tags: added: eoan
Norbert (nrbrtx) wrote :

ATOMS is still broken on Ubuntu 18.04 Bionic Beaver (LONG TERM SUPPORT release)..

Norbert (nrbrtx) wrote :

Even with latest upgrades on eoan, disco, cosmic and bionic - ATOMS does not work.

Please fix it!

Norbert (nrbrtx) on 2019-08-02
summary: - scilab: ATOMS system is broken in Scilab 6 on bionic, cosmic, disco and
- eoan
+ scilab: save, open and ATOMS system is broken in Scilab 6 on bionic,
+ cosmic, disco and eoan
Norbert (nrbrtx) wrote :

Dear Ubuntu, Debian and Scilab developers/maintainers!

The problem is really caused by HDF5.
I have just tested binary 6.0.2 build from scilab.org and compared it to scilab-6.0.1 as it came with 18.04 LTS.

The `test_run('hdf5')` fails with deb-package in 67% () of tests.

Moreover the `save('data.dat')` is also fails:

```
$ scilab-cli -e "x=10; save('x.dat','x');"
Scilab 6.0.1 (May 21 2019, 20:36:16)
HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) thread 140183400724352:
  #000: ../../../src/H5G.c line 553 in H5Gget_info(): invalid argument
    major: Invalid arguments to routine
    minor: Bad value
HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) thread 140183400724352:
  #000: ../../../src/H5D.c line 121 in H5Dcreate2(): not a location ID
    major: Invalid arguments to routine
    minor: Inappropriate type
  #001: ../../../src/H5Gloc.c line 253 in H5G_loc(): invalid object ID
    major: Invalid arguments to routine
    minor: Bad value
HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) thread 140183400724352:
  #000: ../../../src/H5F.c line 749 in H5Fclose(): not a file ID
    major: Invalid arguments to routine
    minor: Inappropriate type
failed to close file
save: Unable to export variable 'x' in file 'x.dat'.

-->
```

And `open('data.dat')` has malfunction too. It does not work with correctly prepared file with variables.

Please take necessary measures to fix the issue. I'm ready to help with testing.

Norbert (nrbrtx) wrote :

Just a small note: in the binary scilab distribution exists interesting file lib/thirdparty/libhdf5.settings and it states that it uses version 1.8:

```
General Information:
-------------------
     HDF5 Version: 1.8.14
...
```

so Scilab should be patched more to support HDF5 1.10 .

Download full text (13.6 KiB)

# Intro

The text below is not related to official binary builds of Scilab (6.0.2 and latest from https://build.scilab.org/view/Scilab%20master/job/scilab-master-linux-64/ ).
They are not affected by the problem as they continue to use HDF 1.8.14 (see `lib/thirdparty/libhdf5.settings` for confirmation).

# Problem

The rpm- and deb- packages in GNU/Linux distributions have broken `save` and `load` functionality.

Currently many Linux distributions switched to HDF 1.10.x branch:

| Distibution | Scilab | HDF5 | `save`/ `load` |
| ------------------- | ------ | ------ | ------------------------ |
| Debian 10 Buster | 6.0.1 | 1.10.4 | !!! bug !!! |
| Ubuntu 18.04 LTS | 6.0.1 | 1.10.0 | !!! bug !!! (LP#1765503) |
| Ubuntu 19.04 | 6.0.2 | 1.10.4 | !!! bug !!! (LP#1765503) |
| Fedora RawHide | 6.0.2 | 1.10.5 | !!! bug !!! |
| Mageia 7 | 6.0.2 | 1.10.5 | !!! bug !!! |
| OpenSuSe Leap 15.1 | 6.0.2 | 1.10.4 | !!! bug !!! |
| OpenSuSe Tumbleweed | 6.0.2 | 1.10.4 | !!! bug !!! |

You can see Scilab packaged version in distributions on Repology - https://repology.org/project/scilab/packages .

Below is a text output of `test_run('hdf5')` from Ubuntu 18.04.2 LTS without Details section as example:

```
--> test_run('hdf5')
   TMPDIR = /tmp/SCI_TMP_213_5dIgF3

 Module 001/001 - [hdf5] :

   001/046 - [hdf5] binary_bit-for-bit..........................failed: Slave Scilab exited with error code 1
   002/046 - [hdf5] h5attr......................................failed: Slave Scilab exited with error code 1
   003/046 - [hdf5] h5close.....................................failed: Slave Scilab exited with error code 1
   004/046 - [hdf5] h5cp........................................failed: error_output not empty
                                                                 Use 'no_check_error_output' option to disable this check.
   005/046 - [hdf5] h5dataset...................................failed: error_output not empty
                                                                 Use 'no_check_error_output' option to disable this check.
   006/046 - [hdf5] h5exists....................................failed: Slave Scilab exited with error code 1
   007/046 - [hdf5] h5flush.....................................failed: error_output not empty
                                                                 Use 'no_check_error_output' option to disable this check.
   008/046 - [hdf5] h5group.....................................failed: error_output not empty
                                                                 Use 'no_check_error_output' option to disable this check.
   009/046 - [hdf5] h5isAttr....................................failed: Slave Scilab exited with error code 1
   010/046 - [hdf5] h5isFile....................................failed: Slave Scilab exited with error code 1
   011/046 - [hdf5] h5isGroup...................................failed: Slave Scilab exited with error code 1
   012/046 - [hdf5] h5isRef.....................................failed: Slave Scilab exited with error code 1
   013/046...

Download full text (13.1 KiB)

Created attachment 4992
Patches for Ubuntu (18.04 LTS and 19.04) and master branch

# Intro

I have analyzed the source code of the `modules/hdf5` folder.
I have used KDevelop IDE for doing so.
It has full feature *Semantic code* analyzer ( https://www.kdevelop.org/screenshots/c-semantic-information-tooltip ).
Without KDevelop you can detect problems with datatypes after adding `-Wconversion`
to the CFGLAGS and CXXFLAGS variables inside `modules/hdf5/Makefile`.

Usually the warning messages from GCC look like:

```
src/c/h5_writeDataToFile.c: In function 'writeStringMatrix':
src/c/h5_writeDataToFile.c:296:27: warning: conversion to 'int' from 'hid_t {aka long int}' may alter its value [-Wconversion]
     status = addAttribute(dset, g_SCILAB_CLASS, g_SCILAB_CLASS_STRING);
                           ^~~~
...
src/c/h5_fileManagement.c: In function 'openHDF5File':
src/c/h5_fileManagement.c:161:12: warning: conversion to 'int' from 'hid_t {aka long int}' may alter its value [-Wconversion]
     return file;
            ^~~~
...
src/c/h5_fileManagement.c: In function 'createHDF5File':
src/c/h5_fileManagement.c:101:12: warning: conversion to 'int' from 'hid_t {aka long int}' may alter its value [-Wconversion]
     return file;
            ^~~~
sci_gateway/cpp/sci_hdf5_listvar_v3.cpp: In function 'bool read_struct(int, VarInfo6&)':
sci_gateway/cpp/sci_hdf5_listvar_v3.cpp:738:42: warning: conversion to 'int' from 'hid_t {aka long int}' may alter its value [-Wconversion]
                 int data = H5Rdereference(refs,
```

and so on.

# Patching

I attach 3 patches to fix HDF-related problems in Ubuntu (18.04 LTS and 19.04) and in master branch.

## Ubuntu 18.04 LTS

Applying patch is as follows:

```
sudo apt-get build-dep scilab
cd ~/Downloads
apt-get source scilab
cd scilab-6.0.1
dpkg-builpackage -b -j$(nproc) # build to see current state
cd modules/hdf5
patch -p1 < ~/Downloads/scilab-6.0.1-ubuntu-bionic.patch
cd ../..
dpkg-builpackage -b -j$(nproc) # build to see changes
```

As the patch is applied then we can test patched Scilab.
The results from `test_run('hdf5')` are the following:

```
-->test_run('hdf5')
   TMPDIR = /tmp/SCI_TMP_2556_AWXxWF

 Module 001/001 - [hdf5] :

   001/046 - [hdf5] binary_bit-for-bit..........................passed
   002/046 - [hdf5] h5attr......................................passed
   003/046 - [hdf5] h5close.....................................passed
   004/046 - [hdf5] h5cp........................................passed
   005/046 - [hdf5] h5dataset...................................passed
   006/046 - [hdf5] h5exists....................................passed
   007/046 - [hdf5] h5flush.....................................passed
   008/046 - [hdf5] h5group.....................................passed
   009/046 - [hdf5] h5isAttr....................................passed
   010/046 - [hdf5] h5isFile....................................passed
   011/046 - [hdf5] h5isGroup...................................passed
   012/046 - [hdf5] h5isRef.....................................passed
   013/046 - [hdf5] h5isSet.....................................passed
   014/046 - [hdf5] h5isSpace...................................pa...

Changed in scilab:
importance: Unknown → High
status: Unknown → Confirmed
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.