lshw -json output invalid

[Rod Smith]

Beginning with Ubuntu 19.10 (Eoan) pre-release versions, the "lshw -json" output appears to be invalid, and can't be parsed with either Python's json library or the jq stand-alone command:

$ sudo lshw -json | jq .
parse error: Expected separator between values at line 25, column 8

The Python json library reports "Expecting ',' delimiter: line 25 column 8 (char 718)".

The first offending snippet looks like this:

  "capabilities" : {
    "smbios-2.7" : "SMBIOS version 2.7",
    "dmi-2.7" : "DMI version 2.7",
    "smp" : "Symmetric Multi-Processing",
    "vsyscall32" : "32-bit processes"
  } {
      "id" : "core",
      "class" : "bus",

Note the "} {" line. In pre-Eoan versions of Ubuntu, the equivalent snippet looks like this:

  "capabilities" : {
    "smbios-2.7" : "SMBIOS version 2.7",
    "dmi-2.7" : "DMI version 2.7",
    "vsyscall32" : "32-bit processes"
  },
  "children" : [
    {
      "id" : "core",
      "class" : "bus",

It looks like the new version is dropping the ', "children" : [' part.

On the offending system:

$ lsb_release -rd
Description: Ubuntu Eoan Ermine (development branch)
Release: 19.10

$ apt-cache policy lshw
lshw:
  Installed: 02.18.85-0.3
  Candidate: 02.18.85-0.3
  Version table:
 *** 02.18.85-0.3 500
        500 http://us.archive.ubuntu.com/ubuntu eoan/main amd64 Packages
        100 /var/lib/dpkg/status

[Jeff Lane  (bladernr)]

https://ezix.org/project/wiki the source is there, however, there are several JSON related bugs, some of which seem to be the same as this, but have been open for several years without update.

We may need to fix this ourselves.

[Yuan-Chen Cheng (ycheng-twn)]

command that can do testing: lshw -json 2>/dev/null | python -m json.tool

[Yuan-Chen Cheng (ycheng-twn)]

per my test, this fix this bug: https://github.com/lyonel/lshw/pull/48

[Yuan-Chen Cheng (ycheng-twn)]

ppa for testing: https://launchpad.net/~ycheng-twn/+archive/ubuntu/lshw
TODO: change log need refine (might need to refine in patch file)

[Rod Smith (rodsmith)]

I've tested the PPA in the application that originally prompted my bug report, and it's working fine. Will this be merged into Ubuntu's version of lshw before 20.04 is released?

[Yuan-Chen Cheng (ycheng-twn)]

fix in #4 is not perfect, since it still will create bad json for command like

lshw -json -c bridge

need to update the patch.

[Yuan-Chen Cheng (ycheng-twn)]

https://launchpad.net/~ycheng-twn/+archive/ubuntu/lshw-3 have the patch to fix the issue.

Please kindly test it and report your test status.

[Jeff Lane  (bladernr)]

Hi, so what are the chances that this will be fixed in 20.04?

[Andrew Cloke (andrew-cloke)]

(To be explicit) note that the Server Certification tests are dependant on the lshw json output, so this issue currently breaks the Server Certification test suite.

[Yuan-Chen Cheng (ycheng-twn)]

My plan was to have the patch in #7 merged in upstream (https://ezix.org/src/pkg/lshw/pulls/28), and then land the patch in Debian/Ubuntu.

However, due to my workload is in peak time, maybe I won't manage to go that path.

I believe we can land the patch in #7 to ubuntu. Feel free to do so.

[Launchpad Janitor (janitor)]

This bug was fixed in the package lshw - 02.18.85-0.3ubuntu1

---------------
lshw (02.18.85-0.3ubuntu1) focal; urgency=medium

  * Cherrypick upstream pull-request/patch to fix json output. LP:
    #1843063

 -- Dimitri John Ledkov <email address hidden> Thu, 30 Jan 2020 16:24:24 +0000

[Yuan-Chen Cheng (ycheng-twn)]

test passed on focal with version 02.18.85-0.3ubuntu1 for me.

[Yuan-Chen Cheng (ycheng-twn)]

Guess it will be trivial to backport to eoan. Shall we?