-json output not valid when -class filter used

Bug #1405873 reported by HuckleBerry on 2014-12-26
42
This bug affects 8 people
Affects Status Importance Assigned to Milestone
lshw (Ubuntu)
Undecided
Unassigned

Bug Description

When lshw is invoked with the -json option, a single JSON object describing the entire system is produced. Hardware components are nested within the JSON object structure. When the -class filter option is used with -json, one would expect an array of JSON objects, each representing one of the matching hardware components. Instead, the output appears to simply exclude the non-relevant parts of the original JSON object, keeping the parts of the structure that match the filter. The result of which is not a valid JSON object, which is sometimes missing required commas and is always missing the enclosing array brackets.

Examples from two machines with varying hardware layouts are attached (with additional -sanitze option used). Machine 1 with the 'disk' class is the most incorrect.

Machine 1
sudo lshw -version = B.02.16
sudo dpkg -s lshw | grep Version = Version: 02.16-2
lsb_release -rd = Ubuntu 13.10
sudo lshw | grep '*-'
  *-core
     *-firmware
     *-memory:0 UNCLAIMED
        *-bank UNCLAIMED
     *-memory:1
        *-bank
     *-cache:0
     *-cache:1
     *-cache:2
     *-cpu
     *-memory:2 UNCLAIMED
     *-memory:3 UNCLAIMED
     *-pci
        *-pci:0
        *-display
        *-multimedia:0
        *-usb:0
        *-communication
        *-network
        *-usb:1
        *-multimedia:1
        *-pci:1
        *-pci:2
           *-network
        *-pci:3
           *-usb
        *-usb:2
        *-isa
        *-storage
        *-serial UNCLAIMED
     *-scsi:0
        *-disk
     *-scsi:1
        *-disk
           *-volume:0
           *-volume:1
           *-volume:2
     *-scsi:2
        *-disk
           *-volume:0
           *-volume:1
           *-volume:2
     *-scsi:3
        *-disk
     *-scsi:4
        *-disk
     *-scsi:5
        *-disk
           *-volume:0
           *-volume:1
  *-power UNCLAIMED

Machine 2
sudo lshw -version = B.02.16
sudo dpkg -s lshw | grep Version = Version: 02.16-2ubuntu1.2
lsb_release -rd = Ubuntu 14.04.1 LTS
 sudo lshw | grep '*-'
  *-core
     *-firmware:0
     *-cpu:0
     *-memory:0
        *-bank:0
        *-bank:1
     *-firmware:1
     *-cpu:1
     *-memory:1
     *-memory:2 UNCLAIMED
     *-memory:3 UNCLAIMED
     *-pci
        *-isa
        *-ide
        *-bridge UNCLAIMED
        *-display UNCLAIMED
        *-generic
  *-network

Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in lshw (Ubuntu):
status: New → Confirmed

Seeing the same bugs (Missing array brackets and missing commas in between objects)

lsb_release -rd
Description: Ubuntu 14.04.2 LTS
Release: 14.04

lshw -version
B.02.16

lshw -json -C storage -C drive

Richard Wall (richardw) wrote :

I'm going to give up on the `lshw -class -json` combination.
 * https://github.com/ClusterHQ/flocker/pull/2769

Murat Uenalan (muenalan) wrote :

Can confirm, still not fixed in B.02.17 (Xenial). Example:

$ sudo lshw -json -c display

  {
[snip .. 3 graphics cards]
  },

which is not JSON. I think the listing of multiple devices it the unsupported case. Should have been

[
  {
[snip .. 3 graphics cards]
  }
]

which has outer brackets, and no trailing comma.

Bryan Seitz (seitz-a) wrote :

hmm is this still broken in the latest git?

Bryan Seitz (seitz-a) wrote :

Confirmed json output is still broken in various ways.

Jeff Silverman (jeffsilverm) wrote :

Still broken:

$ sudo lshw -json -C network | jq .
{
  "id": "network",
  "class": "network",
...
    "100bt-fd": "100Mbit/s (full duplex)",
    "1000bt-fd": "1Gbit/s (full duplex)",
    "autonegotiation": "Auto-negotiation"
  }
}
parse error: Expected value before ',' at line 48, column 4
$

Note that there is no leading [

$ sudo lshw -json -C network | head -5
                                                                                                                                              {
    "id"; : "network",
    "class"; : "network",
    "claimed" : true,
    "handle" : "PCI:0000:00:19.0",

But there is a comma at line 48. Why is jq failing there? Because there is no [. If I insert one, then the problem disappears.

$ sudo lshw -json -C network | head -52 | tail -6
    }
  },
                                                            {
    "id"; : "network",
    "class" : "network",
    "claimed" : true,

Note that there is no ] at the end of the file, which also causes jq to fail.

$ sudo lshw -json -C network | tail -6
    },
    "capabilities" : {
    "ethernet" : true,
    "physical" : "Physical interface"
    }
  }
$

Vern Hart (vern) wrote :

I can confirm, this is still broken.

In my output, I'm missing the leading and trailing square bracket. Also, between some stanzas there are close and open curly braces (on the same line) without a comma in between.

Since those are the only two syntactical errors in the output I'm working with, this simple one-liner works around it:

(echo "["; lshw -json -C network | sed 's/} .*{/}, {/'; echo "]") | jq .

this has actually got worse, as it now (as of 19.10 / 02.18.85-0.3 at least) breaks when no class is used
e.g.

$ lshw -json -quiet | python3 -m json.tool
Expecting ',' delimiter: line 26, column 8 (char 785)

lshw -xml | xmllint -

seems to indicate xml output works ok, but the format is annoying :(

https://github.com/lyonel/lshw/tree/B.02.18 outputs valid json with

lshw -json -quiet

though it does have the problem described in https://bugs.launchpad.net/ubuntu/+source/lshw/+bug/1593233 i.e. can only run as root, and crashes otherwise.

Using git bisect, the change that broke output was https://github.com/lyonel/lshw/pull/28
See also https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=929206

One can see the full extent of the brokenness of the command (in all versions I've seen) using

#!/usr/bin/env python3

from itertools import combinations, chain, repeat
from subprocess import check_output as sco
import json

cmd = ['lshw', '-json', '-quiet']

classes = 'system bus memory processor bridge display generic multimedia storage network'.split()

if __name__ == '__main__':
    for flags in chain(*(combinations(classes, n) for n in range(len(classes)))):
        flags = chain(*zip(repeat('-c'), flags))
        flags = list(flags)
        if True:
            try:
                json.loads(sco(cmd + flags))
                print(" ".join(flags), "was OK")
            except json.decoder.JSONDecodeError as e:
                print(' '.join(flags))
                print(e)

Nilson Lopes (noslin005) wrote :

There are no fixes yet for Ubuntu 19.10 (Eoan Ermine).

sudo lshw -json -quiet | python3 -m json.tool
Expecting ',' delimiter: line 26 column 8 (char 792)

sudo dpkg -s lshw | grep Version
Version: 02.18.85-0.3

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.