Lshw incorrectly detects amount of RAM in system

[Douglas]

When commissioning our new servers, we are only seeing 50% of the memory show up in the MaasUI. Looking at lshw, it looks like only 12 of the 24 DIMMs are listed (we see 384GB but we have 768GB). Everything else seems to be fine, number of cores, disks, etc... When running free on the machine, we get the expected amount of RAM (768GB)

Current:
384GB

Expecting
768GB

root@maas:~# dpkg -l '*maas*'
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-=======================-================-================-===================================================
ii maas 2.3.0-6434-gd354 all "Metal as a Service" is a physical cloud and IPAM
ii maas-cli 2.3.0-6434-gd354 all MAAS client and command-line interface
un maas-cluster-controller <none> <none> (no description available)
ii maas-common 2.3.0-6434-gd354 all MAAS server common files
ii maas-dhcp 2.3.0-6434-gd354 all MAAS DHCP server
ii maas-dns 2.3.0-6434-gd354 all MAAS DNS server
ii maas-proxy 2.3.0-6434-gd354 all MAAS Caching Proxy
ii maas-rack-controller 2.3.0-6434-gd354 all Rack Controller for MAAS
ii maas-region-api 2.3.0-6434-gd354 all Region controller API service for MAAS
ii maas-region-controller 2.3.0-6434-gd354 all Region Controller for MAAS
un maas-region-controller- <none> <none> (no description available)
un python-django-maas <none> <none> (no description available)
un python-maas-client <none> <none> (no description available)
un python-maas-provisionin <none> <none> (no description available)
ii python3-django-maas 2.3.0-6434-gd354 all MAAS server Django web framework (Python 3)
ii python3-maas-client 2.3.0-6434-gd354 all MAAS python API client (Python 3)
ii python3-maas-provisioni 2.3.0-6434-gd354 all MAAS server provisioning libraries (Python 3)

[Andres Rodriguez (andreserl)]

So this is definitely and issue in LSHW. MAAS directly grabs the information from there.

That said, I think it would be nice for MAAS to have an alternate method of gathering the memory in case lshw is completely wrong as is this case.

[Andres Rodriguez (andreserl)]

Actually, I think I may have spoken too soon. Lshw shows two banks of memory:

 lshw:node:
            id: bank:1
            claimed: true
            class: memory
            handle: DMI:001E
            - lshw:description:
              DIMM Synchronous 2400 MHz (0.4 ns)
            - lshw:product:
              72ASS8G72LZ-2G3B2
            - lshw:vendor:
              Micron
            - lshw:physid:
              1
            - lshw:serial:
              191285AE
            - lshw:slot:
              P1-DIMME1
            - lshw:size:
              units: bytes
              34358689792
            - lshw:width:
              units: bits
              64
            - lshw:clock:
              units: Hz
              2400000000
          - lshw:node:
            id: bank:2
            claimed: true
            class: memory
            handle: DMI:0020
            - lshw:description:
              DIMM Synchronous 2400 MHz (0.4 ns)
            - lshw:product:
              72ASS8G72LZ-2G3B2
            - lshw:vendor:
              Micron
            - lshw:physid:
              2
            - lshw:serial:
              191282FF
            - lshw:slot:
              P1-DIMMF1
            - lshw:size:
              units: bytes
              34358689792

[Lee Trager (ltrager)]

Could you please post the lshw output in XML form to allow us to create a unit test?

You can download this from the commissioning tab or using the CLI with
maas <profile> node-script-result download <system_id> current-commissioning filters=00-maas-01-lshw > lshw.log

[Lee Trager (ltrager)]

lshw shows two banks of memory but MAAS is still correctly calculating the output from lshw. There are twelve occurrences of the line <size units="bytes">34358689792</size> which is 384GB. I looked up the part number reported by lshw, 72ASS8G72LZ-2G3B2. According to [1] each DIM should be 64G, not 32G. There is a commit upstream [2] which adds support for DDR4 RAM and another [3] which adds support for larger amounts of RAM found in DDR4 RAM. As [1] says this module is DDR4 I suspect the version of lshw in Ubuntu needs both these patches.

@dgrosvenor could you try building lshw in rescue mode to confirm?

sudo apt install git gcc make
git clone https://github.com/lyonel/lshw.git
make
sudo ./lshw -xml > lshw.log

[1] https://www.micron.com/parts/modules/ddr4-sdram/mta72ass8g72lz-2g3
[2] https://github.com/lyonel/lshw/commit/0ea2e95b3ccdeb3834bec6fe22d0794a65e8c37a
[3] https://github.com/lyonel/lshw/commit/ab43c697ebaff3e5138c43a39a17f69859cde87d

[Lee Trager (ltrager)]

It look like these patches have already been accepted into the Bionic version of lshw.

@dgrosvenor you don't have to compile lshw. You could also deploy Bionic, install lshw, and run sudo lshw -xml > lshw.log to comfirm.

[Douglas (dgrosvenor)]

I tried deploying Bionic out through MaaS but it doesn't seem to be working. Looking at the console of the system, I'm getting a "failed posting event: start: init-local/check-cache: attempting to read from cache [trust]". Screenshot is attached as well (let me know if you want me to open up a separate ticket for this). Since I can't deploy out Bionic, I'll try compiling lshw from scratch.

[Andres Rodriguez (andreserl)]

Are you using a mirror ? That could be due to a mirror not up to date with
the Ubuntu images but I wouldn’t be sure without a full log.

On Fri, Mar 2, 2018 at 4:01 PM Douglas <email address hidden> wrote:

> I tried deploying Bionic out through MaaS but it doesn't seem to be
> working. Looking at the console of the system, I'm getting a "failed
> posting event: start: init-local/check-cache: attempting to read from
> cache [trust]". Screenshot is attached as well (let me know if you want
> me to open up a separate ticket for this). Since I can't deploy out
> Bionic, I'll try compiling lshw from scratch.
>
> ** Attachment added: "maas18.04deploy.png"
>
> https://bugs.launchpad.net/maas/+bug/1752440/+attachment/5067453/+files/maas18.04deploy.png
>
> --
> You received this bug notification because you are subscribed to MAAS.
> https://bugs.launchpad.net/bugs/1752440
>
> Title:
> Lshw incorrectly detects amount of RAM in system
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/maas/+bug/1752440/+subscriptions
>
> Launchpad-Notification-Type: bug
> Launchpad-Bug: product=maas; milestone=2.4.x; status=Triaged;
> importance=Medium; assignee=None;
> Launchpad-Bug: distribution=ubuntu; sourcepackage=lshw; component=main;
> status=Incomplete; importance=Undecided; assignee=None;
> Launchpad-Bug-Information-Type: Public
> Launchpad-Bug-Private: no
> Launchpad-Bug-Security-Vulnerability: no
> Launchpad-Bug-Commenters: andreserl dgrosvenor ltrager
> Launchpad-Bug-Reporter: Douglas (dgrosvenor)
> Launchpad-Bug-Modifier: Douglas (dgrosvenor)
> Launchpad-Message-Rationale: Subscriber (MAAS)
> Launchpad-Message-For: andreserl
>
--
Andres Rodriguez (RoAkSoAx)
Ubuntu Server Developer
MSc. Telecom & Networking
Systems Engineer

[Douglas (dgrosvenor)]

@andreserl I'm not sure if I'm using a mirror or not, I'm just using the built in maas.io images. Where would I see that/what log are you looking for? Once I get Bionic installed or the latest lshw compiled, I'll get you all the logs. Thanks!

[Douglas (dgrosvenor)]

Attached is the latest lshw.log from lshw version B.02.18.76. Looks like its getting the right DIMM size (64GB each).