ipmi-locate fails to detect BMCs described by ACPI

Bug #1784926 reported by dann frazier
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
freeipmi (Ubuntu)
Fix Released
High
dann frazier
Xenial
Fix Released
High
dann frazier
Bionic
Fix Released
High
dann frazier

Bug Description

[Impact]
ipmi-locate will not detect and report BMCs in systems where the firmware describes them only in an ACPI SPMI table. I surveyed several IPMI-capable systems, and all but 1 describe their BMCs in SMBIOS, with a subset of those also describing their BMC in ACPI as well. ipmi-locate works fine on those systems because it successfully parses the SMBIOS (dmidecode) info. But, one system - the HiSilicon D06 - only describes the BMC in ACPI. For this system, ipmi-locate fails to find a BMC.

One way this surfaces to the user is with MAAS. MAAS uses ipmi-locate to look for a BMC to determine which version of the IPMI protocol it supports. The way the query works is that it checks for >= 2.0 support and, if that fails, it assumes 1.5 support. Since ipmi-locate fails to find *any* support, the fallback to 1.5 is triggered and MAAS tries to talk to the system w/ IPMI LAN 1.5 protocol. This system actually supports IPMI LAN 2.0 protocol, which is incompatible with 1.5 LAN protocol, so MAAS is unable to power control the system. Even if you manually enlist the node and tell it the node is IPMI 2.0, commissioning will attempt to re-detect the BMC and reset it to 1.5.

[Test Case]
On a system where /sys/firmware/acpi/tables/SPMI* exists:
$ sudo ipmi-locate

At least one of the "[Probing KCS|SMIC|BT|SSIF] device using ACPI" tests should not report "FAILED".

[Fix]
The following patch series from upstream git fixes the problem:

40ba578f8 Correct order of bytes in specification_revision field of ACPI SPMI table
d92888128 Allow sysfs SPMI parsing on ARM platforms
158a901d7 Add support for parsing SPMI tables exposed via sysfs
184c74649 Split RSDT/XSDT parsing into new function
3c6fa7054 Don't try to separate the header from the ACPI table data
d8673cf67 Fix acpi spmi searching corner case

[Regression Risk]
While neither myself nor upstream are aware of a system where the existing freeipmi ACPI/SPMI parsing code works - and works *correctly* - it is possible that there is such a system. From my experimentation and reading of the code - even if the existing code to extract an SPMI table from /dev/mem were to work, the parsing of that table would be buggy and would not report correct information (see commits 3c6fa7054 and 40ba578f8 for details). However - an earlier version of this code did apparently work at some point on some system - so it's possible that my parsing fixes would now break said system. Note that for this to be an issue for the MAAS use-case, that system would also need to *not* also describe the IPMI device in SMBIOS, which my survey of 4 systems shows to be nearly always the case (except for the 1 case that triggered this whole thing).

dann frazier (dannf)
Changed in freeipmi (Ubuntu Bionic):
status: New → In Progress
importance: Undecided → High
assignee: nobody → dann frazier (dannf)
description: updated
description: updated
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package freeipmi - 1.5.7-2ubuntu1

---------------
freeipmi (1.5.7-2ubuntu1) cosmic; urgency=medium

  * Merge from Debian unstable. Remaining changes:
    - debian/rules: Build with "-pie,-bindnow"
    - Use dh-autoreconf to update libtool for ppc64el.
    - Fix dso linking on the convenience library.
    - Add IPv6 support to ipmipower.
    - Add ipv6 ipmi-discovery.
  * Dropped changes, included upstream:
    - Use kernel DMI table interface, when available, instead of trolling
      /dev/mem to avoid a crash on ARM. LP: #1699933.
  * Cherry-pick patches from upstream that add support to ipmi-locate
    for parsing ACPI/SPMI tables out of sysfs (LP: #1784926):
    - 0001-Fix-acpi-spmi-searching-corner-case.patch
    - 0002-Don-t-try-to-separate-the-header-from-the-ACPI-table.patch
    - 0003-Split-RSDT-XSDT-parsing-into-new-function.patch
    - 0004-Add-support-for-parsing-SPMI-tables-exposed-via-sysf.patch
    - 0005-Allow-sysfs-SPMI-parsing-on-ARM-platforms.patch
    - 0006-Correct-order-of-bytes-in-specification_revision-fie.patch

 -- dann frazier <email address hidden> Thu, 02 Aug 2018 15:13:57 -0600

Changed in freeipmi (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello dann, or anyone else affected,

Accepted freeipmi into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/freeipmi/1.4.11-1.1ubuntu4.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in freeipmi (Ubuntu Bionic):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-bionic
Revision history for this message
dann frazier (dannf) wrote :

Verification on a HiSilicon D06:

...
Setting up freeipmi-common (1.4.11-1.1ubuntu4.1) ...
Setting up libfreeipmi16 (1.4.11-1.1ubuntu4.1) ...
Setting up libipmiconsole2 (1.4.11-1.1ubuntu4.1) ...
Setting up libipmidetect0 (1.4.11-1.1ubuntu4.1) ...
Setting up freeipmi-tools (1.4.11-1.1ubuntu4.1) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...
ubuntu@d06-1:~$ sudo ipmi-locate
Probing KCS device using DMIDECODE... FAILED

Probing SMIC device using DMIDECODE... FAILED

Probing BT device using DMIDECODE... FAILED

Probing SSIF device using DMIDECODE... FAILED

Probing KCS device using SMBIOS... FAILED

Probing SMIC device using SMBIOS... FAILED

Probing BT device using SMBIOS... FAILED

Probing SSIF device using SMBIOS... FAILED

Probing KCS device using ACPI... FAILED

Probing SMIC device using ACPI... FAILED

Probing BT device using ACPI... done
IPMI Version: 2.0
IPMI locate driver: ACPI
IPMI interface: BT
BMC driver device:
BMC memory base address: 0xE4
Register spacing: 1

Probing SSIF device using ACPI... FAILED

Probing KCS device using PCI... FAILED

Probing SMIC device using PCI... FAILED

Probing BT device using PCI... FAILED

Probing SSIF device using PCI... FAILED

Revision history for this message
dann frazier (dannf) wrote :
Download full text (5.7 KiB)

Verification on an HP DL165 G7. Note that the DMIDECODE output hasn't changed, but we now see ACPI output. That ACPI output is inconsistent with the DMIDECODE output (IPMI Version 1.5 vs. 1.0), but that appears to be a bug in the ACPI table - not a parsing issue.

ubuntu@bysen:~$ sudo ipmi-locate
Probing KCS device using DMIDECODE... done
IPMI Version: 1.5
IPMI locate driver: DMIDECODE
IPMI interface: KCS
BMC driver device:
BMC I/O base address: 0xCA2
Register spacing: 1

Probing SMIC device using DMIDECODE... FAILED

Probing BT device using DMIDECODE... FAILED

Probing SSIF device using DMIDECODE... FAILED

Probing KCS device using SMBIOS... FAILED

Probing SMIC device using SMBIOS... FAILED

Probing BT device using SMBIOS... FAILED

Probing SSIF device using SMBIOS... FAILED

Probing KCS device using ACPI... FAILED

Probing SMIC device using ACPI... FAILED

Probing BT device using ACPI... FAILED

Probing SSIF device using ACPI... FAILED

Probing KCS device using PCI... FAILED

Probing SMIC device using PCI... FAILED

Probing BT device using PCI... FAILED

Probing SSIF device using PCI... FAILED

ubuntu@bysen:~$ sudo apt install freeipmi-tools -y
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  freeipmi freeipmi-bmc-watchdog freeipmi-common freeipmi-ipmidetect
  libfreeipmi16 libipmiconsole2 libipmidetect0
The following packages will be upgraded:
  freeipmi freeipmi-bmc-watchdog freeipmi-common freeipmi-ipmidetect
  freeipmi-tools libfreeipmi16 libipmiconsole2 libipmidetect0
8 upgraded, 0 newly installed, 0 to remove and 27 not upgraded.
Need to get 1815 kB of archives.
After this operation, 138 kB disk space will be freed.
Get:1 http://us.archive.ubuntu.com/ubuntu bionic-proposed/main amd64 libipmiconsole2 amd64 1.4.11-1.1ubuntu4.1 [83.9 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu bionic-proposed/main amd64 libipmidetect0 amd64 1.4.11-1.1ubuntu4.1 [25.6 kB]
Get:3 http://us.archive.ubuntu.com/ubuntu bionic-proposed/main amd64 freeipmi-tools amd64 1.4.11-1.1ubuntu4.1 [622 kB]
Get:4 http://us.archive.ubuntu.com/ubuntu bionic-proposed/universe amd64 freeipmi-ipmidetect amd64 1.4.11-1.1ubuntu4.1 [38.4 kB]
Get:5 http://us.archive.ubuntu.com/ubuntu bionic-proposed/universe amd64 freeipmi amd64 1.4.11-1.1ubuntu4.1 [1292 B]
Get:6 http://us.archive.ubuntu.com/ubuntu bionic-proposed/universe amd64 freeipmi-bmc-watchdog amd64 1.4.11-1.1ubuntu4.1 [44.0 kB]
Get:7 http://us.archive.ubuntu.com/ubuntu bionic-proposed/main amd64 libfreeipmi16 amd64 1.4.11-1.1ubuntu4.1 [826 kB]
Get:8 http://us.archive.ubuntu.com/ubuntu bionic-proposed/main amd64 freeipmi-common amd64 1.4.11-1.1ubuntu4.1 [174 kB]
Fetched 1815 kB in 0s (24.9 MB/s)
(Reading database ... 97718 files and directories currently installed.)
Preparing to unpack .../0-libipmiconsole2_1.4.11-1.1ubuntu4.1_amd64.deb ...
Unpacking libipmiconsole2 (1.4.11-1.1ubuntu4.1) over (1.4.11-1.1ubuntu4) ...
Preparing to unpack .../1-libipmidetect0_1.4.11-1.1ubuntu4.1_amd64.deb ...
Unpacking libipmidetect0 (1.4.11-1.1ubuntu4.1) over (1.4.11-1.1ubuntu4) ...
Preparing to unpack .../2-freeipmi-tools_1.4.11-1....

Read more...

Revision history for this message
dann frazier (dannf) wrote :

Verification on a QuantaGrid D52B-1U.

ubuntu@jehan:~$ sudo ipmi-locate
Probing KCS device using DMIDECODE... done
IPMI Version: 2.0
IPMI locate driver: DMIDECODE
IPMI interface: KCS
BMC driver device:
BMC I/O base address: 0xCA2
Register spacing: 1

Probing SMIC device using DMIDECODE... FAILED

Probing BT device using DMIDECODE... FAILED

Probing SSIF device using DMIDECODE... FAILED

Probing KCS device using SMBIOS... ERROR: permission denied

Probing SMIC device using SMBIOS... ERROR: permission denied

Probing BT device using SMBIOS... ERROR: permission denied

Probing SSIF device using SMBIOS... ERROR: permission denied

Probing KCS device using ACPI... done
IPMI Version: 2.0
IPMI locate driver: ACPI
IPMI interface: KCS
BMC driver device:
BMC I/O base address: 0xCA2
Register spacing: 1

Probing SMIC device using ACPI... FAILED

Probing BT device using ACPI... FAILED

Probing SSIF device using ACPI... FAILED

Probing KCS device using PCI... FAILED

Probing SMIC device using PCI... FAILED

Probing BT device using PCI... FAILED

Probing SSIF device using PCI... FAILED

Revision history for this message
dann frazier (dannf) wrote :
Download full text (4.3 KiB)

Verification on an HP DL385 G7:

ubuntu@jellyplum:~$ sudo ipmi-locate
Probing KCS device using DMIDECODE... done
IPMI Version: 2.0
IPMI locate driver: DMIDECODE
IPMI interface: KCS
BMC driver device:
BMC I/O base address: 0xCA2
Register spacing: 1

Probing SMIC device using DMIDECODE... FAILED

Probing BT device using DMIDECODE... FAILED

Probing SSIF device using DMIDECODE... FAILED

Probing KCS device using SMBIOS... FAILED

Probing SMIC device using SMBIOS... FAILED

Probing BT device using SMBIOS... FAILED

Probing SSIF device using SMBIOS... FAILED

Probing KCS device using ACPI... FAILED

Probing SMIC device using ACPI... FAILED

Probing BT device using ACPI... FAILED

Probing SSIF device using ACPI... FAILED

Probing KCS device using PCI... FAILED

Probing SMIC device using PCI... FAILED

Probing BT device using PCI... FAILED

Probing SSIF device using PCI... FAILED

ubuntu@jellyplum:~$ sudo apt install -y freeipmi-tools
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  freeipmi-common libfreeipmi16 libipmiconsole2 libipmidetect0
Suggested packages:
  freeipmi-ipmidetect freeipmi-bmc-watchdog
The following packages will be upgraded:
  freeipmi-common freeipmi-tools libfreeipmi16 libipmiconsole2 libipmidetect0
5 upgraded, 0 newly installed, 0 to remove and 27 not upgraded.
Need to get 1731 kB of archives.
After this operation, 131 kB disk space will be freed.
Get:1 http://us.archive.ubuntu.com/ubuntu bionic-proposed/main amd64 libfreeipmi16 amd64 1.4.11-1.1ubuntu4.1 [826 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu bionic-proposed/main amd64 libipmiconsole2 amd64 1.4.11-1.1ubuntu4.1 [83.9 kB]
Get:3 http://us.archive.ubuntu.com/ubuntu bionic-proposed/main amd64 libipmidetect0 amd64 1.4.11-1.1ubuntu4.1 [25.6 kB]
Get:4 http://us.archive.ubuntu.com/ubuntu bionic-proposed/main amd64 freeipmi-tools amd64 1.4.11-1.1ubuntu4.1 [622 kB]
Get:5 http://us.archive.ubuntu.com/ubuntu bionic-proposed/main amd64 freeipmi-common amd64 1.4.11-1.1ubuntu4.1 [174 kB]
Fetched 1731 kB in 0s (15.7 MB/s)
(Reading database ... 97698 files and directories currently installed.)
Preparing to unpack .../libfreeipmi16_1.4.11-1.1ubuntu4.1_amd64.deb ...
Unpacking libfreeipmi16 (1.4.11-1.1ubuntu4.1) over (1.4.11-1.1ubuntu4) ...
Preparing to unpack .../libipmiconsole2_1.4.11-1.1ubuntu4.1_amd64.deb ...
Unpacking libipmiconsole2 (1.4.11-1.1ubuntu4.1) over (1.4.11-1.1ubuntu4) ...
Preparing to unpack .../libipmidetect0_1.4.11-1.1ubuntu4.1_amd64.deb ...
Unpacking libipmidetect0 (1.4.11-1.1ubuntu4.1) over (1.4.11-1.1ubuntu4) ...
Preparing to unpack .../freeipmi-tools_1.4.11-1.1ubuntu4.1_amd64.deb ...
Unpacking freeipmi-tools (1.4.11-1.1ubuntu4.1) over (1.4.11-1.1ubuntu4) ...
Preparing to unpack .../freeipmi-common_1.4.11-1.1ubuntu4.1_amd64.deb ...
Unpacking freeipmi-common (1.4.11-1.1ubuntu4.1) over (1.4.11-1.1ubuntu4) ...
Processing triggers for install-info (6.5.0.dfsg.1-2) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...
Processing triggers for man-db (2.8.3-2) ...
Setting up freeipmi-common (1.4.11-1.1ubuntu4.1) ...
Setting up libfreeipmi16 (1.4.11-1.1ubuntu4.1)...

Read more...

Revision history for this message
dann frazier (dannf) wrote :

Oops, missed the "before" for the QuantaGrid in Comment #5:

ubuntu@jehan:~$ sudo ipmi-locate
Probing KCS device using DMIDECODE... done
IPMI Version: 2.0
IPMI locate driver: DMIDECODE
IPMI interface: KCS
BMC driver device:
BMC I/O base address: 0xCA2
Register spacing: 1

Probing SMIC device using DMIDECODE... FAILED

Probing BT device using DMIDECODE... FAILED

Probing SSIF device using DMIDECODE... FAILED

Probing KCS device using SMBIOS... ERROR: permission denied

Probing SMIC device using SMBIOS... ERROR: permission denied

Probing BT device using SMBIOS... ERROR: permission denied

Probing SSIF device using SMBIOS... ERROR: permission denied

Probing KCS device using ACPI... ERROR: permission denied

Probing SMIC device using ACPI... ERROR: permission denied

Probing BT device using ACPI... ERROR: permission denied

Probing SSIF device using ACPI... ERROR: permission denied

Probing KCS device using PCI... FAILED

Probing SMIC device using PCI... FAILED

Probing BT device using PCI... FAILED

Probing SSIF device using PCI... FAILED

dann frazier (dannf)
tags: added: verification-done verification-done-bionic
removed: verification-needed verification-needed-bionic
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package freeipmi - 1.4.11-1.1ubuntu4.1

---------------
freeipmi (1.4.11-1.1ubuntu4.1) bionic; urgency=medium

  * Cherry-pick patches from upstream that add support to ipmi-locate
    for parsing ACPI/SPMI tables out of sysfs (LP: #1784926):
    - 0001-Fix-acpi-spmi-searching-corner-case.patch
    - 0002-Don-t-try-to-separate-the-header-from-the-ACPI-table.patch
    - 0003-Split-RSDT-XSDT-parsing-into-new-function.patch
    - 0004-Add-support-for-parsing-SPMI-tables-exposed-via-sysf.patch
    - 0005-Allow-sysfs-SPMI-parsing-on-ARM-platforms.patch
    - 0006-Correct-order-of-bytes-in-specification_revision-fie.patch

 -- dann frazier <email address hidden> Fri, 03 Aug 2018 13:59:53 -0600

Changed in freeipmi (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for freeipmi has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

dann frazier (dannf)
Changed in freeipmi (Ubuntu Xenial):
status: New → In Progress
assignee: nobody → dann frazier (dannf)
importance: Undecided → High
Revision history for this message
Timo Aaltonen (tjaalton) wrote : Please test proposed package

Hello dann, or anyone else affected,

Accepted freeipmi into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/freeipmi/1.4.11-1.1ubuntu4.1~0.16.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in freeipmi (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-xenial
removed: verification-done
Revision history for this message
dann frazier (dannf) wrote :

xenial verification:

ubuntu@d06-4:~$ sudo ipmi-locate
Probing KCS device using DMIDECODE... FAILED

Probing SMIC device using DMIDECODE... FAILED

Probing BT device using DMIDECODE... FAILED

Probing SSIF device using DMIDECODE... FAILED

Probing KCS device using SMBIOS... FAILED

Probing SMIC device using SMBIOS... FAILED

Probing BT device using SMBIOS... FAILED

Probing SSIF device using SMBIOS... FAILED

Probing KCS device using ACPI... FAILED

Probing SMIC device using ACPI... FAILED

Probing BT device using ACPI... done
IPMI Version: 2.0
IPMI locate driver: ACPI
IPMI interface: BT
BMC driver device:
BMC memory base address: 0xE4
Register spacing: 1

Probing SSIF device using ACPI... FAILED

Probing KCS device using PCI... FAILED

Probing SMIC device using PCI... FAILED

Probing BT device using PCI... FAILED

Probing SSIF device using PCI... FAILED

ubuntu@d06-4:~$

tags: added: verification-done verification-done-xenial
removed: verification-needed verification-needed-xenial
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package freeipmi - 1.4.11-1.1ubuntu4.1~0.16.04

---------------
freeipmi (1.4.11-1.1ubuntu4.1~0.16.04) xenial; urgency=medium

  * Backport to xenial.

freeipmi (1.4.11-1.1ubuntu4.1) bionic; urgency=medium

  * Cherry-pick patches from upstream that add support to ipmi-locate
    for parsing ACPI/SPMI tables out of sysfs (LP: #1784926):
    - 0001-Fix-acpi-spmi-searching-corner-case.patch
    - 0002-Don-t-try-to-separate-the-header-from-the-ACPI-table.patch
    - 0003-Split-RSDT-XSDT-parsing-into-new-function.patch
    - 0004-Add-support-for-parsing-SPMI-tables-exposed-via-sysf.patch
    - 0005-Allow-sysfs-SPMI-parsing-on-ARM-platforms.patch
    - 0006-Correct-order-of-bytes-in-specification_revision-fie.patch

 -- dann frazier <email address hidden> Thu, 14 Feb 2019 14:13:56 -0700

Changed in freeipmi (Ubuntu Xenial):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.