Ubuntu
dmidecode package

  1. Bug #1851676
  2. Comment #4

Comment 4 for bug 1851676

Revision history for this message
Pedro Guimarães (pguimaraes) wrote : Re: dmidecode decodes /sys/class/dmi/id/product_serial incorrectly

Hi Tim, I don't think patching kubernetes-master will resolve this issue. Mainly because the issue happens on kubernetes-workers.

We were seeing PVCs failing with: "No VM found". When we look into the logs, actually kubernetes was trying to learn about its kubernetes-workers through SystemUUID.

Essentially:
https://github.com/kubernetes/kubernetes/blob/103e926604de6f79161b78af3e792d0ed282bc06/staging/src/k8s.io/legacy-cloud-providers/vsphere/vsphere_util.go#L656

Recovers UUID from SystemUUID, which according to https://github.com/kubernetes/kubernetes/issues/58927
Comes from product_serial

However, because of this dmidecode issue, the first half of the bytes are inversed (big endian vs. small endian).
What we saw on kubernetes-controller-manager log then was k8s grabbing a half-wrong UUID and trying to fetch it using FindVMUUID method. Of course that broke, since product_uuid held the right values.

To fix that, we have two ways:
1) work-around: run VMs with compatibility set for ESXi 5.5 and later; that forces vsphere to setup those VMs with older HW version (10) and there is no issue with dmidecode
2) Find out why dmidecode diverges when going to version 13 and fix it