Installing libmkl results in incorrect output from linear algebra routines in R, like matrix inversion

[Alex]

Hello maintainers. Thank you for your hard work keeping this wonderful operating system going. This bug report relates to my experience using libmkl on Ubuntu 22.04.1 LTS, via the R programming language. It is possible this is a problem with R and not libmkl, I guess, although I'm not sure how I would tell the difference.

Basic linear algebra routines are giving incorrect results under libmkl, for operations involving all but the smallest of dense matrices.

Referring to the minimal R repro script given below, which inverts a 40x40 toeplitz matrix. I chose this example because you can see at a glance if there are problems in the resulting inverse, which is shown as a heatmap plot (using the image() function in R).

  - The expected output (without mkl): all 9 panels in the generated image showing a diagonal stripe.
  - Actual output (with mkl): nonsensical results, different every invocation. There are no error messages.

See the uploaded image. Expected and actual output are shown on the left and right, respectively.

Details:

The CPU is an Intel i7-5820K.
Description: Ubuntu 22.04.1 LTS
Release: 22.04
libmkl-dev package version: 2020.4.304-2ubuntu3
R version: 4.2.1-2.2204.0 from https://cloud.r-project.org/bin/linux/ubuntu jammy-cran40/ Packages

I can reliably replicate this bug by running the following R script, with and without the libmkl libraries installed. That is, by first running apt-get install libmkl-dev to install mkl and apt-get remove libmkl-dev; apt autoremove to remove it.

#!/usr/bin/Rscript

# create 40x40 positive toeplitz matrix
A <- toeplitz(c(10,-2,rep(0,38)))
stopifnot(det(A) > 0) # A>0

# display heatmap of original, then inverse done 8 times
png('mklbug.png', height=800, width=800)
par(mfrow=c(3,3))
image(A, main="Original matrix A")
for (i in 1:8) {
  image(solve(A), main=sprintf("A inverse (run %d)", i))
}
dev.off()

ProblemType: Bug
DistroRelease: Ubuntu 22.04
Package: libmkl-dev 2020.4.304-2ubuntu3
ProcVersionSignature: Ubuntu 5.15.0-47.51-generic 5.15.46
Uname: Linux 5.15.0-47-generic x86_64
NonfreeKernelModules: nvidia_modeset nvidia
ApportVersion: 2.20.11-0ubuntu82.1
Architecture: amd64
CasperMD5CheckResult: unknown
CurrentDesktop: ubuntu:GNOME
Date: Fri Sep 9 13:58:21 2022
InstallationDate: Installed on 2021-04-02 (525 days ago)
InstallationMedia: Ubuntu 20.04.2.0 LTS "Focal Fossa" - Release amd64 (20210209.1)
SourcePackage: intel-mkl
UpgradeStatus: Upgraded to jammy on 2022-08-15 (24 days ago)

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Pete S (panoply)]

I have reproduced this error in Ubuntu 20.04.
Perhaps it relates to mixing Intel OpenMP and GNU OpenMP, as elaborated here https://www.jyotirmoy.net/posts/2021-10-03-r-mkl-openmp.html and here https://github.com/eddelbuettel/mkl4deb/issues/2

However, implementing the proposed fix by setting the MKL_THREADING_LAYER=GNU environment variable did not resolve the error.

[Pete S (panoply)]

On further exploration, it seems that setting MKL_THREADING_LAYER=GNU or MKL_THREADING_LAYER=SEQUENTIAL fixes the problem.

I was initially fooled because Rstudio doesn't load the shell environment variables, so it needs to be set in the relevant .Renviron. Calling Sys.getenv() will confirm the R session (either on the command line or in Rstudio) is seeing the MKL_THREADING_LAYER variable.