I tried mainline kernel 4.20-rc2, 4.18, 4.17, 4.16 and 4.15. They all passed a 50 run test.
To validate when the issue is introduced, I installed several Ubuntu kernel packages and address the issue/regression happens between 4.15.0-34 and 4.15.0-36.
Finished the kernel bisect and probably found the first bad commit.
$ git bisect log
# bad: [fd01374000c83b59d3ce234fbb970cb91404bd42] UBUNTU: Ubuntu-4.15.0-36.39
# good: [ffaad0a9f46742f6d71af975a6a061ffe6963aca] UBUNTU: Ubuntu-4.15.0-34.37
git bisect start 'Ubuntu-4.15.0-36.39' 'Ubuntu-4.15.0-34.37'
# good: [003ae88ae88d48643e71dc69c18d4eda598339d5] Revert "UBUNTU: SAUCE: net: hns3: Fix for VF mailbox receiving unknown message"
git bisect good 003ae88ae88d48643e71dc69c18d4eda598339d5
# good: [d3d4b20788eee72dcb1ed5ace7dbee6aafbe65cf] net: hns3: Fix for mac pause not disable in pfc mode
git bisect good d3d4b20788eee72dcb1ed5ace7dbee6aafbe65cf
# good: [8e499f213175b65bcc08a3c685ea6717e7668cec] arm64: ssbd: Introduce thread flag to control userspace mitigation
git bisect good 8e499f213175b65bcc08a3c685ea6717e7668cec
# good: [11cdaf61c1986ea682398b48e238fd915393b2e6] KVM: PPC: Check if IOMMU page is contained in the pinned physical page
git bisect good 11cdaf61c1986ea682398b48e238fd915393b2e6
# good: [db6800337d38a5b2adbaa78fcb2c299362222e1d] s390: fix br_r1_trampoline for machines without exrl
git bisect good db6800337d38a5b2adbaa78fcb2c299362222e1d
# good: [f1f016ed54582502d59de991ddbecfe2373722c0] x86/speculation/l1tf: Increase l1tf memory limit for Nehalem+
git bisect good f1f016ed54582502d59de991ddbecfe2373722c0
# good: [f1f016ed54582502d59de991ddbecfe2373722c0] x86/speculation/l1tf: Increase l1tf memory limit for Nehalem+
git bisect good f1f016ed54582502d59de991ddbecfe2373722c0
# bad: [cb3b0751997c8bd45c76a7401e2edb019cdaaab3] UBUNTU: Start new release
git bisect bad cb3b0751997c8bd45c76a7401e2edb019cdaaab3
# bad: [cb3b0751997c8bd45c76a7401e2edb019cdaaab3] UBUNTU: Start new release
git bisect bad cb3b0751997c8bd45c76a7401e2edb019cdaaab3
# bad: [d5bad4136d14cda66cb7c06b10b80c64ae695c78] UBUNTU: Ubuntu-4.15.0-35.38
git bisect bad d5bad4136d14cda66cb7c06b10b80c64ae695c78
# bad: [876dcb5f4576934a1a11b091b40ce548f048340e] UBUNTU: SAUCE: vfio -- release device lock before userspace requests
git bisect bad 876dcb5f4576934a1a11b091b40ce548f048340e
# first bad commit: [876dcb5f4576934a1a11b091b40ce548f048340e] UBUNTU: SAUCE: vfio -- release device lock before userspace requests
I tried mainline kernel 4.20-rc2, 4.18, 4.17, 4.16 and 4.15. They all passed a 50 run test.
To validate when the issue is introduced, I installed several Ubuntu kernel packages and address the issue/regression happens between 4.15.0-34 and 4.15.0-36.
Finished the kernel bisect and probably found the first bad commit. 59d3ce234fbb970 cb91404bd42] UBUNTU: Ubuntu-4.15.0-36.39 f6d71af975a6a06 1ffe6963aca] UBUNTU: Ubuntu-4.15.0-34.37 4.15.0- 36.39' 'Ubuntu- 4.15.0- 34.37' 643e71dc69c18d4 eda598339d5] Revert "UBUNTU: SAUCE: net: hns3: Fix for VF mailbox receiving unknown message" 43e71dc69c18d4e da598339d5 2dcb1ed5ace7dbe e6aafbe65cf] net: hns3: Fix for mac pause not disable in pfc mode dcb1ed5ace7dbee 6aafbe65cf 5bcc08a3c685ea6 717e7668cec] arm64: ssbd: Introduce thread flag to control userspace mitigation bcc08a3c685ea67 17e7668cec a682398b48e238f d915393b2e6] KVM: PPC: Check if IOMMU page is contained in the pinned physical page 682398b48e238fd 915393b2e6 b2adbaa78fcb2c2 99362222e1d] s390: fix br_r1_trampoline for machines without exrl 2adbaa78fcb2c29 9362222e1d 02d59de991ddbec fe2373722c0] x86/speculation /l1tf: Increase l1tf memory limit for Nehalem+ 2d59de991ddbecf e2373722c0 02d59de991ddbec fe2373722c0] x86/speculation /l1tf: Increase l1tf memory limit for Nehalem+ 2d59de991ddbecf e2373722c0 d45c76a7401e2ed b019cdaaab3] UBUNTU: Start new release 45c76a7401e2edb 019cdaaab3 d45c76a7401e2ed b019cdaaab3] UBUNTU: Start new release 45c76a7401e2edb 019cdaaab3 a66cb7c06b10b80 c64ae695c78] UBUNTU: Ubuntu-4.15.0-35.38 66cb7c06b10b80c 64ae695c78 4a1a11b091b40ce 548f048340e] UBUNTU: SAUCE: vfio -- release device lock before userspace requests a1a11b091b40ce5 48f048340e 4a1a11b091b40ce 548f048340e] UBUNTU: SAUCE: vfio -- release device lock before userspace requests
$ git bisect log
# bad: [fd01374000c83b
# good: [ffaad0a9f46742
git bisect start 'Ubuntu-
# good: [003ae88ae88d48
git bisect good 003ae88ae88d486
# good: [d3d4b20788eee7
git bisect good d3d4b20788eee72
# good: [8e499f213175b6
git bisect good 8e499f213175b65
# good: [11cdaf61c1986e
git bisect good 11cdaf61c1986ea
# good: [db6800337d38a5
git bisect good db6800337d38a5b
# good: [f1f016ed545825
git bisect good f1f016ed5458250
# good: [f1f016ed545825
git bisect good f1f016ed5458250
# bad: [cb3b0751997c8b
git bisect bad cb3b0751997c8bd
# bad: [cb3b0751997c8b
git bisect bad cb3b0751997c8bd
# bad: [d5bad4136d14cd
git bisect bad d5bad4136d14cda
# bad: [876dcb5f457693
git bisect bad 876dcb5f4576934
# first bad commit: [876dcb5f457693