| 2022-08-29 00:37:28 |
Olafur Helgi Haraldsson |
bug |
|
|
added bug |
| 2022-08-29 00:43:47 |
Olafur Helgi Haraldsson |
summary |
xfs freeze |
xfs freeze every week on multiple machines |
|
| 2022-08-29 00:50:02 |
Olafur Helgi Haraldsson |
description |
xfs freeze
ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: linux-image-5.4.0-124-generic 5.4.0-124.140
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
Uname: Linux 5.4.0-124-generic x86_64
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
Date: Mon Aug 29 00:34:28 2022
InstallationDate: Installed on 2021-02-10 (564 days ago)
InstallationMedia: Ubuntu-Server 20.04.2 LTS "Focal Fossa" - Release amd64 (20210201.2)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A02
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA02:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc. |
We run multiple machines that have the purpose of acting as a data repository for Veeam backup. Each machine has 2x 264G XFS volumes that use heavy reflinking. This works as intended, for one issue: our xfs freeze once a week, a few minutes after midnight on Monday nights. We can only do a reboot to get the servers working again. Then it works for a week again.
We have been interacting with support from Veeam, but this looks like some XFS kernel issue, or a race condition between the Veeam process and XFS.
ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: linux-image-5.4.0-124-generic 5.4.0-124.140
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
Uname: Linux 5.4.0-124-generic x86_64
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
Date: Mon Aug 29 00:34:28 2022
InstallationDate: Installed on 2021-02-10 (564 days ago)
InstallationMedia: Ubuntu-Server 20.04.2 LTS "Focal Fossa" - Release amd64 (20210201.2)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A02
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA02:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc. |
|
| 2022-08-29 00:51:19 |
Olafur Helgi Haraldsson |
tags |
amd64 apport-bug focal uec-images |
amd64 apport-bug apport-collected focal uec-images |
|
| 2022-08-29 00:51:20 |
Olafur Helgi Haraldsson |
description |
We run multiple machines that have the purpose of acting as a data repository for Veeam backup. Each machine has 2x 264G XFS volumes that use heavy reflinking. This works as intended, for one issue: our xfs freeze once a week, a few minutes after midnight on Monday nights. We can only do a reboot to get the servers working again. Then it works for a week again.
We have been interacting with support from Veeam, but this looks like some XFS kernel issue, or a race condition between the Veeam process and XFS.
ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: linux-image-5.4.0-124-generic 5.4.0-124.140
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
Uname: Linux 5.4.0-124-generic x86_64
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
Date: Mon Aug 29 00:34:28 2022
InstallationDate: Installed on 2021-02-10 (564 days ago)
InstallationMedia: Ubuntu-Server 20.04.2 LTS "Focal Fossa" - Release amd64 (20210201.2)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A02
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA02:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc. |
We run multiple machines that have the purpose of acting as a data repository for Veeam backup. Each machine has 2x 264G XFS volumes that use heavy reflinking. This works as intended, for one issue: our xfs freeze once a week, a few minutes after midnight on Monday nights. We can only do a reboot to get the servers working again. Then it works for a week again.
We have been interacting with support from Veeam, but this looks like some XFS kernel issue, or a race condition between the Veeam process and XFS.
ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: linux-image-5.4.0-124-generic 5.4.0-124.140
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
Uname: Linux 5.4.0-124-generic x86_64
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
Date: Mon Aug 29 00:34:28 2022
InstallationDate: Installed on 2021-02-10 (564 days ago)
InstallationMedia: Ubuntu-Server 20.04.2 LTS "Focal Fossa" - Release amd64 (20210201.2)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A02
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA02:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc.
---
ProblemType: Bug
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
DistroRelease: Ubuntu 20.04
InstallationDate: Installed on 2021-09-16 (346 days ago)
InstallationMedia: Ubuntu-Server 20.04.3 LTS "Focal Fossa" - Release amd64 (20210824)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
Package: linux (not installed)
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
Tags: focal uec-images
Uname: Linux 5.4.0-124-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: N/A
_MarkForUpload: True
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A05
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA05:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc. |
|
| 2022-08-29 00:51:21 |
Olafur Helgi Haraldsson |
attachment added |
|
CRDA.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612207/+files/CRDA.txt |
|
| 2022-08-29 00:51:23 |
Olafur Helgi Haraldsson |
attachment added |
|
CurrentDmesg.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612208/+files/CurrentDmesg.txt |
|
| 2022-08-29 00:51:24 |
Olafur Helgi Haraldsson |
attachment added |
|
Lspci.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612209/+files/Lspci.txt |
|
| 2022-08-29 00:51:25 |
Olafur Helgi Haraldsson |
attachment added |
|
Lspci-vt.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612210/+files/Lspci-vt.txt |
|
| 2022-08-29 00:51:27 |
Olafur Helgi Haraldsson |
attachment added |
|
Lsusb.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612211/+files/Lsusb.txt |
|
| 2022-08-29 00:51:28 |
Olafur Helgi Haraldsson |
attachment added |
|
Lsusb-t.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612212/+files/Lsusb-t.txt |
|
| 2022-08-29 00:51:29 |
Olafur Helgi Haraldsson |
attachment added |
|
Lsusb-v.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612213/+files/Lsusb-v.txt |
|
| 2022-08-29 00:51:30 |
Olafur Helgi Haraldsson |
attachment added |
|
ProcCpuinfo.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612214/+files/ProcCpuinfo.txt |
|
| 2022-08-29 00:51:31 |
Olafur Helgi Haraldsson |
description |
We run multiple machines that have the purpose of acting as a data repository for Veeam backup. Each machine has 2x 264G XFS volumes that use heavy reflinking. This works as intended, for one issue: our xfs freeze once a week, a few minutes after midnight on Monday nights. We can only do a reboot to get the servers working again. Then it works for a week again.
We have been interacting with support from Veeam, but this looks like some XFS kernel issue, or a race condition between the Veeam process and XFS.
ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: linux-image-5.4.0-124-generic 5.4.0-124.140
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
Uname: Linux 5.4.0-124-generic x86_64
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
Date: Mon Aug 29 00:34:28 2022
InstallationDate: Installed on 2021-02-10 (564 days ago)
InstallationMedia: Ubuntu-Server 20.04.2 LTS "Focal Fossa" - Release amd64 (20210201.2)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A02
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA02:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc.
---
ProblemType: Bug
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
DistroRelease: Ubuntu 20.04
InstallationDate: Installed on 2021-09-16 (346 days ago)
InstallationMedia: Ubuntu-Server 20.04.3 LTS "Focal Fossa" - Release amd64 (20210824)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
Package: linux (not installed)
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
Tags: focal uec-images
Uname: Linux 5.4.0-124-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: N/A
_MarkForUpload: True
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A05
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA05:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc. |
We run multiple machines that have the purpose of acting as a data repository for Veeam backup. Each machine has 2x 264G XFS volumes that use heavy reflinking. This works as intended, for one issue: our xfs freeze once a week, a few minutes after midnight on Monday nights. We can only do a reboot to get the servers working again. Then it works for a week again.
We have been interacting with support from Veeam, but this looks like some XFS kernel issue, or a race condition between the Veeam process and XFS.
ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: linux-image-5.4.0-124-generic 5.4.0-124.140
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
Uname: Linux 5.4.0-124-generic x86_64
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
Date: Mon Aug 29 00:34:28 2022
InstallationDate: Installed on 2021-02-10 (564 days ago)
InstallationMedia: Ubuntu-Server 20.04.2 LTS "Focal Fossa" - Release amd64 (20210201.2)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A02
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA02:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc.
---
ProblemType: Bug
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
DistroRelease: Ubuntu 20.04
InstallationDate: Installed on 2021-09-16 (346 days ago)
InstallationMedia: Ubuntu-Server 20.04.3 LTS "Focal Fossa" - Release amd64 (20210824)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
Package: linux (not installed)
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
Tags: focal uec-images
Uname: Linux 5.4.0-124-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: N/A
_MarkForUpload: True
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A05
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA05:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc.
---
ProblemType: Bug
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
DistroRelease: Ubuntu 20.04
InstallationDate: Installed on 2021-02-10 (564 days ago)
InstallationMedia: Ubuntu-Server 20.04.2 LTS "Focal Fossa" - Release amd64 (20210201.2)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
Package: linux (not installed)
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
Tags: focal uec-images
Uname: Linux 5.4.0-124-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: N/A
_MarkForUpload: True
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A02
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA02:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc. |
|
| 2022-08-29 00:51:32 |
Olafur Helgi Haraldsson |
attachment added |
|
ProcCpuinfoMinimal.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612215/+files/ProcCpuinfoMinimal.txt |
|
| 2022-08-29 00:51:33 |
Olafur Helgi Haraldsson |
attachment added |
|
CRDA.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612216/+files/CRDA.txt |
|
| 2022-08-29 00:51:34 |
Olafur Helgi Haraldsson |
attachment added |
|
ProcInterrupts.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612217/+files/ProcInterrupts.txt |
|
| 2022-08-29 00:51:35 |
Olafur Helgi Haraldsson |
attachment added |
|
CurrentDmesg.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612218/+files/CurrentDmesg.txt |
|
| 2022-08-29 00:51:36 |
Olafur Helgi Haraldsson |
attachment added |
|
ProcModules.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612219/+files/ProcModules.txt |
|
| 2022-08-29 00:51:37 |
Olafur Helgi Haraldsson |
attachment added |
|
Lspci.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612220/+files/Lspci.txt |
|
| 2022-08-29 00:51:38 |
Olafur Helgi Haraldsson |
attachment added |
|
Lspci-vt.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612221/+files/Lspci-vt.txt |
|
| 2022-08-29 00:51:40 |
Olafur Helgi Haraldsson |
description |
We run multiple machines that have the purpose of acting as a data repository for Veeam backup. Each machine has 2x 264G XFS volumes that use heavy reflinking. This works as intended, for one issue: our xfs freeze once a week, a few minutes after midnight on Monday nights. We can only do a reboot to get the servers working again. Then it works for a week again.
We have been interacting with support from Veeam, but this looks like some XFS kernel issue, or a race condition between the Veeam process and XFS.
ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: linux-image-5.4.0-124-generic 5.4.0-124.140
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
Uname: Linux 5.4.0-124-generic x86_64
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
Date: Mon Aug 29 00:34:28 2022
InstallationDate: Installed on 2021-02-10 (564 days ago)
InstallationMedia: Ubuntu-Server 20.04.2 LTS "Focal Fossa" - Release amd64 (20210201.2)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A02
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA02:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc.
---
ProblemType: Bug
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
DistroRelease: Ubuntu 20.04
InstallationDate: Installed on 2021-09-16 (346 days ago)
InstallationMedia: Ubuntu-Server 20.04.3 LTS "Focal Fossa" - Release amd64 (20210824)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
Package: linux (not installed)
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
Tags: focal uec-images
Uname: Linux 5.4.0-124-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: N/A
_MarkForUpload: True
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A05
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA05:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc.
---
ProblemType: Bug
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
DistroRelease: Ubuntu 20.04
InstallationDate: Installed on 2021-02-10 (564 days ago)
InstallationMedia: Ubuntu-Server 20.04.2 LTS "Focal Fossa" - Release amd64 (20210201.2)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
Package: linux (not installed)
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
Tags: focal uec-images
Uname: Linux 5.4.0-124-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: N/A
_MarkForUpload: True
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A02
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA02:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc. |
We run multiple machines that have the purpose of acting as a data repository for Veeam backup. Each machine has 2x 264G XFS volumes that use heavy reflinking. This works as intended, for one issue: our xfs freeze once a week, a few minutes after midnight on Monday nights. We can only do a reboot to get the servers working again. Then it works for a week again.
We have been interacting with support from Veeam, but this looks like some XFS kernel issue, or a race condition between the Veeam process and XFS.
ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: linux-image-5.4.0-124-generic 5.4.0-124.140
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
Uname: Linux 5.4.0-124-generic x86_64
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
Date: Mon Aug 29 00:34:28 2022
InstallationDate: Installed on 2021-02-10 (564 days ago)
InstallationMedia: Ubuntu-Server 20.04.2 LTS "Focal Fossa" - Release amd64 (20210201.2)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A02
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA02:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc.
---
ProblemType: Bug
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
DistroRelease: Ubuntu 20.04
InstallationDate: Installed on 2021-09-16 (346 days ago)
InstallationMedia: Ubuntu-Server 20.04.3 LTS "Focal Fossa" - Release amd64 (20210824)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
Package: linux (not installed)
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
Tags: focal uec-images
Uname: Linux 5.4.0-124-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: N/A
_MarkForUpload: True
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A05
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA05:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc.
---
ProblemType: Bug
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
DistroRelease: Ubuntu 20.04
InstallationDate: Installed on 2021-02-10 (564 days ago)
InstallationMedia: Ubuntu-Server 20.04.2 LTS "Focal Fossa" - Release amd64 (20210201.2)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
Package: linux (not installed)
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
Tags: focal uec-images
Uname: Linux 5.4.0-124-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: N/A
_MarkForUpload: True
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A02
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA02:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc.
---
ProblemType: Bug
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:50 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:50 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
DistroRelease: Ubuntu 20.04
InstallationDate: Installed on 2021-02-10 (564 days ago)
InstallationMedia: Ubuntu-Server 20.04.2 LTS "Focal Fossa" - Release amd64 (20210201.2)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
Package: linux (not installed)
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
Tags: focal uec-images
Uname: Linux 5.4.0-124-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: N/A
_MarkForUpload: True
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A07
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA07:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc. |
|
| 2022-08-29 00:51:41 |
Olafur Helgi Haraldsson |
attachment added |
|
CRDA.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612222/+files/CRDA.txt |
|
| 2022-08-29 00:51:41 |
Olafur Helgi Haraldsson |
attachment added |
|
Lsusb.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612223/+files/Lsusb.txt |
|
| 2022-08-29 00:51:42 |
Olafur Helgi Haraldsson |
attachment added |
|
CurrentDmesg.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612224/+files/CurrentDmesg.txt |
|
| 2022-08-29 00:51:43 |
Olafur Helgi Haraldsson |
attachment added |
|
Lsusb-t.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612225/+files/Lsusb-t.txt |
|
| 2022-08-29 00:51:44 |
Olafur Helgi Haraldsson |
attachment added |
|
Lspci.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612226/+files/Lspci.txt |
|
| 2022-08-29 00:51:45 |
Olafur Helgi Haraldsson |
attachment added |
|
Lsusb-v.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612227/+files/Lsusb-v.txt |
|
| 2022-08-29 00:51:46 |
Olafur Helgi Haraldsson |
attachment added |
|
Lspci-vt.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612228/+files/Lspci-vt.txt |
|
| 2022-08-29 00:51:47 |
Olafur Helgi Haraldsson |
attachment added |
|
ProcCpuinfo.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612229/+files/ProcCpuinfo.txt |
|
| 2022-08-29 00:51:48 |
Olafur Helgi Haraldsson |
attachment added |
|
Lsusb.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612230/+files/Lsusb.txt |
|
| 2022-08-29 00:51:49 |
Olafur Helgi Haraldsson |
attachment added |
|
ProcCpuinfoMinimal.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612231/+files/ProcCpuinfoMinimal.txt |
|
| 2022-08-29 00:51:50 |
Olafur Helgi Haraldsson |
attachment added |
|
Lsusb-t.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612232/+files/Lsusb-t.txt |
|
| 2022-08-29 00:51:51 |
Olafur Helgi Haraldsson |
attachment added |
|
ProcInterrupts.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612233/+files/ProcInterrupts.txt |
|
| 2022-08-29 00:51:52 |
Olafur Helgi Haraldsson |
attachment added |
|
Lsusb-v.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612234/+files/Lsusb-v.txt |
|
| 2022-08-29 00:51:54 |
Olafur Helgi Haraldsson |
attachment added |
|
ProcModules.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612235/+files/ProcModules.txt |
|
| 2022-08-29 00:51:54 |
Olafur Helgi Haraldsson |
attachment added |
|
ProcCpuinfo.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612236/+files/ProcCpuinfo.txt |
|
| 2022-08-29 00:51:56 |
Olafur Helgi Haraldsson |
attachment added |
|
ProcCpuinfoMinimal.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612237/+files/ProcCpuinfoMinimal.txt |
|
| 2022-08-29 00:51:57 |
Olafur Helgi Haraldsson |
attachment added |
|
UdevDb.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612238/+files/UdevDb.txt |
|
| 2022-08-29 00:51:58 |
Olafur Helgi Haraldsson |
attachment added |
|
ProcInterrupts.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612239/+files/ProcInterrupts.txt |
|
| 2022-08-29 00:51:59 |
Olafur Helgi Haraldsson |
attachment added |
|
ProcModules.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612240/+files/ProcModules.txt |
|
| 2022-08-29 00:52:00 |
Olafur Helgi Haraldsson |
attachment added |
|
UdevDb.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612241/+files/UdevDb.txt |
|
| 2022-08-29 00:52:02 |
Olafur Helgi Haraldsson |
attachment added |
|
WifiSyslog.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612242/+files/WifiSyslog.txt |
|
| 2022-08-29 00:52:03 |
Olafur Helgi Haraldsson |
attachment added |
|
WifiSyslog.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612243/+files/WifiSyslog.txt |
|
| 2022-08-29 00:52:05 |
Olafur Helgi Haraldsson |
attachment added |
|
acpidump.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612244/+files/acpidump.txt |
|
| 2022-08-29 00:52:07 |
Olafur Helgi Haraldsson |
attachment added |
|
acpidump.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612245/+files/acpidump.txt |
|
| 2022-08-29 00:52:24 |
Olafur Helgi Haraldsson |
description |
We run multiple machines that have the purpose of acting as a data repository for Veeam backup. Each machine has 2x 264G XFS volumes that use heavy reflinking. This works as intended, for one issue: our xfs freeze once a week, a few minutes after midnight on Monday nights. We can only do a reboot to get the servers working again. Then it works for a week again.
We have been interacting with support from Veeam, but this looks like some XFS kernel issue, or a race condition between the Veeam process and XFS.
ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: linux-image-5.4.0-124-generic 5.4.0-124.140
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
Uname: Linux 5.4.0-124-generic x86_64
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
Date: Mon Aug 29 00:34:28 2022
InstallationDate: Installed on 2021-02-10 (564 days ago)
InstallationMedia: Ubuntu-Server 20.04.2 LTS "Focal Fossa" - Release amd64 (20210201.2)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A02
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA02:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc.
---
ProblemType: Bug
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
DistroRelease: Ubuntu 20.04
InstallationDate: Installed on 2021-09-16 (346 days ago)
InstallationMedia: Ubuntu-Server 20.04.3 LTS "Focal Fossa" - Release amd64 (20210824)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
Package: linux (not installed)
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
Tags: focal uec-images
Uname: Linux 5.4.0-124-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: N/A
_MarkForUpload: True
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A05
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA05:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc.
---
ProblemType: Bug
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
DistroRelease: Ubuntu 20.04
InstallationDate: Installed on 2021-02-10 (564 days ago)
InstallationMedia: Ubuntu-Server 20.04.2 LTS "Focal Fossa" - Release amd64 (20210201.2)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
Package: linux (not installed)
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
Tags: focal uec-images
Uname: Linux 5.4.0-124-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: N/A
_MarkForUpload: True
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A02
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA02:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc.
---
ProblemType: Bug
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:50 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:50 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
DistroRelease: Ubuntu 20.04
InstallationDate: Installed on 2021-02-10 (564 days ago)
InstallationMedia: Ubuntu-Server 20.04.2 LTS "Focal Fossa" - Release amd64 (20210201.2)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
Package: linux (not installed)
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
Tags: focal uec-images
Uname: Linux 5.4.0-124-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: N/A
_MarkForUpload: True
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A07
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA07:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc. |
We run multiple machines that have the purpose of acting as a data repository for Veeam backup. Each machine has 2x 256G XFS volumes that use heavy reflinking. This works as intended, for one issue: our xfs freeze once a week, a few minutes after midnight on Monday nights. We can only do a reboot to get the servers working again. Then it works for a week again.
We have been interacting with support from Veeam, but this looks like some XFS kernel issue, or a race condition between the Veeam process and XFS.
ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: linux-image-5.4.0-124-generic 5.4.0-124.140
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
Uname: Linux 5.4.0-124-generic x86_64
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
Date: Mon Aug 29 00:34:28 2022
InstallationDate: Installed on 2021-02-10 (564 days ago)
InstallationMedia: Ubuntu-Server 20.04.2 LTS "Focal Fossa" - Release amd64 (20210201.2)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A02
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA02:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc. |
|
| 2022-08-29 00:55:29 |
Olafur Helgi Haraldsson |
description |
We run multiple machines that have the purpose of acting as a data repository for Veeam backup. Each machine has 2x 256G XFS volumes that use heavy reflinking. This works as intended, for one issue: our xfs freeze once a week, a few minutes after midnight on Monday nights. We can only do a reboot to get the servers working again. Then it works for a week again.
We have been interacting with support from Veeam, but this looks like some XFS kernel issue, or a race condition between the Veeam process and XFS.
ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: linux-image-5.4.0-124-generic 5.4.0-124.140
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
Uname: Linux 5.4.0-124-generic x86_64
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
Date: Mon Aug 29 00:34:28 2022
InstallationDate: Installed on 2021-02-10 (564 days ago)
InstallationMedia: Ubuntu-Server 20.04.2 LTS "Focal Fossa" - Release amd64 (20210201.2)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A02
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA02:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc. |
We run multiple machines that have the purpose of acting as a data repository for Veeam backup. Each machine has 2x 256G XFS volumes that use heavy reflinking. This works as intended, for one issue: our xfs freeze once a week, a few minutes after midnight on Monday nights. We can only do a reboot to get the servers working again. Then it works for a week again.
We have been interacting with support from Veeam, but this looks like some XFS kernel issue, or a race condition between the Veeam process and XFS.
[562599.834734] INFO: task kworker/6:3:3534660 blocked for more than 120 seconds.
[562599.834794] Not tainted 5.4.0-124-generic #140-Ubuntu
[562599.834832] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[562599.834891] kworker/6:3 D 0 3534660 2 0x80004000
[562599.834962] Workqueue: xfs-conv/dm-3 xfs_end_io [xfs]
[562599.834964] Call Trace:
[562599.834975] __schedule+0x2e3/0x740
[562599.835026] ? xfs_log_ticket_put+0x1f/0x30 [xfs]
[562599.835031] ? kmem_cache_free+0x288/0x2b0
[562599.835035] schedule+0x42/0xb0
[562599.835041] rwsem_down_write_slowpath+0x244/0x4d0
[562599.835045] ? __switch_to_asm+0x40/0x70
[562599.835088] ? __xfs_setfilesize+0x31/0x110 [xfs]
[562599.835091] down_write+0x41/0x50
[562599.835137] xfs_ilock+0x7b/0x110 [xfs]
[562599.835178] __xfs_setfilesize+0x31/0x110 [xfs]
[562599.835181] ? __switch_to_asm+0x40/0x70
[562599.835220] xfs_setfilesize_ioend+0x49/0x60 [xfs]
[562599.835257] xfs_end_ioend+0x7b/0x1b0 [xfs]
[562599.835260] ? __switch_to_asm+0x34/0x70
[562599.835298] xfs_end_io+0xb1/0xe0 [xfs]
[562599.835304] process_one_work+0x1eb/0x3b0
[562599.835309] worker_thread+0x4d/0x400
[562599.835312] kthread+0x104/0x140
[562599.835316] ? process_one_work+0x3b0/0x3b0
[562599.835319] ? kthread_park+0x90/0x90
[562599.835322] ret_from_fork+0x35/0x40
[562599.836171] INFO: task veeamagent:3674754 blocked for more than 120 seconds.
[562599.836219] Not tainted 5.4.0-124-generic #140-Ubuntu
[562599.836261] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[562599.836318] veeamagent D 0 3674754 3674651 0x00004000
[562599.836321] Call Trace:
[562599.836326] __schedule+0x2e3/0x740
[562599.836330] schedule+0x42/0xb0
[562599.836333] schedule_timeout+0x10e/0x160
[562599.836335] ? schedule_timeout+0x10e/0x160
[562599.836337] __down+0x82/0xd0
[562599.836341] ? wake_up_q+0x70/0x70
[562599.836383] ? xfs_buf_find.isra.0+0x3bf/0x610 [xfs]
[562599.836391] down+0x47/0x60
[562599.836434] xfs_buf_lock+0x37/0xf0 [xfs]
[562599.836476] xfs_buf_find.isra.0+0x3bf/0x610 [xfs]
[562599.836518] xfs_buf_get_map+0x43/0x2b0 [xfs]
[562599.836557] xfs_buf_read_map+0x2f/0x1d0 [xfs]
[562599.836610] xfs_trans_read_buf_map+0xca/0x350 [xfs]
[562599.836643] xfs_read_agf+0x97/0x130 [xfs]
[562599.836664] ? update_load_avg+0x7c/0x670
[562599.836700] xfs_alloc_read_agf+0x45/0x1a0 [xfs]
[562599.836753] ? xfs_alloc_space_available+0x4a/0xf0 [xfs]
[562599.836783] xfs_alloc_fix_freelist+0x41e/0x4e0 [xfs]
[562599.836786] ? check_preempt_curr+0x7a/0x90
[562599.836788] ? ttwu_do_wakeup+0x1e/0x150
[562599.836793] ? radix_tree_lookup+0xd/0x10
[562599.836836] ? xfs_perag_get+0x2d/0xb0 [xfs]
[562599.836839] ? radix_tree_lookup+0xd/0x10
[562599.836877] ? xfs_perag_get+0x2d/0xb0 [xfs]
[562599.836906] xfs_alloc_vextent+0x19f/0x550 [xfs]
[562599.836938] xfs_bmap_btalloc+0x57b/0x940 [xfs]
[562599.836973] xfs_bmap_alloc+0x34/0x40 [xfs]
[562599.837004] xfs_bmapi_allocate+0xdc/0x2d0 [xfs]
[562599.837043] xfs_bmapi_convert_delalloc+0x26f/0x4b0 [xfs]
[562599.837084] xfs_map_blocks+0x15a/0x3f0 [xfs]
[562599.837123] xfs_do_writepage+0x118/0x420 [xfs]
[562599.837130] write_cache_pages+0x1ae/0x4b0
[562599.837171] ? xfs_vm_releasepage+0x80/0x80 [xfs]
[562599.837209] xfs_vm_writepages+0x6a/0xa0 [xfs]
[562599.837215] do_writepages+0x43/0xd0
[562599.837221] __filemap_fdatawrite_range+0xd5/0x110
[562599.837226] file_write_and_wait_range+0x74/0xc0
[562599.837268] xfs_file_fsync+0x5d/0x230 [xfs]
[562599.837274] ? __do_sys_newfstat+0x61/0x70
[562599.837281] vfs_fsync_range+0x49/0x80
[562599.837284] do_fsync+0x3d/0x70
[562599.837288] __x64_sys_fsync+0x14/0x20
[562599.837295] do_syscall_64+0x57/0x190
[562599.837298] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[562599.837301] RIP: 0033:0x7f4092abd93b
[562599.837308] Code: Bad RIP value.
[562599.837310] RSP: 002b:00007f4066ffc850 EFLAGS: 00000293 ORIG_RAX: 000000000000004a
[562599.837313] RAX: ffffffffffffffda RBX: 00007f4066ffd650 RCX: 00007f4092abd93b
[562599.837315] RDX: 00007f4066ffc800 RSI: 00007f4066ffc800 RDI: 0000000000000b06
[562599.837316] RBP: 00007f40440329e0 R08: 0000000000000000 R09: 00007f40580008d0
[562599.837317] R10: 00007f40580008d0 R11: 0000000000000293 R12: 00007f4044029940
[562599.837319] R13: 0000000000000b06 R14: 0000000000000000 R15: 00007f4044017b70
ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: linux-image-5.4.0-124-generic 5.4.0-124.140
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
Uname: Linux 5.4.0-124-generic x86_64
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
Date: Mon Aug 29 00:34:28 2022
InstallationDate: Installed on 2021-02-10 (564 days ago)
InstallationMedia: Ubuntu-Server 20.04.2 LTS "Focal Fossa" - Release amd64 (20210201.2)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A02
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA02:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc. |
|
| 2022-08-29 01:00:09 |
Ubuntu Kernel Bot |
linux (Ubuntu): status |
New |
Confirmed |
|
| 2022-08-29 01:08:07 |
Olafur Helgi Haraldsson |
description |
We run multiple machines that have the purpose of acting as a data repository for Veeam backup. Each machine has 2x 256G XFS volumes that use heavy reflinking. This works as intended, for one issue: our xfs freeze once a week, a few minutes after midnight on Monday nights. We can only do a reboot to get the servers working again. Then it works for a week again.
We have been interacting with support from Veeam, but this looks like some XFS kernel issue, or a race condition between the Veeam process and XFS.
[562599.834734] INFO: task kworker/6:3:3534660 blocked for more than 120 seconds.
[562599.834794] Not tainted 5.4.0-124-generic #140-Ubuntu
[562599.834832] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[562599.834891] kworker/6:3 D 0 3534660 2 0x80004000
[562599.834962] Workqueue: xfs-conv/dm-3 xfs_end_io [xfs]
[562599.834964] Call Trace:
[562599.834975] __schedule+0x2e3/0x740
[562599.835026] ? xfs_log_ticket_put+0x1f/0x30 [xfs]
[562599.835031] ? kmem_cache_free+0x288/0x2b0
[562599.835035] schedule+0x42/0xb0
[562599.835041] rwsem_down_write_slowpath+0x244/0x4d0
[562599.835045] ? __switch_to_asm+0x40/0x70
[562599.835088] ? __xfs_setfilesize+0x31/0x110 [xfs]
[562599.835091] down_write+0x41/0x50
[562599.835137] xfs_ilock+0x7b/0x110 [xfs]
[562599.835178] __xfs_setfilesize+0x31/0x110 [xfs]
[562599.835181] ? __switch_to_asm+0x40/0x70
[562599.835220] xfs_setfilesize_ioend+0x49/0x60 [xfs]
[562599.835257] xfs_end_ioend+0x7b/0x1b0 [xfs]
[562599.835260] ? __switch_to_asm+0x34/0x70
[562599.835298] xfs_end_io+0xb1/0xe0 [xfs]
[562599.835304] process_one_work+0x1eb/0x3b0
[562599.835309] worker_thread+0x4d/0x400
[562599.835312] kthread+0x104/0x140
[562599.835316] ? process_one_work+0x3b0/0x3b0
[562599.835319] ? kthread_park+0x90/0x90
[562599.835322] ret_from_fork+0x35/0x40
[562599.836171] INFO: task veeamagent:3674754 blocked for more than 120 seconds.
[562599.836219] Not tainted 5.4.0-124-generic #140-Ubuntu
[562599.836261] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[562599.836318] veeamagent D 0 3674754 3674651 0x00004000
[562599.836321] Call Trace:
[562599.836326] __schedule+0x2e3/0x740
[562599.836330] schedule+0x42/0xb0
[562599.836333] schedule_timeout+0x10e/0x160
[562599.836335] ? schedule_timeout+0x10e/0x160
[562599.836337] __down+0x82/0xd0
[562599.836341] ? wake_up_q+0x70/0x70
[562599.836383] ? xfs_buf_find.isra.0+0x3bf/0x610 [xfs]
[562599.836391] down+0x47/0x60
[562599.836434] xfs_buf_lock+0x37/0xf0 [xfs]
[562599.836476] xfs_buf_find.isra.0+0x3bf/0x610 [xfs]
[562599.836518] xfs_buf_get_map+0x43/0x2b0 [xfs]
[562599.836557] xfs_buf_read_map+0x2f/0x1d0 [xfs]
[562599.836610] xfs_trans_read_buf_map+0xca/0x350 [xfs]
[562599.836643] xfs_read_agf+0x97/0x130 [xfs]
[562599.836664] ? update_load_avg+0x7c/0x670
[562599.836700] xfs_alloc_read_agf+0x45/0x1a0 [xfs]
[562599.836753] ? xfs_alloc_space_available+0x4a/0xf0 [xfs]
[562599.836783] xfs_alloc_fix_freelist+0x41e/0x4e0 [xfs]
[562599.836786] ? check_preempt_curr+0x7a/0x90
[562599.836788] ? ttwu_do_wakeup+0x1e/0x150
[562599.836793] ? radix_tree_lookup+0xd/0x10
[562599.836836] ? xfs_perag_get+0x2d/0xb0 [xfs]
[562599.836839] ? radix_tree_lookup+0xd/0x10
[562599.836877] ? xfs_perag_get+0x2d/0xb0 [xfs]
[562599.836906] xfs_alloc_vextent+0x19f/0x550 [xfs]
[562599.836938] xfs_bmap_btalloc+0x57b/0x940 [xfs]
[562599.836973] xfs_bmap_alloc+0x34/0x40 [xfs]
[562599.837004] xfs_bmapi_allocate+0xdc/0x2d0 [xfs]
[562599.837043] xfs_bmapi_convert_delalloc+0x26f/0x4b0 [xfs]
[562599.837084] xfs_map_blocks+0x15a/0x3f0 [xfs]
[562599.837123] xfs_do_writepage+0x118/0x420 [xfs]
[562599.837130] write_cache_pages+0x1ae/0x4b0
[562599.837171] ? xfs_vm_releasepage+0x80/0x80 [xfs]
[562599.837209] xfs_vm_writepages+0x6a/0xa0 [xfs]
[562599.837215] do_writepages+0x43/0xd0
[562599.837221] __filemap_fdatawrite_range+0xd5/0x110
[562599.837226] file_write_and_wait_range+0x74/0xc0
[562599.837268] xfs_file_fsync+0x5d/0x230 [xfs]
[562599.837274] ? __do_sys_newfstat+0x61/0x70
[562599.837281] vfs_fsync_range+0x49/0x80
[562599.837284] do_fsync+0x3d/0x70
[562599.837288] __x64_sys_fsync+0x14/0x20
[562599.837295] do_syscall_64+0x57/0x190
[562599.837298] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[562599.837301] RIP: 0033:0x7f4092abd93b
[562599.837308] Code: Bad RIP value.
[562599.837310] RSP: 002b:00007f4066ffc850 EFLAGS: 00000293 ORIG_RAX: 000000000000004a
[562599.837313] RAX: ffffffffffffffda RBX: 00007f4066ffd650 RCX: 00007f4092abd93b
[562599.837315] RDX: 00007f4066ffc800 RSI: 00007f4066ffc800 RDI: 0000000000000b06
[562599.837316] RBP: 00007f40440329e0 R08: 0000000000000000 R09: 00007f40580008d0
[562599.837317] R10: 00007f40580008d0 R11: 0000000000000293 R12: 00007f4044029940
[562599.837319] R13: 0000000000000b06 R14: 0000000000000000 R15: 00007f4044017b70
ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: linux-image-5.4.0-124-generic 5.4.0-124.140
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
Uname: Linux 5.4.0-124-generic x86_64
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
Date: Mon Aug 29 00:34:28 2022
InstallationDate: Installed on 2021-02-10 (564 days ago)
InstallationMedia: Ubuntu-Server 20.04.2 LTS "Focal Fossa" - Release amd64 (20210201.2)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A02
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA02:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc. |
We run multiple machines that have the purpose of acting as a data repository for Veeam backup. Each machine has 2x 256G XFS volumes that use heavy reflinking. This works as intended, for one issue: our xfs freeze once a week, a few minutes after midnight on Monday nights. We can only do a reboot to get the servers working again. Then it works for a week again.
We have been interacting with support from Veeam, but this looks like some XFS kernel issue, or a race condition between the Veeam process and XFS.
[562599.834734] INFO: task kworker/6:3:3534660 blocked for more than 120 seconds.
[562599.834794] Not tainted 5.4.0-124-generic #140-Ubuntu
[562599.834832] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[562599.834891] kworker/6:3 D 0 3534660 2 0x80004000
[562599.834962] Workqueue: xfs-conv/dm-3 xfs_end_io [xfs]
[562599.834964] Call Trace:
[562599.834975] __schedule+0x2e3/0x740
[562599.835026] ? xfs_log_ticket_put+0x1f/0x30 [xfs]
[562599.835031] ? kmem_cache_free+0x288/0x2b0
[562599.835035] schedule+0x42/0xb0
[562599.835041] rwsem_down_write_slowpath+0x244/0x4d0
[562599.835045] ? __switch_to_asm+0x40/0x70
[562599.835088] ? __xfs_setfilesize+0x31/0x110 [xfs]
[562599.835091] down_write+0x41/0x50
[562599.835137] xfs_ilock+0x7b/0x110 [xfs]
[562599.835178] __xfs_setfilesize+0x31/0x110 [xfs]
[562599.835181] ? __switch_to_asm+0x40/0x70
[562599.835220] xfs_setfilesize_ioend+0x49/0x60 [xfs]
[562599.835257] xfs_end_ioend+0x7b/0x1b0 [xfs]
[562599.835260] ? __switch_to_asm+0x34/0x70
[562599.835298] xfs_end_io+0xb1/0xe0 [xfs]
[562599.835304] process_one_work+0x1eb/0x3b0
[562599.835309] worker_thread+0x4d/0x400
[562599.835312] kthread+0x104/0x140
[562599.835316] ? process_one_work+0x3b0/0x3b0
[562599.835319] ? kthread_park+0x90/0x90
[562599.835322] ret_from_fork+0x35/0x40
[562599.836171] INFO: task veeamagent:3674754 blocked for more than 120 seconds.
[562599.836219] Not tainted 5.4.0-124-generic #140-Ubuntu
[562599.836261] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[562599.836318] veeamagent D 0 3674754 3674651 0x00004000
[562599.836321] Call Trace:
[562599.836326] __schedule+0x2e3/0x740
[562599.836330] schedule+0x42/0xb0
[562599.836333] schedule_timeout+0x10e/0x160
[562599.836335] ? schedule_timeout+0x10e/0x160
[562599.836337] __down+0x82/0xd0
[562599.836341] ? wake_up_q+0x70/0x70
[562599.836383] ? xfs_buf_find.isra.0+0x3bf/0x610 [xfs]
[562599.836391] down+0x47/0x60
[562599.836434] xfs_buf_lock+0x37/0xf0 [xfs]
[562599.836476] xfs_buf_find.isra.0+0x3bf/0x610 [xfs]
[562599.836518] xfs_buf_get_map+0x43/0x2b0 [xfs]
[562599.836557] xfs_buf_read_map+0x2f/0x1d0 [xfs]
[562599.836610] xfs_trans_read_buf_map+0xca/0x350 [xfs]
[562599.836643] xfs_read_agf+0x97/0x130 [xfs]
[562599.836664] ? update_load_avg+0x7c/0x670
[562599.836700] xfs_alloc_read_agf+0x45/0x1a0 [xfs]
[562599.836753] ? xfs_alloc_space_available+0x4a/0xf0 [xfs]
[562599.836783] xfs_alloc_fix_freelist+0x41e/0x4e0 [xfs]
[562599.836786] ? check_preempt_curr+0x7a/0x90
[562599.836788] ? ttwu_do_wakeup+0x1e/0x150
[562599.836793] ? radix_tree_lookup+0xd/0x10
[562599.836836] ? xfs_perag_get+0x2d/0xb0 [xfs]
[562599.836839] ? radix_tree_lookup+0xd/0x10
[562599.836877] ? xfs_perag_get+0x2d/0xb0 [xfs]
[562599.836906] xfs_alloc_vextent+0x19f/0x550 [xfs]
[562599.836938] xfs_bmap_btalloc+0x57b/0x940 [xfs]
[562599.836973] xfs_bmap_alloc+0x34/0x40 [xfs]
[562599.837004] xfs_bmapi_allocate+0xdc/0x2d0 [xfs]
[562599.837043] xfs_bmapi_convert_delalloc+0x26f/0x4b0 [xfs]
[562599.837084] xfs_map_blocks+0x15a/0x3f0 [xfs]
[562599.837123] xfs_do_writepage+0x118/0x420 [xfs]
[562599.837130] write_cache_pages+0x1ae/0x4b0
[562599.837171] ? xfs_vm_releasepage+0x80/0x80 [xfs]
[562599.837209] xfs_vm_writepages+0x6a/0xa0 [xfs]
[562599.837215] do_writepages+0x43/0xd0
[562599.837221] __filemap_fdatawrite_range+0xd5/0x110
[562599.837226] file_write_and_wait_range+0x74/0xc0
[562599.837268] xfs_file_fsync+0x5d/0x230 [xfs]
[562599.837274] ? __do_sys_newfstat+0x61/0x70
[562599.837281] vfs_fsync_range+0x49/0x80
[562599.837284] do_fsync+0x3d/0x70
[562599.837288] __x64_sys_fsync+0x14/0x20
[562599.837295] do_syscall_64+0x57/0x190
[562599.837298] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[562599.837301] RIP: 0033:0x7f4092abd93b
[562599.837308] Code: Bad RIP value.
[562599.837310] RSP: 002b:00007f4066ffc850 EFLAGS: 00000293 ORIG_RAX: 000000000000004a
[562599.837313] RAX: ffffffffffffffda RBX: 00007f4066ffd650 RCX: 00007f4092abd93b
[562599.837315] RDX: 00007f4066ffc800 RSI: 00007f4066ffc800 RDI: 0000000000000b06
[562599.837316] RBP: 00007f40440329e0 R08: 0000000000000000 R09: 00007f40580008d0
[562599.837317] R10: 00007f40580008d0 R11: 0000000000000293 R12: 00007f4044029940
[562599.837319] R13: 0000000000000b06 R14: 0000000000000000 R15: 00007f4044017b70
ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: linux-image-5.4.0-124-generic 5.4.0-124.140
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
Uname: Linux 5.4.0-124-generic x86_64
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
Date: Mon Aug 29 00:34:28 2022
InstallationDate: Installed on 2021-02-10 (564 days ago)
InstallationMedia: Ubuntu-Server 20.04.2 LTS "Focal Fossa" - Release amd64 (20210201.2)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A02
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA02:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc.
---
ProblemType: Bug
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
DistroRelease: Ubuntu 20.04
InstallationDate: Installed on 2021-09-16 (346 days ago)
InstallationMedia: Ubuntu-Server 20.04.3 LTS "Focal Fossa" - Release amd64 (20210824)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
Package: linux (not installed)
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
Tags: focal uec-images
Uname: Linux 5.4.0-124-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: N/A
_MarkForUpload: True
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A05
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA05:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc. |
|
| 2022-08-29 01:08:08 |
Olafur Helgi Haraldsson |
attachment added |
|
CRDA.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612246/+files/CRDA.txt |
|
| 2022-08-29 01:08:09 |
Olafur Helgi Haraldsson |
attachment added |
|
CurrentDmesg.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612247/+files/CurrentDmesg.txt |
|
| 2022-08-29 01:08:11 |
Olafur Helgi Haraldsson |
attachment added |
|
Lspci.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612248/+files/Lspci.txt |
|
| 2022-08-29 01:08:12 |
Olafur Helgi Haraldsson |
attachment added |
|
Lspci-vt.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612249/+files/Lspci-vt.txt |
|
| 2022-08-29 01:08:13 |
Olafur Helgi Haraldsson |
attachment added |
|
Lsusb.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612250/+files/Lsusb.txt |
|
| 2022-08-29 01:08:14 |
Olafur Helgi Haraldsson |
attachment added |
|
Lsusb-t.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612251/+files/Lsusb-t.txt |
|
| 2022-08-29 01:08:16 |
Olafur Helgi Haraldsson |
attachment added |
|
Lsusb-v.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612252/+files/Lsusb-v.txt |
|
| 2022-08-29 01:08:17 |
Olafur Helgi Haraldsson |
attachment added |
|
ProcCpuinfo.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612253/+files/ProcCpuinfo.txt |
|
| 2022-08-29 01:08:18 |
Olafur Helgi Haraldsson |
attachment added |
|
ProcCpuinfoMinimal.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612254/+files/ProcCpuinfoMinimal.txt |
|
| 2022-08-29 01:08:19 |
Olafur Helgi Haraldsson |
attachment added |
|
ProcInterrupts.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612255/+files/ProcInterrupts.txt |
|
| 2022-08-29 01:08:21 |
Olafur Helgi Haraldsson |
attachment added |
|
ProcModules.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612256/+files/ProcModules.txt |
|
| 2022-08-29 01:08:22 |
Olafur Helgi Haraldsson |
attachment added |
|
UdevDb.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612257/+files/UdevDb.txt |
|
| 2022-08-29 01:08:24 |
Olafur Helgi Haraldsson |
attachment added |
|
WifiSyslog.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612258/+files/WifiSyslog.txt |
|
| 2022-08-29 01:08:26 |
Olafur Helgi Haraldsson |
attachment added |
|
acpidump.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612259/+files/acpidump.txt |
|
| 2022-08-29 01:09:32 |
Olafur Helgi Haraldsson |
description |
We run multiple machines that have the purpose of acting as a data repository for Veeam backup. Each machine has 2x 256G XFS volumes that use heavy reflinking. This works as intended, for one issue: our xfs freeze once a week, a few minutes after midnight on Monday nights. We can only do a reboot to get the servers working again. Then it works for a week again.
We have been interacting with support from Veeam, but this looks like some XFS kernel issue, or a race condition between the Veeam process and XFS.
[562599.834734] INFO: task kworker/6:3:3534660 blocked for more than 120 seconds.
[562599.834794] Not tainted 5.4.0-124-generic #140-Ubuntu
[562599.834832] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[562599.834891] kworker/6:3 D 0 3534660 2 0x80004000
[562599.834962] Workqueue: xfs-conv/dm-3 xfs_end_io [xfs]
[562599.834964] Call Trace:
[562599.834975] __schedule+0x2e3/0x740
[562599.835026] ? xfs_log_ticket_put+0x1f/0x30 [xfs]
[562599.835031] ? kmem_cache_free+0x288/0x2b0
[562599.835035] schedule+0x42/0xb0
[562599.835041] rwsem_down_write_slowpath+0x244/0x4d0
[562599.835045] ? __switch_to_asm+0x40/0x70
[562599.835088] ? __xfs_setfilesize+0x31/0x110 [xfs]
[562599.835091] down_write+0x41/0x50
[562599.835137] xfs_ilock+0x7b/0x110 [xfs]
[562599.835178] __xfs_setfilesize+0x31/0x110 [xfs]
[562599.835181] ? __switch_to_asm+0x40/0x70
[562599.835220] xfs_setfilesize_ioend+0x49/0x60 [xfs]
[562599.835257] xfs_end_ioend+0x7b/0x1b0 [xfs]
[562599.835260] ? __switch_to_asm+0x34/0x70
[562599.835298] xfs_end_io+0xb1/0xe0 [xfs]
[562599.835304] process_one_work+0x1eb/0x3b0
[562599.835309] worker_thread+0x4d/0x400
[562599.835312] kthread+0x104/0x140
[562599.835316] ? process_one_work+0x3b0/0x3b0
[562599.835319] ? kthread_park+0x90/0x90
[562599.835322] ret_from_fork+0x35/0x40
[562599.836171] INFO: task veeamagent:3674754 blocked for more than 120 seconds.
[562599.836219] Not tainted 5.4.0-124-generic #140-Ubuntu
[562599.836261] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[562599.836318] veeamagent D 0 3674754 3674651 0x00004000
[562599.836321] Call Trace:
[562599.836326] __schedule+0x2e3/0x740
[562599.836330] schedule+0x42/0xb0
[562599.836333] schedule_timeout+0x10e/0x160
[562599.836335] ? schedule_timeout+0x10e/0x160
[562599.836337] __down+0x82/0xd0
[562599.836341] ? wake_up_q+0x70/0x70
[562599.836383] ? xfs_buf_find.isra.0+0x3bf/0x610 [xfs]
[562599.836391] down+0x47/0x60
[562599.836434] xfs_buf_lock+0x37/0xf0 [xfs]
[562599.836476] xfs_buf_find.isra.0+0x3bf/0x610 [xfs]
[562599.836518] xfs_buf_get_map+0x43/0x2b0 [xfs]
[562599.836557] xfs_buf_read_map+0x2f/0x1d0 [xfs]
[562599.836610] xfs_trans_read_buf_map+0xca/0x350 [xfs]
[562599.836643] xfs_read_agf+0x97/0x130 [xfs]
[562599.836664] ? update_load_avg+0x7c/0x670
[562599.836700] xfs_alloc_read_agf+0x45/0x1a0 [xfs]
[562599.836753] ? xfs_alloc_space_available+0x4a/0xf0 [xfs]
[562599.836783] xfs_alloc_fix_freelist+0x41e/0x4e0 [xfs]
[562599.836786] ? check_preempt_curr+0x7a/0x90
[562599.836788] ? ttwu_do_wakeup+0x1e/0x150
[562599.836793] ? radix_tree_lookup+0xd/0x10
[562599.836836] ? xfs_perag_get+0x2d/0xb0 [xfs]
[562599.836839] ? radix_tree_lookup+0xd/0x10
[562599.836877] ? xfs_perag_get+0x2d/0xb0 [xfs]
[562599.836906] xfs_alloc_vextent+0x19f/0x550 [xfs]
[562599.836938] xfs_bmap_btalloc+0x57b/0x940 [xfs]
[562599.836973] xfs_bmap_alloc+0x34/0x40 [xfs]
[562599.837004] xfs_bmapi_allocate+0xdc/0x2d0 [xfs]
[562599.837043] xfs_bmapi_convert_delalloc+0x26f/0x4b0 [xfs]
[562599.837084] xfs_map_blocks+0x15a/0x3f0 [xfs]
[562599.837123] xfs_do_writepage+0x118/0x420 [xfs]
[562599.837130] write_cache_pages+0x1ae/0x4b0
[562599.837171] ? xfs_vm_releasepage+0x80/0x80 [xfs]
[562599.837209] xfs_vm_writepages+0x6a/0xa0 [xfs]
[562599.837215] do_writepages+0x43/0xd0
[562599.837221] __filemap_fdatawrite_range+0xd5/0x110
[562599.837226] file_write_and_wait_range+0x74/0xc0
[562599.837268] xfs_file_fsync+0x5d/0x230 [xfs]
[562599.837274] ? __do_sys_newfstat+0x61/0x70
[562599.837281] vfs_fsync_range+0x49/0x80
[562599.837284] do_fsync+0x3d/0x70
[562599.837288] __x64_sys_fsync+0x14/0x20
[562599.837295] do_syscall_64+0x57/0x190
[562599.837298] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[562599.837301] RIP: 0033:0x7f4092abd93b
[562599.837308] Code: Bad RIP value.
[562599.837310] RSP: 002b:00007f4066ffc850 EFLAGS: 00000293 ORIG_RAX: 000000000000004a
[562599.837313] RAX: ffffffffffffffda RBX: 00007f4066ffd650 RCX: 00007f4092abd93b
[562599.837315] RDX: 00007f4066ffc800 RSI: 00007f4066ffc800 RDI: 0000000000000b06
[562599.837316] RBP: 00007f40440329e0 R08: 0000000000000000 R09: 00007f40580008d0
[562599.837317] R10: 00007f40580008d0 R11: 0000000000000293 R12: 00007f4044029940
[562599.837319] R13: 0000000000000b06 R14: 0000000000000000 R15: 00007f4044017b70
ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: linux-image-5.4.0-124-generic 5.4.0-124.140
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
Uname: Linux 5.4.0-124-generic x86_64
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
Date: Mon Aug 29 00:34:28 2022
InstallationDate: Installed on 2021-02-10 (564 days ago)
InstallationMedia: Ubuntu-Server 20.04.2 LTS "Focal Fossa" - Release amd64 (20210201.2)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A02
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA02:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc.
---
ProblemType: Bug
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
DistroRelease: Ubuntu 20.04
InstallationDate: Installed on 2021-09-16 (346 days ago)
InstallationMedia: Ubuntu-Server 20.04.3 LTS "Focal Fossa" - Release amd64 (20210824)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
Package: linux (not installed)
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
Tags: focal uec-images
Uname: Linux 5.4.0-124-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: N/A
_MarkForUpload: True
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A05
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA05:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc. |
We run multiple machines that have the purpose of acting as a data repository for Veeam backup. Each machine has 2x 256G XFS volumes that use heavy reflinking. This works as intended, for one issue: our xfs freeze once a week, a few minutes after midnight on Monday nights. We can only do a reboot to get the servers working again. Then it works for a week again.
We have been interacting with support from Veeam, but this looks like some XFS kernel issue, or a race condition between the Veeam process and XFS.
[562599.834734] INFO: task kworker/6:3:3534660 blocked for more than 120 seconds.
[562599.834794] Not tainted 5.4.0-124-generic #140-Ubuntu
[562599.834832] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[562599.834891] kworker/6:3 D 0 3534660 2 0x80004000
[562599.834962] Workqueue: xfs-conv/dm-3 xfs_end_io [xfs]
[562599.834964] Call Trace:
[562599.834975] __schedule+0x2e3/0x740
[562599.835026] ? xfs_log_ticket_put+0x1f/0x30 [xfs]
[562599.835031] ? kmem_cache_free+0x288/0x2b0
[562599.835035] schedule+0x42/0xb0
[562599.835041] rwsem_down_write_slowpath+0x244/0x4d0
[562599.835045] ? __switch_to_asm+0x40/0x70
[562599.835088] ? __xfs_setfilesize+0x31/0x110 [xfs]
[562599.835091] down_write+0x41/0x50
[562599.835137] xfs_ilock+0x7b/0x110 [xfs]
[562599.835178] __xfs_setfilesize+0x31/0x110 [xfs]
[562599.835181] ? __switch_to_asm+0x40/0x70
[562599.835220] xfs_setfilesize_ioend+0x49/0x60 [xfs]
[562599.835257] xfs_end_ioend+0x7b/0x1b0 [xfs]
[562599.835260] ? __switch_to_asm+0x34/0x70
[562599.835298] xfs_end_io+0xb1/0xe0 [xfs]
[562599.835304] process_one_work+0x1eb/0x3b0
[562599.835309] worker_thread+0x4d/0x400
[562599.835312] kthread+0x104/0x140
[562599.835316] ? process_one_work+0x3b0/0x3b0
[562599.835319] ? kthread_park+0x90/0x90
[562599.835322] ret_from_fork+0x35/0x40
[562599.836171] INFO: task veeamagent:3674754 blocked for more than 120 seconds.
[562599.836219] Not tainted 5.4.0-124-generic #140-Ubuntu
[562599.836261] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[562599.836318] veeamagent D 0 3674754 3674651 0x00004000
[562599.836321] Call Trace:
[562599.836326] __schedule+0x2e3/0x740
[562599.836330] schedule+0x42/0xb0
[562599.836333] schedule_timeout+0x10e/0x160
[562599.836335] ? schedule_timeout+0x10e/0x160
[562599.836337] __down+0x82/0xd0
[562599.836341] ? wake_up_q+0x70/0x70
[562599.836383] ? xfs_buf_find.isra.0+0x3bf/0x610 [xfs]
[562599.836391] down+0x47/0x60
[562599.836434] xfs_buf_lock+0x37/0xf0 [xfs]
[562599.836476] xfs_buf_find.isra.0+0x3bf/0x610 [xfs]
[562599.836518] xfs_buf_get_map+0x43/0x2b0 [xfs]
[562599.836557] xfs_buf_read_map+0x2f/0x1d0 [xfs]
[562599.836610] xfs_trans_read_buf_map+0xca/0x350 [xfs]
[562599.836643] xfs_read_agf+0x97/0x130 [xfs]
[562599.836664] ? update_load_avg+0x7c/0x670
[562599.836700] xfs_alloc_read_agf+0x45/0x1a0 [xfs]
[562599.836753] ? xfs_alloc_space_available+0x4a/0xf0 [xfs]
[562599.836783] xfs_alloc_fix_freelist+0x41e/0x4e0 [xfs]
[562599.836786] ? check_preempt_curr+0x7a/0x90
[562599.836788] ? ttwu_do_wakeup+0x1e/0x150
[562599.836793] ? radix_tree_lookup+0xd/0x10
[562599.836836] ? xfs_perag_get+0x2d/0xb0 [xfs]
[562599.836839] ? radix_tree_lookup+0xd/0x10
[562599.836877] ? xfs_perag_get+0x2d/0xb0 [xfs]
[562599.836906] xfs_alloc_vextent+0x19f/0x550 [xfs]
[562599.836938] xfs_bmap_btalloc+0x57b/0x940 [xfs]
[562599.836973] xfs_bmap_alloc+0x34/0x40 [xfs]
[562599.837004] xfs_bmapi_allocate+0xdc/0x2d0 [xfs]
[562599.837043] xfs_bmapi_convert_delalloc+0x26f/0x4b0 [xfs]
[562599.837084] xfs_map_blocks+0x15a/0x3f0 [xfs]
[562599.837123] xfs_do_writepage+0x118/0x420 [xfs]
[562599.837130] write_cache_pages+0x1ae/0x4b0
[562599.837171] ? xfs_vm_releasepage+0x80/0x80 [xfs]
[562599.837209] xfs_vm_writepages+0x6a/0xa0 [xfs]
[562599.837215] do_writepages+0x43/0xd0
[562599.837221] __filemap_fdatawrite_range+0xd5/0x110
[562599.837226] file_write_and_wait_range+0x74/0xc0
[562599.837268] xfs_file_fsync+0x5d/0x230 [xfs]
[562599.837274] ? __do_sys_newfstat+0x61/0x70
[562599.837281] vfs_fsync_range+0x49/0x80
[562599.837284] do_fsync+0x3d/0x70
[562599.837288] __x64_sys_fsync+0x14/0x20
[562599.837295] do_syscall_64+0x57/0x190
[562599.837298] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[562599.837301] RIP: 0033:0x7f4092abd93b
[562599.837308] Code: Bad RIP value.
[562599.837310] RSP: 002b:00007f4066ffc850 EFLAGS: 00000293 ORIG_RAX: 000000000000004a
[562599.837313] RAX: ffffffffffffffda RBX: 00007f4066ffd650 RCX: 00007f4092abd93b
[562599.837315] RDX: 00007f4066ffc800 RSI: 00007f4066ffc800 RDI: 0000000000000b06
[562599.837316] RBP: 00007f40440329e0 R08: 0000000000000000 R09: 00007f40580008d0
[562599.837317] R10: 00007f40580008d0 R11: 0000000000000293 R12: 00007f4044029940
[562599.837319] R13: 0000000000000b06 R14: 0000000000000000 R15: 00007f4044017b70
ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: linux-image-5.4.0-124-generic 5.4.0-124.140
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
Uname: Linux 5.4.0-124-generic x86_64
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
Date: Mon Aug 29 00:34:28 2022
InstallationDate: Installed on 2021-02-10 (564 days ago)
InstallationMedia: Ubuntu-Server 20.04.2 LTS "Focal Fossa" - Release amd64 (20210201.2)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A02
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA02:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc.
---
ProblemType: Bug
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
DistroRelease: Ubuntu 20.04
InstallationDate: Installed on 2021-09-16 (346 days ago)
InstallationMedia: Ubuntu-Server 20.04.3 LTS "Focal Fossa" - Release amd64 (20210824)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
Package: linux (not installed)
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
Tags: focal uec-images
Uname: Linux 5.4.0-124-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: N/A
_MarkForUpload: True
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A05
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA05:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc.
---
ProblemType: Bug
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Aug 22 11:51 seq
crw-rw---- 1 root audio 116, 33 Aug 22 11:51 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.24
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperMD5CheckResult: pass
DistroRelease: Ubuntu 20.04
InstallationDate: Installed on 2021-09-16 (346 days ago)
InstallationMedia: Ubuntu-Server 20.04.3 LTS "Focal Fossa" - Release amd64 (20210824)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
MachineType: Dell Inc. PowerEdge M630
Package: linux (not installed)
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-124-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
RelatedPackageVersions:
linux-restricted-modules-5.4.0-124-generic N/A
linux-backports-modules-5.4.0-124-generic N/A
linux-firmware 1.187.33
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
Tags: focal uec-images
Uname: Linux 5.4.0-124-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: N/A
_MarkForUpload: True
dmi.bios.date: 07/05/2022
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.15.0
dmi.board.name: 0R10KJ
dmi.board.vendor: Dell Inc.
dmi.board.version: A05
dmi.chassis.type: 25
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: PowerEdge M1000e
dmi.modalias: dmi:bvnDellInc.:bvr2.15.0:bd07/05/2022:svnDellInc.:pnPowerEdgeM630:pvr:rvnDellInc.:rn0R10KJ:rvrA05:cvnDellInc.:ct25:cvrPowerEdgeM1000e:
dmi.product.name: PowerEdge M630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge M630
dmi.sys.vendor: Dell Inc. |
|
| 2022-08-29 01:09:32 |
Olafur Helgi Haraldsson |
attachment added |
|
CRDA.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612260/+files/CRDA.txt |
|
| 2022-08-29 01:09:34 |
Olafur Helgi Haraldsson |
attachment added |
|
CurrentDmesg.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612261/+files/CurrentDmesg.txt |
|
| 2022-08-29 01:09:35 |
Olafur Helgi Haraldsson |
attachment added |
|
Lspci.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612262/+files/Lspci.txt |
|
| 2022-08-29 01:09:36 |
Olafur Helgi Haraldsson |
attachment added |
|
Lspci-vt.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612263/+files/Lspci-vt.txt |
|
| 2022-08-29 01:09:38 |
Olafur Helgi Haraldsson |
attachment added |
|
Lsusb.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612264/+files/Lsusb.txt |
|
| 2022-08-29 01:09:39 |
Olafur Helgi Haraldsson |
attachment added |
|
Lsusb-t.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612265/+files/Lsusb-t.txt |
|
| 2022-08-29 01:09:41 |
Olafur Helgi Haraldsson |
attachment added |
|
Lsusb-v.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612266/+files/Lsusb-v.txt |
|
| 2022-08-29 01:09:42 |
Olafur Helgi Haraldsson |
attachment added |
|
ProcCpuinfo.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612267/+files/ProcCpuinfo.txt |
|
| 2022-08-29 01:09:43 |
Olafur Helgi Haraldsson |
attachment added |
|
ProcCpuinfoMinimal.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612268/+files/ProcCpuinfoMinimal.txt |
|
| 2022-08-29 01:09:45 |
Olafur Helgi Haraldsson |
attachment added |
|
ProcInterrupts.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612269/+files/ProcInterrupts.txt |
|
| 2022-08-29 01:09:46 |
Olafur Helgi Haraldsson |
attachment added |
|
ProcModules.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612270/+files/ProcModules.txt |
|
| 2022-08-29 01:09:47 |
Olafur Helgi Haraldsson |
attachment added |
|
UdevDb.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612271/+files/UdevDb.txt |
|
| 2022-08-29 01:09:48 |
Olafur Helgi Haraldsson |
attachment added |
|
WifiSyslog.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612272/+files/WifiSyslog.txt |
|
| 2022-08-29 01:09:50 |
Olafur Helgi Haraldsson |
attachment added |
|
acpidump.txt https://bugs.launchpad.net/bugs/1987997/+attachment/5612273/+files/acpidump.txt |
|
