CIFS module is not included in 6.2.0-1016

[Jean-Francois Simoneau]

Between 6.2.0-1015 and 6.2.0-1016, the CIFS module was moved from fs/cifs/* to fs/smb/client/*, fs/smb/common/* and fs/smb/server/*. The inclusion list (root/debian.azure-6.2/control.d/azure.inclusion-list) was not updated for this change, so the module is not included in the linux-modules-6.2.0-1026-azure package.

WORKAROUND: cifs.ko is provided in the linux-modules-extra-azure (as noted in comment #7); a 'sudo apt install linux-modules-extra-azure' will add it in.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Kalen Emhof (kemhof)]

I can confirm this:

6.2.0-1015 Kernel
~# uname -a
Linux pi-geosvrd01 6.2.0-1015-azure #15~22.04.1-Ubuntu SMP Fri Oct 6 13:20:44 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

~# lsmod | grep cifs
cifs 1363968 2
cifs_arc4 16384 1 cifs
cifs_md4 16384 1 cifs
fscache 380928 1 cifs

~# ls -lh /lib/modules/6.2.0-1015-azure/kernel/fs/cifs
-rw-r--r-- 1 root root 2.8M Oct 6 07:20 cifs.ko

~# ls -lh /lib/modules/6.2.0-1015-azure/kernel/fs
total 128K
drwxr-xr-x 2 root root 4.0K Oct 31 09:00 9p
drwxr-xr-x 2 root root 4.0K Oct 31 09:00 autofs
-rw-r--r-- 1 root root 38K Oct 6 07:20 binfmt_misc.ko
drwxr-xr-x 2 root root 4.0K Oct 31 09:00 btrfs
drwxr-xr-x 2 root root 4.0K Oct 31 09:00 cachefiles
drwxr-xr-x 2 root root 4.0K Oct 31 09:00 ceph
drwxr-xr-x 2 root root 4.0K Oct 31 09:00 cifs
drwxr-xr-x 2 root root 4.0K Oct 31 09:00 dlm
drwxr-xr-x 2 root root 4.0K Oct 31 09:00 fat
drwxr-xr-x 2 root root 4.0K Oct 31 09:00 fscache
drwxr-xr-x 2 root root 4.0K Oct 31 09:00 fuse
drwxr-xr-x 2 root root 4.0K Oct 31 09:00 isofs
drwxr-xr-x 2 root root 4.0K Oct 31 09:00 lockd
drwxr-xr-x 2 root root 4.0K Oct 31 09:00 netfs
drwxr-xr-x 5 root root 4.0K Oct 31 09:00 nfs
drwxr-xr-x 2 root root 4.0K Oct 31 09:00 nfs_common
drwxr-xr-x 2 root root 4.0K Oct 31 09:00 nfsd
drwxr-xr-x 2 root root 4.0K Oct 31 09:00 nls
drwxr-xr-x 2 root root 4.0K Oct 31 09:00 overlayfs
drwxr-xr-x 2 root root 4.0K Oct 31 09:00 smbfs_common
drwxr-xr-x 2 root root 4.0K Oct 31 09:00 udf
drwxr-xr-x 2 root root 4.0K Oct 31 09:00 ufs
drwxr-xr-x 2 root root 4.0K Oct 31 09:00 xfs

After installing the 6.2.0-1016 kernel and rebooting, system is unable to mount CIFS filesystems:
~# uname -a
Linux pi-geosvrd01 6.2.0-1016-azure #16~22.04.1-Ubuntu SMP Tue Oct 10 17:11:51 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

~# modprobe cifs
modprobe: FATAL: Module cifs not found in directory /lib/modules/6.2.0-1016-azure

~#
mount error: cifs filesystem not supported by the system
mount error(19): No such device

~# ls -lh /lib/modules/6.2.0-1016-azure/kernel/fs/
total 120K
drwxr-xr-x 2 root root 4.0K Oct 31 09:36 9p
drwxr-xr-x 2 root root 4.0K Oct 31 09:36 autofs
-rw-r--r-- 1 root root 38K Oct 10 11:03 binfmt_misc.ko
drwxr-xr-x 2 root root 4.0K Oct 31 09:36 btrfs
drwxr-xr-x 2 root root 4.0K Oct 31 09:36 cachefiles
drwxr-xr-x 2 root root 4.0K Oct 31 09:36 ceph
drwxr-xr-x 2 root root 4.0K Oct 31 09:36 dlm
drwxr-xr-x 2 root root 4.0K Oct 31 09:36 fat
drwxr-xr-x 2 root root 4.0K Oct 31 09:36 fscache
drwxr-xr-x 2 root root 4.0K Oct 31 09:36 fuse
drwxr-xr-x 2 root root 4.0K Oct 31 09:36 isofs
drwxr-xr-x 2 root root 4.0K Oct 31 09:36 lockd
drwxr-xr-x 2 root root 4.0K Oct 31 09:36 netfs
drwxr-xr-x 5 root root 4.0K Oct 31 09:36 nfs
drwxr-xr-x 2 root root 4.0K Oct 31 09:36 nfs_common
drwxr-xr-x 2 root root 4.0K Oct 31 09:36 nfsd
drwxr-xr-x 2 root root 4.0K Oct 31 09:36 nls
drwxr-xr-x 2 root root 4.0K Oct 31 09:36 overlayfs
drwxr-xr-x 2 root root 4.0K Oct 31 09:36 udf
drwxr-xr-x 2 root root 4.0K Oct 31 09:36 ufs
drwxr-xr-x 2 root root 4.0K Oct 31 09:36 xfs

[Luke Dearden (lukedearden)]

Ran into this issue just now.

Looks like new VMs in Azure North Europe are now being deployed with 6.2.0-1016-azure whereas earlier today it was 6.2.0-1015-azure

Our cifs mounts obviously fail now.

I was able to rollback using the following

## Install older kernel
sudo apt install linux-image-6.2.0-1015-azure

## Remove newer kernel (select NO when asked)
sudo apt remove linux-image-6.2.0-1016-azure

## Reboot
sudo reboot

[Kevin Klinke (keklinke)]

We are impacted by this as well, and all our new VMs that were spun up in Azure were failing due to mount issues. We were able to mitigate by adding the following to our cloud-init:

bootcmd:
  - apt update
  - apt-mark hold linux-image-6.2.0-1016-azure
  - apt-mark hold linux-cloud-tools-6.2.0-1016-azure
  - apt-mark hold linux-headers-6.2.0-1016-azure
  - apt-mark hold linux-tools-6.2.0-1016-azure

This is obviously a temporary fix and if this is not addressed in the next release we'll regress again and it also prevents us from getting any bug fixes or security fixes in this most recent version. This feels like a huge regression in this kernel version and I hope it gets fixed and we prevent this from happening in future releases.

[Jean-Francois Simoneau (jfsimoneau)]

On our side, we install the linux-modules-extra-azure package as a workaround. CIFS gets installed as part of this package on 6.2.0-1016 but we have more modules we don't need.

[C de-Avillez (hggdh2)]

given cifs.ko (and associated kmods) is now in modules-extra... was this an oversight, or is it where it will be from now on?

Added a WORKAROUND to the description, thank you Jean-Francois.

[Luke Dearden (lukedearden)]

FYI I created a ticket with Azure Support and they say "We have found the bug and Microsoft is working with the Canonical side. waiting for the update from canonical side"

[Dean Attewell (eitsop)]

Related But.. and fixed in this https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2042546