CIFS mount error: iocharset utf8 not found

[ayanamist]

After i apt upgrade from linux-image-5.4.0-89-generic to linux-image-5.4.0-90-generic, mount.cifs failed with dmesg "CIFS VFS: CIFS mount error: iocharset utf8 not found", and i rollback it and it restores.
---
ProblemType: Bug
AlsaDevices:
 total 0
 crw-rw---- 1 root audio 116, 1 Nov 24 20:16 seq
 crw-rw---- 1 root audio 116, 33 Nov 24 20:16 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.21
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
DistroRelease: Ubuntu 20.04
InstallationDate: Installed on 2021-08-07 (109 days ago)
InstallationMedia: Ubuntu-Server 20.04.2 LTS "Focal Fossa" - Release amd64 (20210201.2)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
Lspci:

Lspci-vt: -[0000:00]-
Lsusb: Error: command ['lsusb'] failed with exit code 1:
Lsusb-t:

Lsusb-v: Error: command ['lsusb', '-v'] failed with exit code 1:
MachineType: Microsoft Corporation Virtual Machine
Package: linux (not installed)
PciMultimedia:

ProcEnviron:
 TERM=screen-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/zsh
ProcFB: 0 hyperv_fb
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-5.4.0-89-generic root=UUID=fd942b77-19c9-4959-a01c-c8e3f7b583b6 ro
ProcVersionSignature: Ubuntu 5.4.0-89.100-generic 5.4.143
RelatedPackageVersions:
 linux-restricted-modules-5.4.0-89-generic N/A
 linux-backports-modules-5.4.0-89-generic N/A
 linux-firmware N/A
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
Tags: focal uec-images
Uname: Linux 5.4.0-89-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: adm cdrom dip lxd plugdev sudo
_MarkForUpload: True
dmi.bios.date: 12/17/2019
dmi.bios.vendor: Microsoft Corporation
dmi.bios.version: Hyper-V UEFI Release v4.0
dmi.board.asset.tag: None
dmi.board.name: Virtual Machine
dmi.board.vendor: Microsoft Corporation
dmi.board.version: Hyper-V UEFI Release v4.0
dmi.chassis.asset.tag: 1425-2986-4285-1154-1589-7450-41
dmi.chassis.type: 3
dmi.chassis.vendor: Microsoft Corporation
dmi.chassis.version: Hyper-V UEFI Release v4.0
dmi.modalias: dmi:bvnMicrosoftCorporation:bvrHyper-VUEFIReleasev4.0:bd12/17/2019:svnMicrosoftCorporation:pnVirtualMachine:pvrHyper-VUEFIReleasev4.0:rvnMicrosoftCorporation:rnVirtualMachine:rvrHyper-VUEFIReleasev4.0:cvnMicrosoftCorporation:ct3:cvrHyper-VUEFIReleasev4.0:
dmi.product.family: Virtual Machine
dmi.product.name: Virtual Machine
dmi.product.sku: None
dmi.product.version: Hyper-V UEFI Release v4.0
dmi.sys.vendor: Microsoft Corporation

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1952094

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Tim Gardner (timg-tpi)]

There are 2 patches against fs/cifs in that interval:

git log --pretty=oneline Ubuntu-5.4.0-89.100..Ubuntu-5.4.0-90.101 -- fs/cifs/
bc3d24f536d5e79ba474f192280a18d060a4c33f cifs: fix wrong release in sess_alloc_buffer() failed path
3d5631a27ec4767ac80dbf553f9ae501b18e07e3 CIFS: Fix a potencially linear read overflow

The second patch touches fs/cifs/cifs_unicode.c which makes it a strong candidate for having caused this regression. Please try the following kernel which has that patch reverted:

wget https://kernel.ubuntu.com/~rtg/lp1952094/amd64/linux-image-unsigned-5.4.0-91-generic_5.4.0-91.102~lp1952094.1_amd64.deb
wget https://kernel.ubuntu.com/~rtg/lp1952094/amd64/linux-modules-5.4.0-91-generic_5.4.0-91.102~lp1952094.1_amd64.deb
wget https://kernel.ubuntu.com/~rtg/lp1952094/amd64/linux-modules-extra-5.4.0-91-generic_5.4.0-91.102~lp1952094.1_amd64.deb
sudo dpkg -i linux*.deb
sudo reboot

[ayanamist (ayanamist)]

@timg-tpi Yes, the new version works fine now.

[Tim Gardner (timg-tpi)]

Patch to revert that commit has been submitted: https://lists.ubuntu.com/archives/kernel-team/2021-November/126144.html

[Tim Gardner (timg-tpi)]

@ayanamist - Can you provide some more details about the full mount path and server type (Windows or Linux and versions) ? Upstream is skeptical that the patch I've reverted is involved in a connection attempt, though empirical evidence suggests otherwise.

[Thadeu Lima de Souza Cascardo (cascardo)]

Maybe this is the case of a missing linux-modules-extra installed? nls_utf8 module seems to be shipped on linux-modules-extra, and the missing package would explain the failure.

Cascardo.

[ayanamist (ayanamist)]

I have installed them by installed linux-generic, and today i install the latest official 5.4.0-91 kernel, but it is even worse now.

mount command:
mount -t cifs //192.168.58.13/share /mnt/share/ -o username=guest,password=,noperm,noacl,file_mode=0644,iocharset=utf8,vers=2.1

Server is a windows server 2019 without any changes during the issues.

In 5.4.0-91, it seems that cifs is not loaded automatically, and after i modprobe cifs, it still fails with "mount error(13): Permission denied" with dmesg
[ 189.086484] CIFS: Attempting to mount //192.168.58.13/share
[ 189.090361] CIFS VFS: could not allocate crypto cmac(aes)
[ 189.093222] CIFS VFS: could not allocate crypto cmac(aes)
[ 189.099002] CIFS VFS: could not allocate crypto md4
[ 189.099789] Status code returned 0xc000006d STATUS_LOGON_FAILURE
[ 189.099796] CIFS VFS: \\192.168.58.13 Send error in SessSetup = -13
[ 189.099923] CIFS VFS: cifs_mount failed w/return code = -13

If i rollback to 5.4.0-89, everything works fine.

> dpkg -l |fgrep -- -89
ii linux-headers-5.4.0-89 5.4.0-89.100 all Header files related to Linux kernel version 5.4.0
ii linux-headers-5.4.0-89-generic 5.4.0-89.100 amd64 Linux kernel headers for version 5.4.0 on 64 bit x86 SMP
ii linux-image-5.4.0-89-generic 5.4.0-89.100 amd64 Signed kernel image generic
ii linux-modules-5.4.0-89-generic 5.4.0-89.100 amd64 Linux kernel extra modules for version 5.4.0 on 64 bit x86 SMP
ii linux-modules-extra-5.4.0-89-generic 5.4.0-89.100 amd64 Linux kernel extra modules for version 5.4.0 on 64 bit x86 SMP
ii linux-tools-5.4.0-89 5.4.0-89.100 amd64 Linux kernel version specific tools for version 5.4.0-89
ii linux-tools-5.4.0-89-generic 5.4.0-89.100 amd64 Linux kernel version specific tools for version 5.4.0-89

> dpkg -l |fgrep -- -91
ii linux-headers-5.4.0-91 5.4.0-91.102 all Header files related to Linux kernel version 5.4.0
ii linux-headers-5.4.0-91-generic 5.4.0-91.102 amd64 Linux kernel headers for version 5.4.0 on 64 bit x86 SMP
ii linux-image-5.4.0-91-generic 5.4.0-91.102 amd64 Signed kernel image generic
ii linux-modules-5.4.0-91-generic 5.4.0-91.102 amd64 Linux kernel extra modules for version 5.4.0 on 64 bit x86 SMP
ii linux-modules-extra-5.4.0-91-generic 5.4.0-91.102 amd64 Linux kernel extra modules for version 5.4.0 on 64 bit x86 SMP
ii linux-tools-5.4.0-91 5.4.0-91.102 amd64 Linux kernel version specific tools for version 5.4.0-91
ii linux-tools-5.4.0-91-generic 5.4.0-91.102 amd64 Linux kernel version specific tools for version 5.4.0-91
ii linux-tools-common 5.4.0-91.102 all Linux kernel versio...

[Tim Gardner (timg-tpi)]

@ayanamist - When you say 'roll back to 5.4.0-89', do you mean you interrupt the grub timeout and select a kernel to boot ?

[Tim Gardner (timg-tpi)]

@ayanamist - also, can you attach a complete dmesg of the failing kernel after the mount attempt ?

[ayanamist (ayanamist)]

Yes, i use below in my /etc/default/grub and switch kernel during boot
```
GRUB_SAVEDEFAULT=true
GRUB_DEFAULT=saved
GRUB_TIMEOUT=10
```

The dmesg for both kernel is attached. Note that in 5.4.0-91-generic kernel, i modprobe cifs and mount again manually after boot, so cifs loaded log is much later(about 200 seconds)

[Tim Gardner (timg-tpi)]

@ayanamist - I am somewhat baffled. Upstream does not think the patch I reverted is the root cause, so lets get back to basics and ensure this isn't something local to your system. This kernel is vanilla 5.4.0-90.101, but built by me. If this fails then I think we're going to have to do a full bisect. That will likely take 6-8 kernel versions to narrow in on the offending commit.

wget https://kernel.ubuntu.com/~rtg/lp1952094/5.4.0-90.101-lp1952094.2/amd64/linux-image-unsigned-5.4.0-90-generic_5.4.0-90.101~lp1952094.2_amd64.deb
wget https://kernel.ubuntu.com/~rtg/lp1952094/5.4.0-90.101-lp1952094.2/amd64/linux-modules-5.4.0-90-generic_5.4.0-90.101~lp1952094.2_amd64.deb
wget https://kernel.ubuntu.com/~rtg/lp1952094/5.4.0-90.101-lp1952094.2/amd64/linux-modules-extra-5.4.0-90-generic_5.4.0-90.101~lp1952094.2_amd64.deb
sudo dpkg -i *.deb

Please attach the full dmesg from this kerenel.

[ayanamist (ayanamist)]

Fine, i'm glad to help.

This one works fine.

[Tim Gardner (timg-tpi)]

@ayanamist - Given your last result I have to say that this does not look like a kernel problem.

This should be the kernel that will be released on Jan 4. Please give it a try.

wget https://launchpad.net/~canonical-kernel-team/+archive/ubuntu/ppa/+files/linux-image-unsigned-5.4.0-92-generic_5.4.0-92.103_amd64.deb
wget https://launchpad.net/~canonical-kernel-team/+archive/ubuntu/ppa/+files/linux-modules-5.4.0-92-generic_5.4.0-92.103_amd64.deb
wget https://launchpad.net/~canonical-kernel-team/+archive/ubuntu/ppa/+files/linux-modules-extra-5.4.0-92-generic_5.4.0-92.103_amd64.deb
sudo dpkg -i *.deb

[ayanamist (ayanamist)]

This 5.4.0-92 kernel works fine. Thank you.

[Tim Gardner (timg-tpi)]

@ayanamist - I don't know what to do at this point. Its clearly not a kernel issue. As such, I'm marking the kernel bug invalid.