Comment 2 for bug 1947677

I have since found that executing in Terminal, the command
  𝚡𝚖𝚘𝚍𝚖𝚊𝚙 -𝚎 "𝚌𝚕𝚎𝚊𝚛 𝙻𝚘𝚌𝚔"
is a temporary workaround to fix this behaviour; however, I would definitely think this issue should be fixed so it works out-of-the-box for new users.
It's a very problematic behaviour for one's primary backspace to be also toggling caps lock, affecting daily typing.