[20.04] Keyboard layout changes during installation before typing username/password

[Dag Bjerkeli]

During a fresh install of Ubuntu 20.04, selecting Norwegian keyboard is provided and keys are responding correctly at that stage. But later when entering user information the keyboard setting is wrong.

It looks like it have fallen back to English keyboard-layout.
I failed to log in after my first install, so a new attempt I tried to write some special letters in the name field, and noticed that I got the wrong characters for the key.

The install is done in VMware workstation 15.

In the attached screendump, the characters (';[":{) after my name should have been (æøåÆØÅ) that is norwegian characters.

Problem analysis
================

* ubiquity installs open-vm-tools.
* open-vm-tools calls "udevadm trigger" in it's postinst script (line 8).
* udevadm triggers all udev rules which includes all input devices.
* gdm-x-session reads /etc/default/keyboard and sets the keyboard layout.
* gnome-shell sets the keyboard layout as well.

ubiquity already sets /etc/default/keyboard, but does not change the keyboard layout in the GNOME session.

Proposed solution
=================

Let ubiquity also configure the GNOME session to use the selected keyboard layout.

ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: ubiquity (not installed)
ProcVersionSignature: Ubuntu 5.4.0-26.30-generic 5.4.30
Uname: Linux 5.4.0-26-generic x86_64
ApportVersion: 2.20.11-0ubuntu27
Architecture: amd64
CasperMD5CheckResult: skip
CurrentDesktop: ubuntu:GNOME
Date: Sat Apr 25 19:27:41 2020
InstallCmdLine: file=/cdrom/preseed/ubuntu.seed initrd=/casper/initrd quiet splash --- maybe-ubiquity
InstallationDate: Installed on 2020-04-25 (0 days ago)
InstallationMedia: Ubuntu 20.04 LTS "Focal Fossa" - Release amd64 (20200423)
SourcePackage: ubiquity
Symptom: installer
UpgradeStatus: No upgrade log present (probably fresh install)

[Jean-Do Veuve (jeando)]

I installed a fresh Ubuntu 20.04 LTS on an Oracle VM. During the installation, I choosed “french language” and “Belgian keyboard”. Later, the installer asked me to type my name and password and THERE the keyboard is in Qwerty with no support of the numeric pad (Qwerty 88)

I checked it with two different install to be sure of the bug. And with a German keyboard, same problem.

After reboot, it returns back in Keyb BE as defined.

Potential login problem :

If your name doesn’t contain any letter as “z-q-a-w or m” nor a numeric AND neither does your password, you are fine. But if only your password does, you’re fried because you cannot detect the problem during the install and at the next restart : bam ! “your password is invalid”.

The work around is to type the password with the “Qwerty 88” design in your head, a design easy to find on internet.

[Simos Xenitellis  (simosx)]

I suggest to change the title of this report to something like
"[20.04] Keyboard layout not enabled immediately during installation when typing username/password".

It looks like an issue of not activating the selected keyboard layout during the installation, in order to type your username/password/computer name.

I have not tested for this and I do not recall an issue, probably because I use the English US layout as the primary layout.

[Coeur Noir (coeur-noir)]

I faced the same problem while installing Ubuntu 18.04 and 20.04.

No VM here. And French for locale.

It's OK if I first « try without installing » in my language, then later launch the installer.
In this scenario, no problem with keyboard at the username/password stage, it remains french/azerty.

It's not OK when I go straight to « install ». Ubiquity stops using french/azerty keyboard at username/password stage.
And once there, there is no way to go back or abort safely.

By chance rough power off had no dramatic consequence.

Reproducible any time. And also with UbuntuBudgie ( and I guess any flavor using Ubiquity ).

[Dimitri John Ledkov (xnox)]

Did you try groovy, and is this issue fixed there for you? We have recently had keyboard improvement there.

Groovy images are available from http://cdimage.ubuntu.com/daily-live/pending/

[Jean-Do Veuve (jeando)]

I checked the Groovy version proposed by @Dimitri John Ledkov (xnox) ab

NO progress: still a US in place of the chosen localized keyboard at the introduction of the name+password.

Progress: 102 touch in place of 88 touch keyboard (numeric keyboard worked) at that moment

I checked the Groovy version proposed by @Dimitri John Ledkov with the same configuration : VM Oracle. I went directly to the minimal install, without trying the OS before as described by @coeur-noir, and the "AZERTY Belgian Keyb" configuration is recognize as such but at the moment of the introduction of name+password. There, it is still a 102 US Keyboard.

[Jean-Do Veuve (jeando)]

Another problem appears while installing Groovy as of now, FR speaking version, on an Oracle VM machine, the installer cannot install the bootloader (FR: "chargeur d'amorçage") with the following message (in french) "Désolé, une erreur s'est produite. Il n'a pas été possible d'installer le chargeur d'amorçage à l'emplacement indiqué." and propose to install it on another media.

I tried it twice on 2 different virtual disks but on the same physical Solid-State Disk

[Jean-Do Veuve (jeando)]

Worse : in an english speaking install with a AZERTY Belgian keyboard, we are back to the 88 touch US design (no numeric pad) at the same moment : introduction of name+password. I insist : 88 US Keyb. It is different in a french speaking install were the Keyboard is a 102 touch US Keyb (numeric pad work).

And for the third time, it still cannot install the boot loader.

[Dimitri John Ledkov (xnox)]

@jeando

The expectation is not that the keyboard is set to localised one.

But that the alternative layouts are available in the keyboard indicator in the top right, and one can switch to a localised layout. Or using Super+Space to change layouts.

Also this bug report is about norwegian language, which is very different to azerty keyboards.

Please open separate new bug report about french / french belgian / azerty keybaords. Please keep this bug report about nb_NO only.

For other bug reports about installation failures, please complete them in english and open separate bug reports.

It's not helpful to use one issue to report lots of different things.

[Jean-Do Veuve (jeando)]

@xnox

Okay, I double check on base of your remarks and I got deeper in the tests and eliminate the other bugs in this report. Thank you for helping to reformulate and motivate me to goes on :-)

I am more precise now and I think I may say it is exactly the same bug in all 3 languages I checked. The selected keyboard layout is NOT recognize at any point during the install process, period. I check 3 keyboards layout (and languages): Norwegian, French and Deutsch (aka German) on my VM.

In other words: at no point of the install but the moment of the selection itself, you get the keyboard layout you selected. You got a 82 US layout in the first version of 20.04, with "Groovie" it's a 102 US layout (with numeric keypad). A little progress, so.

The original bug description said: "During a fresh install of Ubuntu 20.04, selecting Norwegian keyboard is provided and keys are responding correctly at that stage. But later when entering user information the keyboard setting is wrong." Word for word, the same remark applies except, as I said before, that the problem appears sooner, too.

It may help to trace the bug that some problem exist in US layout too: the keyboard layout fall to 88 touch during the install process.

[Jean-Do Veuve (jeando)]

This bug is a regression: with older version of Ubuntu, such as 18.04 LTS, during the install process, you get your selected keyboard layout all along during the install process.

As I said before, this bug makes that, if you don't realize that the keyboard layout fell to US, it might happen that at the place where you choose your password, where the letters appears as "*", you use one or more characters where US and your selected keyboard layouts differs without realizing it. In that case, at the reboot (where your keyboard selection is correctly activated), you will receive the message "invalid password" on and on. If you don't realize where the bug is, it is possible that you choose to reinstall Ubuntu again and again until you choose a password where the 2 keyboard layout don't differ.

[Yaron (sh-yaron)]

I think that there should be a way to select layout during installation and even make a list of languages that should be optional after the installation.

In Hebrew we have a different problem where Hebrew layout is automatically selected and when we're trying to type the password for logging in after installation we're being kicked out because the layout defaults to Hebrew.

I think emphasizing the layout selection along this process will be more beneficial for users that need to switch to their layout or switch to English, maybe having some tooltip explaining how to switch keyboard layout or some accessible button at the bottom or top of the screen (just a suggestion, it's probably not that good).

[Brian Murray (brian-murray)]

There is an updated version of ubiquity (20.04.15.1) in the -proposed pocket which may address this issue. Could you please test it?

[Gunnar Hjalmarsson (gunnarhj)]

On 2020-06-25 18:24, Brian Murray wrote:
> There is an updated version of ubiquity (20.04.15.1) in the
> -proposed pocket which may address this issue. Could you please test
> it?

How would you go about doing that? Is there an ISO with that ubiquity version somewhere?

[Dag Bjerkeli (dag-e)]

I would also need an ISO to start with.

Den 25.06.2020 18:55, skrev Gunnar Hjalmarsson:
> On 2020-06-25 18:24, Brian Murray wrote:
>> There is an updated version of ubiquity (20.04.15.1) in the
>> -proposed pocket which may address this issue. Could you please test
>> it?
> How would you go about doing that? Is there an ISO with that ubiquity
> version somewhere?
>

[Jean-Do Veuve (jeando)]

@gunnarhj @dag-e
@xnox gave the adress for ISO "Groovy images are available from http://cdimage.ubuntu.com/daily-live/pending/ "

[Gunnar Hjalmarsson (gunnarhj)]

That's groovy, and Brian talked about focal-proposed. But sure, if this bug is present also with the latest groovy daily build, I suppose we can conclude that the version in focal-proposed is not sufficient to fix this bug.

[Jean-Do Veuve (jeando)]

I checked with the last Groovy I found 45 minutes ago and the bug is still there in French-Belgian language/keyboard layout.

[Jean-Do Veuve (jeando)]

And still ther in German (Deutsch) also

[Gunnar Hjalmarsson (gunnarhj)]

I started an install using the 2020-06-25 groovy ISO, selected Swedish, but found that the effective keyboard layout in the window for entering name, password, etc. was English (US).

[Steve Langasek (vorlon)]

Gunnar, since there are multiple different code paths related to keyboard configuration, could you please clarify exactly which way you selected Swedish with the focal daily image?
- Did you boot under BIOS or UEFI?
- If BIOS, did you select the keyboard from the bootloader (purple screen)?
- Which boot option did you use?

[Gunnar Hjalmarsson (gunnarhj)]

On 2020-06-29 21:16, Steve Langasek wrote:
> Gunnar, since there are multiple different code paths related to
> keyboard configuration, could you please clarify exactly which way
> you selected Swedish with the focal daily image?

Please note that it was the groovy daily image (from 2020-06-25) I used.

> - Did you boot under BIOS or UEFI?

UEFI

> - Which boot option did you use?

Install Ubuntu (safe graphics)

But just now I tried again, and made an observation which may be important.

When achieving the result in comment #20 I did:

* Selected the Swedish language on the Welcome window

* When the window for keyboard layout selection showed up, the Swedish layout was pre-selected, so I just clicked "Continue".

* In the window for entering name, password etc. the English (US) layout was effective.

However - this is today's experiment:

* Selected the Swedish language on the Welcome window

* When the window for keyboard layout selection showed up, the Swedish layout was pre-selected. But I still changed to some other layout, and then changed back to the Swedish one before clicking "Continue".

* In the window for entering name, password etc. the Swedish layout was effective.

This indicates that the issue with English (US) always being effective when entering name, password etc. only happens if you accept the pre-selected layout without touching other layouts, and not when you explicitly select a layout of your choice.

[Gunnar Hjalmarsson (gunnarhj)]

I think I answered the questions Steve asked...

[Dag Bjerkeli (dag-e)]

I've tried groovy-desktop-amd64 downloaded yesterday (July 3rd).

Attempt 1
-----------------------------------------------------------------------
- Did you boot under BIOS or UEFI?
BIOS

- If BIOS, did you select the keyboard from the bootloader (purple screen)?
Not sure what screen this should be.

- Which boot option did you use?
Ubuntu

**Welcome**
   -> Selecting Norsk Bokmål (Norwegian)
   -> Installer Ubuntu (Install Ubuntu)

**Tastaturutforming** (Keyboard layout?)
Testing the keyboard in the input field -> OK Norwegian characters appears as they should.

Selecting defaults for remaining screens until **Hvem er du** (Who are you?)
Entering Norwegian keys in the name field gives wrong characters.

Attempt 2
-----------------------------------------------------------------------
Same as Attempt 1. But at the window **Tastaturutforming** I selected
US keyboard
Then Continue
Then Back from the Oppdateringer og annen programvare (Updates and other software?) window.
Selecting Norwegian as keyboard layout.

Selecting defaults for remaining screens until **Hvem er du** (Who are you?)
Entering Norwegian keys in the name field gives CORRECT characters.

so I can confirm Gunnar Hjalmarssons observations.

[Jessica Van den Hende (jessicavdh)]

I've found how to overcome this bug.

I needed Belgian Azerty and made the selection Belgian - Belgian, and testing it always showed Azerty. Yet when I needed to fill in my username and password it always changed back to Qwerty..
I tried about 8 different combinations to try and get Azerty.. Yet always the same thing occured.

So now the solution;
When you want for example Belgian Azerty, then you have to select Belgian in the first option, then select Belgian in the second option AND then go back to the first option and double click your selection. It then goes to the next screen. You select back, and go to the previous screen. Now you see it has the selection Belgian - Belgian. If you then go to the user and password screen it gives Azerty like it should do.

If you select the first one, and then double click it (without selecting the second one, EVEN if it is already selected automaticly) then you'll see that when you return to the previous screen it jumped back to the automaticly selected first option (for me this was English) and that's why I always got Qwerty instead of Azerty.

Hope this helps. :)

[Łukasz Zemczak (sil2100)]

Can anyone check if this is still happening on the daily groovy images?

http://cdimage.ubuntu.com/ubuntu/daily-live/current/groovy-desktop-amd64.iso

I have tried reproducing the issue on a VM with both the groovy and focal dailies with no success. That being said, as it was a quick VM check I did boot only via BIOS, but it at least feels that the boot method should not matter in this case (will double check later anyway). I tried various outlined combinations, both from #22 and #25.

I would appreciate if someone who was encountering it in the past could try reproducing it with current groovy images. Thank you!

[Gunnar Hjalmarsson (gunnarhj)]

Hi Łukasz!

Checked with today's daily image for groovy, and could no longer reproduce the issue. Tested both on a VM and by doing a complete install.

Tested with yesterday's daily image for focal too (only VM, though), and couldn't reproduce the issue with that either.

[Jean-Do Veuve (jeando)]

@sil2100

Problem solved :-)

I check the daily-live I download from the indicated address and made exactly the same install (Oracle VM running on a Ubuntu 18.04, vanilla config of the VM) that failed since 2020-04-27 and this time, at the moment of introducing the new login and password, the keyboard was in Azerty Belgian aka Belgian Keyboard as asked.

To be sure, I'll check with some other languages and if it failed, I'll write it here but I'm 99% confident that all will be OK.

You are welcome.

[Jean-Do Veuve (jeando)]

Hi Lukasz

I checked also in Dutch (Nederlands) and it is also solved.

Thanks

[Dag Bjerkeli (dag-e)]

I've checked with Norwegian keyboard and settings as I originally did. And I did not experience the problem described earlier.

I've tested both UTF-8 characters and keyboard layout.

So this appears to be solved.

Good woork!

/dag

[Łukasz Zemczak (sil2100)]

Thanks a lot everyone! Let's close off this bug in that case.

[Shlomi Shehebar (shlomzions)]

After installing Ubuntu 20.04.1 LTS - Hebrew keyboard appeared to exist and super+space seems to switch between them however... no hebrew comes out...:(

Removing the keyboard from the Input Sources and immediately adding it back , solves the Issue ... however it returns after boot or after loading some browser or terminal...

ANNOYING!

[Gunnar Hjalmarsson (gunnarhj)]

@Shlomi: Your issue sounds more like bug #1890875.

[Gunnar Hjalmarsson (gunnarhj)]

Please disregard my latest comment; that bug is about bionic.

[Gunnar Hjalmarsson (gunnarhj)]

Unfortunately this bug seems to have resurrected in hirsute. It's actually even worse than before: When testing in a VM it uses *always* the English (US) keyboard layout in the window for entering name and password, irrespective of my previous choices of display language and keyboard layout.

Probably I should have submitted a new bug, but since a few users who were attentive to the issue previously are subscribed to this one, it would be great if others could test the latest hirsute build in this respect. Maybe it's just me...

[Gunnar Hjalmarsson (gunnarhj)]

Added the rls-hh-incoming tag. But the problem is still there in focal too, at least. <https://askubuntu.com/q/1308016> is a recent example.

Please give this the attention it deserves.

[Dag Bjerkeli (dag-e)]

I've conducted a test for some ISO's that I've got:

* Groovy from 8th Oct 2020
* 20.04.01
* 20.10
* Hirsute from 17th Jan 2021

The last three were downloaded 17th Jan.

All versions had correct keyboard layout where one could test the keyboard.
Likewise all installs had correct layout after installation.

But 20.04.01 and Hirsute had incorrect keyboard-layout when username and password were to be chosen.

The keyboard layout that I tested was Norwegian.

[Gunnar Hjalmarsson (gunnarhj)]

On 2021-01-18 20:49, Dag Bjerkeli wrote:
> But 20.04.01 and Hirsute had incorrect keyboard-layout when username
> and password were to be chosen.

Right, thanks for confirming. And that's what this bug is about.

[Michel-Ekimia (michel.ekimia)]

We tested again on 20.04.2 :

- Exactly the same problem as described

- Also when you type your password for ciphered LUKS which is more difficult to fix ...

I think at least we should have the 'eye' to see what password is being typed ....

[Łukasz Zemczak (sil2100)]

This is weird. I just tried reproducing this issue on both 20.04.2.0 release images and on the latest hirsute (21.04) dailies (Ubuntu Desktop) and failed. Steps I used to reproduce it:

1. Booted up the image on a VM
2. Ubiquity starts, on the left-hand side list I selected 'Norsk bokmål' and then the equivalent of 'Install Ubuntu'.
3. Moved through all the stages, installing the system on disk
4. On the username and password screen, I typed '[' on my keyboard and got 'å' on the input field instead - meaning the correct keyboard layout was used.

Am I doing something wrong? Can anyone tell me if I'm missing a step? Or is it no longer reproducible on both focal and hirsute *again*?

Cheers.

[Gunnar Hjalmarsson (gunnarhj)]

Hi Łukasz!

I tested the latest (2021-03-11) hirsute daily build and the 20.04.2 ISO.

As regards hirsute, I can no longer reproduce the issue. If nobody has explicitly addressed this bug, the code seems to be very fragile. But let's hope that it will work now until the release. After all the installer is about to be replaced with new software, and it's more important to focus on the new installer and the coming 22.04 LTS.

As regards 20.04.2, the issue (i.e. getting [ on the username/password window instead of å when pressing the [ key) was still there for me as previously reported. It was there when I followed your steps exactly as well as when I accomplished a couple of other variants involving Swedish.

Closed the hirsute task. It would be great if you could try the 20.04.2 ISO again. Do we have an annoying "sometimes" bug?

[Dag Bjerkeli (dag-e)]

I have just tested 20.10 and 20.04.02.
For 20.10 it's OK
For 20.04.02 the bug is still present.

These tests were done on both VMware 16.1 and VirtualBox 6.1 both of those on same Ubuntu 20.04.02 host.

[Gunnar Hjalmarsson (gunnarhj)]

Ehh.. Sorry Łukasz, but the issue was back in hirsute when I played with today's daily ISO in a VM. Fragile indeed. :(

[Gunnar Hjalmarsson (gunnarhj)]

Additional observation as regards hirsute: Different behavior depending on from where the installation is started:

* The issue is present if I press "Install Ubuntu" from ubiquity's welcome screen, i.e. without trying Ubuntu.

* The issue is not present if I press "Try Ubuntu" and later start the installation via the desktop icon in the live session.

[Gunnar Hjalmarsson (gunnarhj)]

Spam reversed.

[Gunnar Hjalmarsson (gunnarhj)]

@Kadir: If you have found a solution to this bug, please show us. If not, stop changing the status and assignment of this bug!

[Łukasz Zemczak (sil2100)]

Hello Gunnar, sorry to poke about this again, but can you still reproduce it?

[Gunnar Hjalmarsson (gunnarhj)]

@Łukasz: Without further testing with this in focus, my answer would be: Yes, sometimes.

I have installed daily impish builds of a couple of flavors lately, and I did see it at least for one of them. So the thing is still fragile. Race condition?

But assuming you are primarily focused on standard Ubuntu, I'd need to grab a fresh ISO and check again.

[Nicolas Mouart (nm9999)]

Same problem during installation using Hirsute Hippo 21.04 Desktop ISO image on virtualbox 6.1.26.

I choose "Français" at the beginning and setup changes correctly to French language.

Then I pick "French" (topmost choice in keyboard mapping list) at the keyboard layout stage and I can confirm in the text bar, underneath the mapping list on the same screen, that the keyboard mapping is correct (AZERTY).

Then during partitioning, I choose automated partitioning with LVM and Encryption

First issue : Keyboard mapping is set back to US layout (confirmed by trying change the location of the security key and typing in the contextual folder path ).
NOTE: All steps are displayed in French language as expected.

Second issue : Again, during creation of the first user, the keyboard mapping is also set to US and not French.

Upon restart, the disk decryption fails since the keyboard mapping was wrong during setup. (Trying FR to US mapping translation of the password fails as well ). I had to reinstall.

New install using test password admin123 -> I can see qd;in!@# instead (US standard mapping )

=> It seems the problem happens just after the change of keyboard mapping, it is not taken into account by the installer but it is later on after install.

Workaround : I typed the desired password in US mapping (qd,in&é" on a French standard keyboard) during installation.

IMPORTANT: Using the workaround, after restart at the end of installation, the mapping is correct ( french) from then on (disk decryption and login succeeded using admin123 password on French keyboard, etc ...).

[Michel-Ekimia (michel.ekimia)]

Just to say our clients face this as we ship in oem-config mode , which leads that they cannot login to their computer anymore and they did not see what QWERTY password they typed !

This is then a very high problem which has been seen even in 16.04 from time to time

Could we at least have a patch that fix this on a branch and we coudl ship the custom version ?

because right now , we just want to stop Ubuntu until this is fixed.

[Łukasz Zemczak (sil2100)]

Okay, giving this another try. I'll start off by checking the hirsute ISO as indicated by Nicolas in comment #49. Hopefully I'll have more luck getting this reproduced.

[Michel-Ekimia (michel.ekimia)]

This should be a bit less critical as 22.04 shall include this

https://github.com/canonical/ubuntu-desktop-installer/issues/589

[Łukasz Zemczak (sil2100)]

Of course I have problems reproducing this, I tried multiple times and got nothing. It's hard to pinpoint this without a reproducer. I'll try to scan the code for any funny parts where it could break, but I'd at least like to know if the problem here is with setxkbmap failing, or ubiquity setting it to a different value at some weird moment.

If anyone is able to reproduce the issue right now, I'd welcome some logs/info from a running ubiquity instance there:
 * I'd love the ubiquity syslog from /var/log/syslog when ubiquity is running
 * Might be good to get some locale and layout info straight away. So if someone notices at some ubiquity page that the layout is wrong, opening a terminal with ctrl+alt+t and then running `setxkbmap -query` and `locale`, pasting their output here

From the initial logs attached to the bug here I was maybe thinking that there's some race that could cause setxkbmap maybe fail when an invalid/incompatible locale is running, but then again I couldn't actually lead to such situation myself. I'd really appreciate as much logs as possible from a running system.

@Nicolas, @Michel - could you possibly help out? Thank you!

[Łukasz Zemczak (sil2100)]

I would love to actually get to the bottom of this issue. But as it is hard without a reproducer or solid logs, maybe the approach of being able to show the password could be a good workaround. I can go ahead with that, but people that do not check the password that they type-in will still be broken.

[Łukasz Zemczak (sil2100)]

Ok, submitted an MP for having a helpful visibility toggle. But even with that released, we need to find the source of the issue anyway - so logs are more than welcome!

[Dag Bjerkeli (dag-e)]

Hi I just tried install of jammy-desktop-amd64.iso (download yesterday) and ran into the keybord issue straight away. So I can provide some info:
'locale'
LANG=C.UTF-8
LANGUAGE=
LC_CTYPE="C.UTF-8"
LC_NUMERIC="C.UTF-8"
LC_TIME="C.UTF-8"
LC_COLLATE="C.UTF-8"
LC_MONETARY="C.UTF-8"
LC_MESSAGES="C.UTF-8"
LC_PAPER="C.UTF-8"
LC_NAME="C.UTF-8"
LC_ADDRESS="C.UTF-8"
LC_TELEPHONE="C.UTF-8"
LC_MEASUREMENT="C.UTF-8"
LC_IDENTIFICATION="C.UTF-8"
LC_ALL=

ubuntu@ubuntu:/var/log$ setxkbmap -query
rules: evdev
model: pc105
layout: us,us
variant: ,
ubuntu@ubuntu:/var/log$

syslog attached as file: syslog-jammy-desktop-20220120.txt

[Dag Bjerkeli (dag-e)]

The installation process is at the point where I'm creating user, that is entering username and password. Noticed that typing Norwegian characters in name field produces unwanted result.

[Brian Murray (brian-murray)]

The Hirsute Hippo has reached End of Life, so this bug will not be fixed for that release.

[Łukasz Zemczak (sil2100)]

Thank you Dag! Let me try looking at the provided logs and see if I can pinpoint it better.

In the meantime, the visibility toggle has been committed for GTK + I prepared a MP for the KDE counterpart.

[Łukasz Zemczak (sil2100)]

Just noting down my observations here: I think combining the new logs with the old ones I see one suspicious thing that I already tried investigating earlier, but now I'm more certain of it. It looks like that after setting the locale, there's a lot of errors like:

Jan 20 20:46:18 ubuntu ubiquity: perl: warning: Setting locale failed.
Jan 20 20:46:18 ubuntu ubiquity: perl: warning: Please check that your locale settings:
Jan 20 20:46:18 ubuntu ubiquity: #011LANGUAGE = "nb_NO.UTF-8",
(...)
Jan 20 20:46:18 ubuntu ubiquity: are supported and installed on your system.

My earlier guess was that due to the locale not being able to be correctly set, maybe the keyboard layout setting via setxkbmap can also fail due to some incompatibilities with the locale. I tried various combinations of locale-layout though and nothing really seemed to have had the same symptoms. Still, makes me wonder: why does that even happen? Are the respective locale somehow not locale-gen'd?

[Nicolas Mouart (nm9999)]

Hello Lukasz,

As mentioned earlier, I replicated using Virtualbox 6.1.26. Looking at the syslog and after further investigation,
the problem happens following open-vm-tools-desktop being installed.

This triggers gdm-x-session to update the layout and, for some unclear reason, set the xkbmap to the gnome-session settings instead of the ones
defined by ubiquity under /etc/default/keyboard.
Since setxkbmap is a Xsession tool and settings are not kept per se, ubiquity should ensure that it enforces desired keyboard settings, after ubuntu-drivers has finished installing potentially required packages ( which may affect gdm-x-session).

Workaround 2 : On the update software page, choose minimal install, click continue. Wait for a moment that the ubuntu-drivers list-oem finishes ( in the background, you can't see it unless looking at the syslog, the display blinks very quickly at some point in my case), then go back to the keyboard page and continue again.
This time the keyboard settings are kept as desired during partitioning and user steps.

Workaround 3 : Modify the virtualbox machine settings when turned off

Under Virtualbox Machine's Settings > Display > Graphics Controller
Changed VMSVGA (default) to VBoxSVGA
As a result, open-vm-tools is not installed, gdm-x-session doesn't update the layout and keyboard settings do not change.

I hope this helps.

[Nicolas Mouart (nm9999)]

Note: this is probably caused, in my case, by "udevadm trigger" being run in "/var/lib/dpkg/info/open-vm-tools-desktop.postinst configure".
This command is run by dpkg during install of open-vm-tools.

Replicated problem as follow :

in a terminal :

1 setxbkmap fr
2 setxkbmap -query -> returns fr layout
3 sudo udevadm trigger
4. setxkbmap -query -> returns us layout (gnome-session layout)

in syslog I can see gdm-x-session updating and the display blinks/freezes for a short period

[Nicolas Mouart (nm9999)]

Looked at it further.

Basically gnome-shell is overriding ubiquity's changes when ubuntu-drivers installs silently oem drivers on VMware products env ( or Virtualbox with default/recommended VMSVGA controller), namely open-vm-tools package.

udevadm trigger is executed and as a result Xorg and gnome-shell re-read their config including keyboard layouts. gnome-shell overrides changes made by Xorg which uses /etc/default/keyboard ( written/updated by ubiaquity) settings.

It doesn't seem possible to stop gnome-shell updating xkb layouts.

A possible solution would be to make sure the gnome-session behind ubiquity also uses the same keyboard layout ( which makes sense during a direct ubuntu install).
Something like that :

gsettings set org.gnome.desktop.input-sources sources "[('xkb', 'fr')]"

[Ubuntu QA Website (ubuntuqa)]

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
http://iso.qa.ubuntu.com/qatracker/reports/bugs/1875062

[Dag Bjerkeli (dag-e)]

I've done some more testing on the daily images from today and yesterday. In one sequence I'm able to get correct keyboard throughout the installation. I've tried to illustrate it with this ASCII flow-chart

Welcome --(NOR)-+
   | |
 (ENG) TRY? --- YES --+
   | | |
  TRY INST INST
   | | |
  INST FAIL OK
   |
 (NOR)
   |
  FAIL

From welcome screen

1) No language selection, select try ubuntu. From desktop start installation and choose Norwegian
2) Select Norwegian go straight to installation
3) Select Norwegian, select try ubuntu. From desktop start installation and verify Norwegian

All installations 3rd party and updates were selected. And were done in vmWare Workstation.

I hope this might be of some help in reproducing the situation.

[Dag Bjerkeli (dag-e)]

The ascii flow didn't turn out well. Attaching a PNG that hopefully is better.

[Benjamin Drung (bdrung)]

Thanks to Nicolas Mouart and Dag Bjerkeli for analyzing the problem. I could easily reproduce/simulate the problem:

Bug report
==========

Tested on Ubuntu 21.10 with virt-manager. Created a virtual machine and selected "Ubuntu 21.04" as OS (which uses machine type pc-q35-6.0) and changed the video to virtio with 3D acceleration. Used jammy-desktop-amd64.iso from 2022-04-05.

* Install Welcome -> Try Ubuntu (keep English as language)
* Install Ubuntu 22.04 LTS
* Welcome -> Continue (keep English as language)
* Keyboard Layout -> Change to German (Neo 2), but other layouts will work too -> Continue
* Updates and other software -> Continue
* Installation type -> select what you want -> Install now
* Where are you? -> Continue
* Open a terminal and run:
```
sudo udevadm trigger
```

and boom. They keyboard layout changed back to English.

ubiquity sets /etc/default/keyboard when you select a keyboard layout, but does not change the keyboard set in GNOME.

Workaround
==========

Before installation: Open Settings -> Keyboard -> Input Sources -> Add (plus sign) your keyboard layout to the input sources. Then select your keyboard layout in the keyboard selector.

Fix
===

gnome-session settings should be changed when a keyboard layout is selected in ubiquity.

[Benjamin Drung (bdrung)]

Workaround
==========

Used jammy-desktop-amd64.iso from 2022-04-05.

* Install Welcome -> Try Ubuntu (keep English as language)
* Add your keyboard layout, e.g. by opening a terminal and run:
```
gsettings set org.gnome.desktop.input-sources sources "[('xkb', 'de+neo'), ('xkb', 'us'), ('xkb', 'au'), ('xkb', 'cm'), ('xkb', 'gb')]"
```
* Then select your language in the GNOME indicator
* Start the installation:
```
ubiquity --debug gtk_ui
```
* Welcome -> Continue (keep English as language)
* Keyboard Layout -> Change to German (Neo 2) -> Continue
* Updates and other software -> Continue
* Installation type -> select what you want -> Install now
* Where are you? -> Continue
* Open a terminal and run:
```
sudo udevadm trigger
```

After the udevadm command, the keyboard layout in the installer is still the same (Neo 2 in my case).

First (failing) draft to fix the problem
========================================

I changed ubiquity (see 0001-Set-GNOME-keyboard-layout-v2.patch) to add the selected keyboard layout to org.gnome.desktop.input-sources sources.

Used jammy-desktop-amd64.iso from 2022-04-05.

* Install Welcome -> Try Ubuntu (keep English as language)
* Applied 0001-Set-GNOME-keyboard-layout-v2.patch to /usr/lib/ubiquity/plugins/ubi-console-setup.py
* Open a terminal and watch settings:
```
watch gsettings get org.gnome.desktop.input-sources sources
```
* Start installer in debug mode in second terminal:
```
ubiquity --debug gtk_ui
```
* Welcome -> Continue (keep English as language)
* Keyboard Layout -> Change to German (Neo 2) -> Continue

The watch command in the first terminal shows the new entries:

```
[('xkb', 'de+neo'), ('xkb', 'de'), ('xkb', 'us'), ('xkb', 'au'), ('xkb', 'cm'), ('xkb', 'gb')]
```

The strange and unexpected behavior is that the new entries do not appear in the GNOME keyboard indicator (and neither in "gnome-control-center keyboard").

Log output:

```
ubuntu@ubuntu:~$ grep "###" /var/log/installer/debug
Apr 6 16:39:52 ubiquity: ### org.gnome.desktop.input-sources sources = [('xkb', 'us'), ('xkb', 'au'), ('xkb', 'cm'), ('xkb', 'gb')]
Apr 6 16:39:52 ubiquity: ### Keyboard layout ('xkb', 'us') already in GNOME input sources: [('xkb', 'us'), ('xkb', 'au'), ('xkb', 'cm'), ('xkb', 'gb')]
Apr 6 16:39:59 ubiquity: ### org.gnome.desktop.input-sources sources = [('xkb', 'us'), ('xkb', 'au'), ('xkb', 'cm'), ('xkb', 'gb')]
Apr 6 16:39:59 ubiquity: ### Calling: gsettings set org.gnome.desktop.input-sources sources [('xkb', 'de'), ('xkb', 'us'), ('xkb', 'au'), ('xkb', 'cm'), ('xkb', 'gb')]
Apr 6 16:40:52 ubiquity: ### org.gnome.desktop.input-sources sources = [('xkb', 'de'), ('xkb', 'us'), ('xkb', 'au'), ('xkb', 'cm'), ('xkb', 'gb')]
Apr 6 16:40:52 ubiquity: ### Calling: gsettings set org.gnome.desktop.input-sources sources [('xkb', 'de+neo'), ('xkb', 'de'), ('xkb', 'us'), ('xkb', 'au'), ('xkb', 'cm'), ('xkb', 'gb')]
Apr 6 16:40:57 ubiquity: ### org.gnome.desktop.input-sources sources = [('xkb', 'de+neo'), ('xkb', 'de'), ('xkb', 'us'), ('xkb', 'au'), ('xkb', 'cm'), ('xkb', 'gb')]
Apr 6 16:40:57 ubiquity: ### Keyboard layout ('xkb', 'de+neo') already in GNOME input sources: [('xkb', 'de+neo'), ('xkb', 'de'), ('xkb', ...

[Benjamin Drung (bdrung)]

Problem analysis
================

* ubiquity installs open-vm-tools.
* open-vm-tools calls "udevadm trigger" in it's postinst script (line 8).
* udevadm triggers all udev rules which includes all input devices.
* gdm-x-session reads /etc/default/keyboard and sets the keyboard layout.
* gnome-shell sets the keyboard layout as well.

ubiquity already sets /etc/default/keyboard, but does not change the keyboard layout in the GNOME session.

Proposed solution
=================

Let ubiquity also configure the GNOME session to use the selected keyboard layout.

Analysis details
================

The change of the keyboard layout can be triggered by running udevadm trigger for the keyboard device. Example for QEMU:

```
sudo udevadm trigger -v /sys/devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
```

joural log on a fresh live system (before starting the installer):

```
systemd-logind[1207]: Watching system buttons on /dev/input/event0 (Power Button)
/usr/libexec/gdm-x-session[4624]: (II) config/udev: removing device Power Button
/usr/libexec/gdm-x-session[4624]: (**) Option "fd" "27"
/usr/libexec/gdm-x-session[4624]: (II) event0 - Power Button: device removed
/usr/libexec/gdm-x-session[4624]: (II) UnloadModule: "libinput"
/usr/libexec/gdm-x-session[4624]: (II) systemd-logind: releasing fd for 13:64
/usr/libexec/gdm-x-session[4624]: (II) config/udev: Adding input device Power Button (/dev/input/event0)
/usr/libexec/gdm-x-session[4624]: (**) Power Button: Applying InputClass "libinput keyboard catchall"
/usr/libexec/gdm-x-session[4624]: (II) Using input driver 'libinput' for 'Power Button'
/usr/libexec/gdm-x-session[4624]: (II) systemd-logind: got fd for /dev/input/event0 13:64 fd 27 paused 0
/usr/libexec/gdm-x-session[4624]: (**) Power Button: always reports core events
/usr/libexec/gdm-x-session[4624]: (**) Option "Device" "/dev/input/event0"
/usr/libexec/gdm-x-session[4624]: (II) event0 - Power Button: is tagged by udev as: Keyboard
/usr/libexec/gdm-x-session[4624]: (II) event0 - Power Button: device is a keyboard
/usr/libexec/gdm-x-session[4624]: (II) event0 - Power Button: device removed
/usr/libexec/gdm-x-session[4624]: (**) Option "config_info" "udev:/sys/devices/LNXSYSTM:00/LNXPWRBN:00/input/input0/event0"
/usr/libexec/gdm-x-session[4624]: (II) XINPUT: Adding extended input device "Power Button" (type: KEYBOARD, id 6)
/usr/libexec/gdm-x-session[4624]: (**) Option "xkb_model" "pc105"
/usr/libexec/gdm-x-session[4624]: (**) Option "xkb_layout" "us"
/usr/libexec/gdm-x-session[4624]: (WW) Option "xkb_variant" requires a string value
/usr/libexec/gdm-x-session[4624]: (WW) Option "xkb_options" requires a string value
/usr/libexec/gdm-x-session[4624]: (II) event0 - Power Button: is tagged by udev as: Keyboard
/usr/libexec/gdm-x-session[4624]: (II) event0 - Power Button: device is a keyboard
gnome-shell[4794]: Window manager warning: Overwriting existing binding of keysym 31 with keysym 31 (keycode a).
gnome-shell[4794]: Window manager warning: Overwriting existing binding of keysym 32 with keysym 32 (keycode b).
gnome-shell[4794]: Window manager warning: Overwriting existing binding of keysym 33 with keysym 33 (keycode c).
gnome-shell[4794]: Wi...

[Benjamin Drung (bdrung)]

I have a working patch for ubiquity that sets the GNOME keyboard there as well. Next steps (next week):
* finish test cases
* test Kubuntu (KDE)
* then create merge request

I created bug #1968354 for open-vm-tools to make the udev trigger more precise.

Site notes
==========

If you do following steps in the live system:
```
sudo gedit # close without any change
ps auxff | grep dconf-servic[e]
ubiquity --debug gtk_ui
```

`sudo gedit` will start a dconf-service process owned by root. Then ubiquity will launch another dconf-service process owned by ubuntu:

```
ubuntu@ubuntu:~$ ps aux | grep dconf-servic[e]
ubuntu 1596 0.0 0.1 156984 6020 ? Ssl 17:28 0:00 /usr/libexec/dconf-service
root 5615 0.0 0.1 156856 5708 ? Sl 17:29 0:00 /usr/libexec/dconf-service
ubuntu 5998 0.0 0.1 156956 6060 ? Sl 17:29 0:00 /usr/libexec/dconf-service
```

In that case, gsettings changes by ubiquity will be send to the "wrong" dconf-service and the gnome-shell will not get these changes.

Solution: Run ubiquity the same way the ubiquity.desktop runs it:
```
sudo --preserve-env=DBUS_SESSION_BUS_ADDRESS,XDG_DATA_DIRS,XDG_RUNTIME_DIR sh -c 'ubiquity --debug gtk_ui'
```

Note 2: I also run into bug #1755452 during testing.

[Benjamin Drung (bdrung)]

Finished the test cases, successfully tested on Kubuntu and Xubuntu jammy live images, created merge request: https://code.launchpad.net/~ubuntu-installer/ubiquity/+git/ubiquity/+merge/419219

[Gunnar Hjalmarsson (gunnarhj)]

Thanks a lot for your hard work with this, Benjamin. It looks promising indeed.

While I suppose it's considered too late for the 22.04 release, I really hope that somebody will review and merge your proposal soon after that. If it proves to work without side issues, it could then be included in the 22.04.1 point release.

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubiquity - 22.04.15

---------------
ubiquity (22.04.15) jammy; urgency=medium

  * Set GNOME keyboard layout during installation (LP: #1875062)

 -- Benjamin Drung <email address hidden> Wed, 13 Apr 2022 15:56:30 -0700

[Gunnar Hjalmarsson (gunnarhj)]

I tested with the Ubuntu 22.04 daily build ISO from yesterday, and can no longer reproduce the issue with incorrect effective layout in the window for entering username, password, etc.

Well done, Benjamin! :) I'd love to see the fix backported to focal too.

[Nicolas Mouart (nm9999)]

Thanks Benjamin!

[Mark Smith (juglugs1974)]

Hi guys,

I'm sorry to say that this bug is back in 24.04 Beta.

[Dag Bjerkeli (dag-e)]

I've just tested this, and can confirm that there is a bug regarding keyboard layout in 22.04 beta. As this time the error also appears when you select the keyboard initially it could a new error.

I also have a preview image of 22.04 dated mars 27th that does not have the error, even after updated installer.

[Mark Smith (juglugs1974)]

Dag,

Can you confirm you mean 24.04 and not 22.04, please?

On Mon, 15 Apr 2024 at 17:25, Dag Bjerkeli <email address hidden>
wrote:

> I've just tested this, and can confirm that there is a bug regarding
> keyboard layout in 22.04 beta. As this time the error also appears when
> you select the keyboard initially it could a new error.
>
> I also have a preview image of 22.04 dated mars 27th that does not have
> the error, even after updated installer.
>
>
> ** Attachment added: "Entering æøå after selecting Norwegian keyboard does
> not show correct keys"
>
> https://bugs.launchpad.net/ubuntu/+source/ubiquity/+bug/1875062/+attachment/5765721/+files/not%20norwegian%20in%20install.png
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1875062
>
> Title:
> [20.04] Keyboard layout changes during installation before typing
> username/password
>
> Status in Release Notes for Ubuntu:
> Fix Released
> Status in ubiquity package in Ubuntu:
> Fix Released
> Status in ubiquity source package in Focal:
> Triaged
> Status in ubiquity source package in Hirsute:
> Won't Fix
>
> Bug description:
> During a fresh install of Ubuntu 20.04, selecting Norwegian keyboard
> is provided and keys are responding correctly at that stage. But later
> when entering user information the keyboard setting is wrong.
>
> It looks like it have fallen back to English keyboard-layout.
> I failed to log in after my first install, so a new attempt I tried to
> write some special letters in the name field, and noticed that I got the
> wrong characters for the key.
>
> The install is done in VMware workstation 15.
>
> In the attached screendump, the characters (';[":{) after my name
> should have been (æøåÆØÅ) that is norwegian characters.
>
> Problem analysis
> ================
>
> * ubiquity installs open-vm-tools.
> * open-vm-tools calls "udevadm trigger" in it's postinst script (line 8).
> * udevadm triggers all udev rules which includes all input devices.
> * gdm-x-session reads /etc/default/keyboard and sets the keyboard layout.
> * gnome-shell sets the keyboard layout as well.
>
> ubiquity already sets /etc/default/keyboard, but does not change the
> keyboard layout in the GNOME session.
>
> Proposed solution
> =================
>
> Let ubiquity also configure the GNOME session to use the selected
> keyboard layout.
>
> ProblemType: Bug
> DistroRelease: Ubuntu 20.04
> Package: ubiquity (not installed)
> ProcVersionSignature: Ubuntu 5.4.0-26.30-generic 5.4.30
> Uname: Linux 5.4.0-26-generic x86_64
> ApportVersion: 2.20.11-0ubuntu27
> Architecture: amd64
> CasperMD5CheckResult: skip
> CurrentDesktop: ubuntu:GNOME
> Date: Sat Apr 25 19:27:41 2020
> InstallCmdLine: file=/cdrom/preseed/ubuntu.seed initrd=/casper/initrd
> quiet splash --- maybe-ubiquity
> InstallationDate: Installed on 2020-04-25 (0 days ago)
> InstallationMedia: Ubuntu 20.04 LTS "Focal Fossa" - Release amd64
> (20200423)
> SourcePackage: ubiquity
> Symptom: installer
> UpgradeStatus: No upgrade log present (probably fresh install)
>
> To manage noti...

[Dag Bjerkeli (dag-e)]

Sorry, it was of course 24.04 I were testing. 24.04 beta and the preview image was also of 24.04.