The root cause of this problem is plymouth, indeed.
The 'chvt N' command blocks if the VT/tty is in VT_AUTO + KD_GRAPHICS state.
In this state the kernel bails out early in the ioctl(VT_ACTIVATE) syscall
and does not post the VT_EVENT_SWITCH event the ioctl(VT_WAITACTIVE) syscall
will be waiting for -- causing chvt to block.
gdm properly sets the VT out of VT_AUTO mode (which causes chvt not to block)
after it tells plymouth to deactivate.
BUT plymouth can set it back to VT_AUTO mode afterward, regardless, while it
handles the udev event of the DRM/DRI graphics card addition, as that causes
the VT/tty to be reconfigured.
This can be verified with plymouth debugging, e.g., kernel boot option
'plymouth.debug=file:/run/plymouth.debug', plus source code inspection:
[ply-device-manager.c:LINE] create_devices_from_udev:Timeout elapsed, looking for devices from udev
...
[ply-device-manager.c:LINE] create_devices_for_terminal_and_renderer_type:creating devices for /dev/dri/card0 (renderer type: 1) (terminal: /dev/tty1)
...
[./plugin.c:LINE] load_driver:Opening '/dev/dri/card0'
[ply-terminal.c:LINE] ply_terminal_open:trying to open terminal '/dev/tty1'
3) init calls 'plymouth quit --retain-splash' which goes into
ply_terminal_close() again, and since watching is true, it
sets the VT into VT_AUTO again (see calls in #1 above) ...
(*after* gdm had already set the VT up out of VT_AUTO).
[ply-boot-server.c:LINE] print_connection_process_identity:connection is from pid PID (/bin/plymouth quit --retain-splash) with parent pid PID (/sbin/init splash)
...
[ply-terminal.c:LINE] ply_terminal_close:restoring color palette
[ply-terminal.c:LINE] ply_terminal_close:stop watching tty fd
....
That depends on timing (plymouth udev event watch timeout + device detection)
and this probably explains why the problem does not happen every single time
(but apparently it's the case often; the problem reproduces most of the time
in this KVM guest with Ubuntu 18.04.2 Desktop).
After understanding that this behavior / code path is responsible for the
problem, I found there's an upstream for this in plymouth, which realizes
that after 'plymouth deactivate' the udev events should not be reacted on,
which prevents re-setting the VT_AUTO mode.
Interestingly this fix is already applied in Ubuntu Cosmic and later, for
LP: #1795637, due to a different problem (cayses wayland/xorg to fail).
The patch needed just a small refresh to apply to Bionic, and a test kernel
with it applied successfully passes all 'chvt' tests, multiple times.
1) While gdm is in the login screen, 'ssh <guest> -- sudo chvt 4'
2) With gdm autologin, try the same.
Possible workarounds for this are disabling plymouth (remove the 'splash'
option from kernel/grub boot options) OR setting the kernel console to
a device other than tty0/tty1 (check it with 'dmesg | grep console'),
for example, console=ttyS0 or console=ttyS1 (serial/non-graphic consoles).
This causes plymouth not to mess with the VT used by gdm (a graphic one).
The root cause of this problem is plymouth, indeed.
The 'chvt N' command blocks if the VT/tty is in VT_AUTO + KD_GRAPHICS state.
In this state the kernel bails out early in the ioctl(VT_ACTIVATE) syscall WAITACTIVE) syscall
and does not post the VT_EVENT_SWITCH event the ioctl(VT_
will be waiting for -- causing chvt to block.
The function path is:
vt_ioctl(tty0, VT_ACTIVATE, ...) console_ callback( ); return 0; <<-- continue to send event. change_ console( ) post(VT_ EVENT_SWITCH, ...)
-> set_console()
-> if (... VT_AUTO && KD_GRAPHICS ...) return -EINVAL; <<-- bails out.
-> schedule_
-> console_callback()
-> change_console()
-> complete_
-> vt_event_
vt_ioctl(tty0, VT_WAITACTIVE, ...) wait(VT_ EVENT_SWITCH) <<-- blocks/wait to receive event.
-> vt_waitactive()
-> __vt_event_
gdm properly sets the VT out of VT_AUTO mode (which causes chvt not to block)
after it tells plymouth to deactivate.
BUT plymouth can set it back to VT_AUTO mode afterward, regardless, while it
handles the udev event of the DRM/DRI graphics card addition, as that causes
the VT/tty to be reconfigured.
This can be verified with plymouth debugging, e.g., kernel boot option debug=file: /run/plymouth. debug', plus source code inspection:
'plymouth.
1) gdm calls 'plymouth deactivate', which calls terminal_ close() stop_watching_ for_vt_ changes( ) >is_watching_ for_vt_ changes == true) ioctl(VT_SETMODE, VT_AUTO) >is_watching_ for_vt_ changes = false
ply_
-> ply_terminal_
-> if (terminal-
-> terminal-
[ply-boot- server. c:LINE] print_connectio n_process_ identity: connection is from pid PID (/bin/plymouth deactivate) with parent pid PID (/usr/sbin/gdm) c:LINE] ply_terminal_ close:restoring color palette c:LINE] ply_terminal_ close:stop watching tty fd
...
[ply-terminal.
[ply-terminal.
...
2) plymouth udev event timeout expires, it notices the DRM/DRI devices,
and re-enables the VT watching while processing those; in the calls:
ply_ terminal_ open() watch_for_ vt_changes( ) >is_watching_ for_vt_ changes = true;
-> ply_terminal_
-> terminal-
[ply-device- manager. c:LINE] create_ devices_ from_udev: Timeout elapsed, looking for devices from udev manager. c:LINE] create_ devices_ for_terminal_ and_renderer_ type:creating devices for /dev/dri/card0 (renderer type: 1) (terminal: /dev/tty1) c:LINE] ply_terminal_ open:trying to open terminal '/dev/tty1'
...
[ply-device-
...
[./plugin.c:LINE] load_driver:Opening '/dev/dri/card0'
[ply-terminal.
3) init calls 'plymouth quit --retain-splash' which goes into terminal_ close() again, and since watching is true, it
ply_
sets the VT into VT_AUTO again (see calls in #1 above) ...
(*after* gdm had already set the VT up out of VT_AUTO).
[ply-boot- server. c:LINE] print_connectio n_process_ identity: connection is from pid PID (/bin/plymouth quit --retain-splash) with parent pid PID (/sbin/init splash) c:LINE] ply_terminal_ close:restoring color palette c:LINE] ply_terminal_ close:stop watching tty fd
...
[ply-terminal.
[ply-terminal.
....
That depends on timing (plymouth udev event watch timeout + device detection)
and this probably explains why the problem does not happen every single time
(but apparently it's the case often; the problem reproduces most of the time
in this KVM guest with Ubuntu 18.04.2 Desktop).
After understanding that this behavior / code path is responsible for the
problem, I found there's an upstream for this in plymouth, which realizes
that after 'plymouth deactivate' the udev events should not be reacted on,
which prevents re-setting the VT_AUTO mode.
Interestingly this fix is already applied in Ubuntu Cosmic and later, for
LP: #1795637, due to a different problem (cayses wayland/xorg to fail).
The patch needed just a small refresh to apply to Bionic, and a test kernel
with it applied successfully passes all 'chvt' tests, multiple times.
1) While gdm is in the login screen, 'ssh <guest> -- sudo chvt 4'
2) With gdm autologin, try the same.
Possible workarounds for this are disabling plymouth (remove the 'splash'
option from kernel/grub boot options) OR setting the kernel console to
a device other than tty0/tty1 (check it with 'dmesg | grep console'),
for example, console=ttyS0 or console=ttyS1 (serial/non-graphic consoles).
This causes plymouth not to mess with the VT used by gdm (a graphic one).