X server fails to start on post-final Beta Lubuntu Vivid desktop images due to missing xserver-xorg-video-all package

[Lyn Perrine]

STEPS TO REPRODUCE
1. Boot daily post-final Beta Lubuntu Vivid desktop image
2. Select "Try Lubuntu without installing" or "Install Lubuntu" from the boot menu

EXPECTED RESULTS
X server starts

ACTUAL RESULTS
* X server fails
* /var/log/Xorg.0.log shows that video drivers are not found in /usr/lib/xorg/modules/drivers (xserver-xorg-video-all is not installed)
* system seems "hung" displaying startup info

AFFECTS
* virtual machines
  * VirtualBox 4.3.26, whether upstream or not
  * VMWare Workstation 8-10
* hardware
  * not all hardware is affected by this, but those requiring the drivers are (see dupe #1432843)
* images
  * Lubuntu Desktop amd64/i386

DOES NOT AFFECT
* images
  * Ubuntu
  * Xubuntu
  * Ubuntu GNOME
  * Kubuntu

NOTES
* 1:7.7+7ubuntu4 version of xserver-xorg set xserver-xorg-video-all from an explicit dependency to a recommends
* lubuntu seed includes "no-follow-recommends"
* removing the latter or explicitly calling for the former will fix the problem

[Ubuntu QA Website (ubuntuqa)]

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
http://iso.qa.ubuntu.com/qatracker/reports/bugs/1441843

[Walter Lapchynski (wxl)]

Confirmed on i386 in VirtualBox 4.3.26 from upstream repos. Was able to get to the virtual terminal and indeed the system is not frozen but nothing doing. lightdm fails to start and there seem to be errors setitng up the display in X.

Not to speak for him but Kamilion mentioned that this also failed with VMWare Workstation 11.1.

[Walter Lapchynski (wxl)]

Also note that there are no problems on bare metal.

[Walter Lapchynski (wxl)]

Calling this high because it makes testing difficult and vivid final is right around the corner.

[Walter Lapchynski (wxl)]

Also note Xubuntu is not affected. Other flavors should be checked.

[Walter Lapchynski (wxl)]

/var/log/X.org.0.log (http://paste.ubuntu.com/10795553/) shows:

[ 109.070] (EE) Device(s) detected, but none match those in the config file.
[ 109.070] (EE)
Fatal server error:
[ 109.070] (EE) no screens found(EE)

/var/log/lightdm/lightdm.log (http://paste.ubuntu.com/10794305/) shows:

[+0.75s] DEBUG: Seat seat0: Creating display server of type x
[+0.75s] DEBUG: Using VT 7
[+0.75s] DEBUG: Seat seat0: Starting local X display on VT 7
[+0.75s] DEBUG: DisplayServer x-0: Logging to /var/log/lightdm/x-0.log
[+0.75s] DEBUG: DisplayServer x-0: Writing X server authority to /var/run/lightdm/root/:0
[+0.75s] DEBUG: DisplayServer x-0: Launching X Server
[+0.76s] DEBUG: Launching process 1504: /usr/bin/X -core :0 -seat seat0 -auth /var/run/lightdm/root/:0 -nolisten tcp vt7 -novtswitch
[+0.76s] DEBUG: DisplayServer x-0: Waiting for ready signal from X server :0
[+1.39s] DEBUG: Process 1504 terminated with signal 6
[+1.39s] DEBUG: DisplayServer x-0: X server stopped
[+1.39s] DEBUG: Releasing VT 7
[+1.39s] DEBUG: DisplayServer x-0: Removing X server authority /var/run/lightdm/root/:0
[+1.39s] DEBUG: Seat seat0: Display server stopped
[+1.39s] DEBUG: Seat seat0: Stopping; greeter display server failed to start
[+1.39s] DEBUG: Seat seat0: Stopping
[+1.39s] DEBUG: Seat seat0: Stopping session
[+1.39s] DEBUG: Seat seat0: Session stopped
[+1.39s] DEBUG: Seat seat0: Stopped
[+1.39s] DEBUG: Required seat has stopped
[+1.39s] DEBUG: Stopping display manager
[+1.39s] DEBUG: Display manager stopped
[+1.39s] DEBUG: Stopping daemon
[+1.40s] DEBUG: Seat seat0: Stopping session
[+1.42s] DEBUG: Exiting with return value 1

/var/log/syslog (http://paste.ubuntu.com/10794321/) shows:

Apr 10 20:13:56 lubuntu systemd[1]: Starting Light Display Manager...
Apr 10 20:13:57 lubuntu systemd[1]: Started Light Display Manager.
Apr 10 20:13:58 lubuntu systemd[1]: lightdm.service: main process exited, code=exited, status=1/FAILURE
Apr 10 20:13:58 lubuntu systemd[1]: Unit lightdm.service entered failed state.
Apr 10 20:13:58 lubuntu systemd[1]: lightdm.service failed.
Apr 10 20:13:58 lubuntu systemd[1]: lightdm.service holdoff time over, scheduling restart.
Apr 10 20:13:58 lubuntu systemd[1]: start request repeated too quickly for lightdm.service
Apr 10 20:13:58 lubuntu systemd[1]: Failed to start Light Display Manager.
Apr 10 20:13:58 lubuntu systemd[1]: Unit lightdm.service entered failed state.
Apr 10 20:13:58 lubuntu systemd[1]: lightdm.service failed.

[Kamilion (kamilion)]

For what it's worth, I have some previous dailys of when it was last working somewhere between febuary and march.
http://files.sllabs.com/files/long-term/downloads/isos/vivid-desktop-2-18-15-i386.iso
http://files.sllabs.com/files/long-term/downloads/isos/vivid-desktop-2-19-15-amd64.iso
Unfortunately, I didn't save the generated hashes, so you can only take my word they're official dailies.

This one, on the other hand, I've unpacked 2-19-15 with ubuntu-builder, ran apt-get update && apt-get distupgrade, and exported the ISO back out with no other modifications on 3-30-15. It is operational.
http://files.sllabs.com/files/long-term/downloads/isos/kamikazi-15.04-beta2-amd64.iso
*this is not an official daily image!*

Should you wish to replicate the process I followed, one can get a hold of the kamikazi-14.10 release from here:
https://github.com/kamilion/kamikazi-deploy
Which includes the (xen patched) version of ubuntu-builder used in the above "kamikazi-15.04-beta2-amd64.iso".

Please use these resources to track down the problem; I've been beating on it for a few weeks now but have made no headway.
From what I can see, the VBE fallback is not activating on the current (april) dailies, and yet it does on kamikazi-15.04-beta2.
There must have been a configuration snippet somewhere that no longer exists in a newer package, and I know that 3.19 landed somewhere late February or early march along with some changes to KMS/DRM bringup. That is where I suspect we are missing a configuration line that once was there in the past.

[Walter Lapchynski (wxl)]

Looks like we have the problem: xserver-xorg-video-* drivers missing: https://puu.sh/hd0j7/83445c6f1e.jpg

[Marco Parillo (marco-parillo)]

Does not seem to apply to Kubuntu (at least on VMware Player 6.0.5 build-2443746) on today's ISO ( http://cdimage.ubuntu.com/kubuntu/daily-live/20150413/vivid-desktop-i386.iso).

[Kamilion (kamilion)]

Alright, I think I've tracked it down with WXL's help.
http://files.sllabs.com/files/long-term/screenshots/kamikazi/daily-on-left-and-good-on-right.jpg
(permanent version of https://puu.sh/hd0j7/83445c6f1e.jpg as puush will expire the screenshot after a while)

[Walter Lapchynski (wxl)]

My guess is that https://bugs.launchpad.net/ubuntu/+source/xorg/+bug/1432843 has the same basic problem: missing xserver-xorg-video-* packages.

[Kamilion (kamilion)]

I've spent some more time in tracking this down.

Here's the change that made this occur:
http://files.sllabs.com/files/long-term/screenshots/kamikazi/vmware-broken-for-lubuntu-and-this-is-the-culprit-commit.jpg

On march 16th, xserver-xorg-video-all was moved from Depends to Recommends in the xserver-xorg package.

This happens to us for two reasons:
One: lubuntu's seed specifies no-follow-recommends.
Two: We do not have an explicit dependency on xserver-xorg-video-all.

I propose that an explicit dependency is added in either lubuntu's seed, or in one of the metapackages, to follow suit with the rest of the crowd. They all have the packages installed, one way or another, and we are the odd-man-out.

ubuntu-mate seems to have added their explicit dependency in the vivid cycle.
edubuntu had the explicit dependency in utopic, so they never noticed.
Xubuntu does not have no-follow-recommends, so they never noticed.

The others (ubuntu+Unity, kubuntu, etc) are all primarily based on GL compositors these days, and simply barf on the simple 2D cards that most of these remaining packages support.

Specifically:
xserver-xorg-video-vmware
xserver-xorg-video-vesa
xserver-xorg-video-mga
xserver-xorg-video-cirrus
xserver-xorg-video-mach64
xserver-xorg-video-trident

MGA is required to support the Matrox G200e-compatible VGA adapter found in Nuvoton and ASPEED baseboard management controllers on various supermicro X8 and X9 series motherboards.
Mach64 is required to support some older supermicro X7-series on-motherboard VGA adapter.
cirrus and trident are required for some emulation environments like qemu.

Most of the pieces of actual hardware that these drivers were built for are no longer used; but they are old enough to be "the popular virtual devices to emulate".

I am not sure if these drivers will ever be updated to support DRM or KMS, or even if it is worth it for them to be changed, with the progress on wayland+pixman.

This comment is found in several of the seeds, and no longer seems to apply as of the march 16th commit to xserver-xorg.
# this includes xserver-xorg->xserver-xorg-{input|video}-* (no need to add specific drivers manually)

Here is my factual corroboration of the above set of opinions.

[code]
ubuntu@kamikazi-builder:~/seedfinder/seeds$ grep -r "xserver-xorg" | grep -v "xserver-xorg-input" | grep -v "dbg" | grep -v "geode"
edubuntu.utopic/desktop-gnome: * (xserver-xorg-video-all)
edubuntu.vivid/desktop-gnome: * (xserver-xorg-video-all)
kubuntu.utopic/active: * xorg # this includes xserver-xorg->xserver-xorg-{input|video}-* (no need to add specific drivers manually)
kubuntu-active.utopic/active: * xorg # this includes xserver-xorg->xserver-xorg-{input|video}-* (no need to add specific drivers manually)
kubuntu-active.vivid/active: * xorg # this includes xserver-xorg->xserver-xorg-{input|video}-* (no need to add specific drivers manually)
platform.utopic/desktop-common: * xorg # this includes xserver-xorg->xserver-xorg-{input|video}-* (no need to add specific drivers manually)
platform.utopic/installer-gtk: * xserver-xorg-core-u...

[Walter Lapchynski (wxl)]

Seed problem confirmed. Julien has been informed and will deal with this right away.

[Walter Lapchynski (wxl)]

Just linked the branch where Julien fixed the problem by explicitly requiring xserver-xorg-video/input-all.

[Julien Lavergne (gilir)]

@Adam : I'm not sure the change you done on http://bazaar.launchpad.net/~ubuntu-core-dev/ubuntu-seeds/platform.vivid/revision/1991 will be send to lubuntu seed, since we don't install recommends. Worse, I try to override it on lubuntu-seed, but I didn't succeed (it seems that the change on desktop-common seed take the priority).

[Adam Conrad (adconrad)]

Oh, your lack of recommends make things entertaining. I keep forgetting about that. We really need to sort out how to get it so you no longer have to do that.

Arguably, we could revert your seed change and mine and just flip the xorg deps back around.

[Adam Conrad (adconrad)]

Anyhow, I reverted my change, and rebuilt your meta, and that seems to DTRT. So, I'll upload that for now, but think on whether flipping the xorg deps around might be the better solution before release.

[Launchpad Janitor (janitor)]

This bug was fixed in the package lubuntu-meta - 0.59

---------------
lubuntu-meta (0.59) vivid; urgency=medium

  * Refreshed dependencies (LP: #1441843)
  * Added xserver-xorg-input-all to core
  * Added xserver-xorg-video-all to core
 -- Adam Conrad <email address hidden> Tue, 14 Apr 2015 15:52:41 -0600

[Walter Lapchynski (wxl)]

FWIW MATE has the same lack of recommends, but they explicitly added the depend.

[Walter Lapchynski (wxl)]

Confirmed working by checking the manifest. Thanks again everyone!

[Istimsak (saqman2060)]

After looking up the definition of this package on the debian packages site, https://packages.debian.org/squeeze/xserver-xorg-video-all, it says this package depends on the full suite of output drivers for Xorg X server. It does not provide any drivers itself but can be removed if the user requires certain drivers.

If it does not provide any drivers itself, why is it needed? It sounds as if this package acts like a container holding a bunch of output drivers for the Xserver. If so, a distribution developed with this package depends on the drivers from this package. Removing this package will cause the Xserver to fail, just like what I was experiencing.

This package holds all, or the required drivers for Xserver to operate on any GPU device that is supports, I am inferring.

[Julien Lavergne (gilir)]

Thanks Adam :-)