Upgrade from 14.04.4→ 16.04 dies midway taking out the session.

Bug #1555237 reported by Dave Morley on 2016-03-09
156
This bug affects 20 people
Affects Status Importance Assigned to Milestone
dbus (Ubuntu)
Critical
Mathieu Trudel-Lapierre
systemd (Ubuntu)
Critical
Martin Pitt

Bug Description

STEPS:
1. Install 14.04.4 on a uefi securebooted system
2. update it
3. Install openssh-server for debug access
4. reboot
5. Hit alt+f2
6. Type update-manager -d and hit enter

EXPECTED:
I expected a smooth transition to 16.04

ACTUAL:
System is completely taken out mid way through the process killing the session and x also no tty when hitting ctrl+alt+f1-6

---

TESTS RESULTS (from cyphermox):

On amd64 (no EFI) under qemu/libvirt, qxl graphics:
- upgrading just libglib2.0-0 from 16.04 on top of 14.04.4: PASS
- upgrading ubuntu-desktop (running gnome-shell) 14.04.4 to 16.04: PASS
- upgrading ubuntu-desktop (running xubuntu-session) 14.04.4 to 16.04: PASS
- manually upgrading compiz and other packages to 16.04 versions; after adding xenial sources:
  - compiz and related packages (http://paste.ubuntu.com/15415280/) : PASS
  - xserver-xorg and related: PASS
  - unity and related: PASS
  - ubuntu-desktop: PASS
  - libgl1-mesa-dri: PASS
  - apt-get dist-upgrade: FAIL (screensaver may have kicked in)
- standard upgrading 14.04.4 to 16.04; screensaver disabled: FAIL (looked fine, but upgrade eventually froze, might well be a different issue)
- re-enabling screensaver after upgrade, before reboot: TODO (above failed)

On ppc64el:
- upgrading ubuntu-server 14.04.4 -> 16.04: PASS

---

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: ubuntu-release-upgrader-core 1:0.220.8
ProcVersionSignature: Ubuntu 4.2.0-30.36~14.04.1-generic 4.2.8-ckt3
Uname: Linux 4.2.0-30-generic x86_64
ApportVersion: 2.14.1-0ubuntu3.19
Architecture: amd64
CrashDB: ubuntu
Date: Wed Mar 9 17:01:27 2016
InstallationDate: Installed on 2016-03-04 (5 days ago)
InstallationMedia: Ubuntu 14.04.4 LTS "Trusty Tahr" - Beta amd64 (20160224)
PackageArchitecture: all
SourcePackage: ubuntu-release-upgrader
Symptom: ubuntu-release-upgrader
UpgradeStatus: Upgraded to xenial on 2016-03-09 (0 days ago)
VarLogDistupgradeTermlog:

Dave Morley (davmor2) wrote :
tags: added: qa-daily
Changed in ubuntu-release-upgrader (Ubuntu):
importance: Undecided → Critical
Dave Morley (davmor2) wrote :

Happens on both hardware and kvm

Will Cooke (willcooke) wrote :

From Dave's Apt Term Log he got as far as being asked a question in the terminal:

There are services installed on your system which need to be restarted
when certain libraries, such as libpam, libc, and libssl, are upgraded.
Since these restarts may cause interruptions of service for the system,
you will normally be prompted on each upgrade for the list of services
you wish to restart. You can choose this option to avoid being
prompted; instead, all necessary restarts will be done for you
automatically so you can avoid being asked questions on each
library upgrade.

Restart services during package upgrades without asking?

I got as far as this, then expanded the terminal view in the upgrader, answered "Y" and the install carried on.

And then it crashed later on....

(From my term log):

insserv: Default-Start undefined, assuming empty start runlevel(s) for script `cron'
insserv: Default-Stop undefined, assuming empty stop runlevel(s) for script `cron'
insserv: warning: script 'cron' missing LSB tags and overrides
insserv: Default-Start undefined, assuming empty start runlevel(s) for script `cron'
insserv: Default-Stop undefined, assuming empty stop runlevel(s) for script `cron'
Setting up bash-completion (1:2.1-4.2ubuntu1) ...
Setting up liblocale-gettext-perl (1.07-1build1) ...

However - I noticed that the Upgrader window became grey and unresponsive slightly before that. Maybe as much as 5 mins earlier.

I have attached my apt term log.

So question 1: Should that prompt actually open a window to prompt the user?
Question 2: why did mine crash later on?

Will Cooke (willcooke) wrote :

Note, I did not see a complete session crash.

Will Cooke (willcooke) wrote :

Ignore comment 4.
I had an active shell via SSH still but when I tried to reboot the box it froze. When I reset the machine and try to boot I now have a kernel panic.

Will Cooke (willcooke) wrote :

I've not been able to get the kernel panic logs, I've attached a screenshot in the vain hope it might help.

Will Cooke (willcooke) on 2016-03-14
tags: added: 1604-upgrade
Will Cooke (willcooke) on 2016-03-15
Changed in ubuntu-release-upgrader (Ubuntu):
status: New → Confirmed
Steve Langasek (vorlon) on 2016-03-17
Changed in ubuntu-release-upgrader (Ubuntu):
assignee: nobody → Mathieu Trudel-Lapierre (mathieu-tl)

The interesting thing to notice here is that if I don't let the screensaver come up doing a manual upgrade (because I'm watching it more), it seems like compiz doesn't crash. It needs more testing, but it looks like the only reason the upgrade failed was because the screensaver kicked in.

description: updated
description: updated
Changed in ubuntu-release-upgrader (Ubuntu):
status: Confirmed → In Progress
Will Cooke (willcooke) wrote :

I disabled the screensaver in my tests, and it still crashed - just later on.

I SSH'd in to the box and ran compiz like this:

$ DISPLAY=:0 compiz --replace --debug

and then started the upgrade (with the screensaver disabled). This is the error I got:

XIO: fatal IO error 11 (Resource temporarily unavailable) on X server ":0"
      after 24793 requests (24793 known processed) with 1 events remaining.

Trevinho suggested compiling Compiz with more debugging and trying that. Assigned to Eleni for more testing.

Changed in compiz (Ubuntu):
status: New → Confirmed
assignee: nobody → Eleni Maria Stea (hikiko)
importance: Undecided → Critical
Brian Murray (brian-murray) wrote :

Its worth noting DistUpgradeQuirks.py contains some code to disable gnome-screensaver.

It doesn't seem like the screensaver really is properly disabled during upgrades, at least looking very naively at what happens during the upgrade -- if I go out of my way to disable it, I got the screen to completely freeze (but at least everything did not crash down to lightdm).

FWIW, I've managed to somewhat make sure the issues were in one of this set for upgrade:
The following packages will be REMOVED:
  gnome-system-monitor libatkmm-1.6-1 libcairomm-1.0-1 libglibmm-2.4-1c2a
  libgtkmm-3.0-1 libmetacity-private0a libpangomm-1.4-1 libsigc++-2.0-0c2a
  overlay-scrollbar-gtk3
The following NEW packages will be installed:
  adwaita-icon-theme gcc-5-base jayatana libappstream-glib8
  libboost-filesystem1.58.0 libboost-system1.58.0 libcolord2 libgcab-1.0-0
  libgcrypt20 libglew1.13 libglewmx1.13 libglibmm-2.4-1v5
  libgnome-desktop-3-12 libgtop-2.0-10 libmetacity-private3a libmirclient9
  libmircommon5 libmirprotobuf3 libprotobuf-lite9v5 libprotobuf9v5
  libsigc++-2.0-0v5 libsystemd0 libyaml-0-2 session-shortcuts unity-schemas
The following packages will be upgraded:
  bamfdaemon compiz-core compiz-gnome compiz-plugins-default fontconfig-config
  gnome-accessibility-themes gnome-desktop3-data indicator-appmenu libatk1.0-0
  libatk1.0-data libbamf3-2 libcairo2 libcompizconfig0 libdecoration0
  libfontconfig1 libgail-3-0 libgpg-error0 libgtk-3-0 libgtk-3-common
  libgtop2-common libharfbuzz0b liblcms2-2 libnux-4.0-0 libnux-4.0-common
  libpango-1.0-0 libpango1.0-0 libpangocairo-1.0-0 libpangoft2-1.0-0
  libpangoxft-1.0-0 libsoup2.4-1 libstdc++6 libunity-core-6.0-9
  libunity-protocol-private0 libunity9 libwayland-client0 libwayland-cursor0
  libxkbcommon0 libxrandr2 metacity-common overlay-scrollbar ubuntu-desktop
  unity unity-services
43 upgraded, 25 newly installed, 9 to remove and 1262 not upgraded.
Need to get 17.9 MB of archives.
After this operation, 12.2 MB of additional disk space will be used.

It certainly still points to an issue in compiz and needs further debugging, but I'll let Eleni deal with it and focus on the screensaver bits in u-r-u, at least to make sure it does disable the screensaver correctly and that I was just mistaken.

description: updated
Ubuntu QA Website (ubuntuqa) wrote :

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
http://iso.qa.ubuntu.com/qatracker/reports/bugs/1555237

tags: added: iso-testing
Erick Brunzell (lbsolost) wrote :

I always disable the screensaver before I begin my upgrade tests. (screen lock = off and blank screen = never) This has happened to me both with Ubuntu and Ubuntu GNOME. Please see duplicate bug #1549067.

Tim (darkxst) wrote :

gnome-screensaver is no longer used on GNOME (its builtin as part of gnome-shell) so those quirks won't work there, I also thought Unity moved away from it as well?

on GNOME I think the following gsetting would disable screensaver
org.gnome.desktop.session idle-delay to 0

On Thu, Mar 24, 2016 at 6:19 AM, Tim <email address hidden> wrote:

> gnome-screensaver is no longer used on GNOME (its builtin as part of
> gnome-shell) so those quirks won't work there, I also thought Unity
> moved away from it as well?
>
> on GNOME I think the following gsetting would disable screensaver
> org.gnome.desktop.session idle-delay to 0
>

Nevertheless, that code should apply anyway -- every screensaver thing
should understand what xdg-screensaver tries to do, hopefully. It seems
like it generally works, and if the reset command doesn't, then
suspend/resume does.

/ Matt

Hunter Laux (jolting) wrote :

I logged into ssh and ran dpkg --configure -a before reboot and my upgrade was successful. Unfortunately, not everyone is going to be so lucky.

I think it was in the middle of configuring bluez when the server stops. It's interesting that dpkg process remains in a stopped state when X goes away.

Bruce Pieterse (octoquad) wrote :
Download full text (13.1 KiB)

I tried upgrading in Virtualbox while tailing the dist-upgrade logs and in the UI it says it is configuring libparted2 and then the UI freezes. However the apt-term.log is still receiving writes and then writing stops at bluez:

...
Setting up libparted2:amd64 (3.2-15) ...
Setting up parted (3.2-15) ...
Setting up udisks2 (2.1.7-1) ...
Setting up gvfs-common (1.27.92-1ubuntu1) ...
Setting up gvfs-libs:amd64 (1.27.92-1ubuntu1) ...
Setting up gvfs-daemons (1.27.92-1ubuntu1) ...
Setting up gvfs:amd64 (1.27.92-1ubuntu1) ...
Setting up psmisc (22.21-2.1build1) ...
Setting up python-cairo (1.8.8-2) ...
Setting up python-gi-cairo (3.18.2-2build1) ...
Setting up libxapian22v5:amd64 (1.2.22-2) ...
Setting up python-xapian (1.2.22-2build1) ...
Setting up python-apt (1.1.0~beta1build1) ...
Setting up python-pkg-resources (20.1.1-1) ...
Setting up python-chardet (2.3.0-2) ...
Setting up python-six (1.10.0-3) ...
Setting up python-debian (0.1.27ubuntu1) ...
Setting up python-dbus (1.2.0-3) ...
Setting up python-aptdaemon (1.1.1+bzr982-0ubuntu14) ...
Setting up libvte-2.91-common (0.42.5-1ubuntu1) ...
Setting up libvte-2.91-0:amd64 (0.42.5-1ubuntu1) ...
Setting up gir1.2-vte-2.91:amd64 (0.42.5-1ubuntu1) ...
Setting up aptdaemon-data (1.1.1+bzr982-0ubuntu14) ...
Setting up python-aptdaemon.gtk3widgets (1.1.1+bzr982-0ubuntu14) ...
Setting up python-lxml (3.5.0-1build1) ...
Setting up policykit-1-gnome (0.105-2ubuntu2) ...
Setting up libglib2.0-data (2.47.6-1) ...
Setting up libglib2.0-bin (2.47.6-1) ...
Setting up python-dirspec (13.10-1ubuntu1) ...
Setting up ca-certificates (20160104ubuntu1) ...
debconf: unable to initialize frontend: Dialog
debconf: (Dialog frontend requires a screen at least 13 lines tall and 31 columns wide.)
debconf: falling back to frontend: Readline
Setting up python-httplib2 (0.9.1+dfsg-1) ...
Setting up python-blinker (1.3.dfsg2-1build1) ...
Setting up python-cffi-backend (1.5.2-1ubuntu1) ...
Setting up python-enum34 (1.1.2-1) ...
Setting up python-idna (2.0-3) ...
Setting up python-ipaddress (1.0.16-1) ...
Setting up python-pyasn1 (0.1.9-1) ...
Setting up python-cryptography (1.2.2-2) ...
Setting up python-jwt (1.3.0-1) ...
Setting up python-oauthlib (1.0.3-1) ...
Setting up python-openssl (0.15.1-2build1) ...
Setting up python-twisted-bin (16.0.0-1) ...
Setting up python-zope.interface (4.1.3-1build1) ...
Setting up python-attr (15.2.0-1) ...
Setting up python-pyasn1-modules (0.0.7-0.1) ...
Setting up python-service-identity (16.0.0-2) ...
Setting up python-twisted-core (16.0.0-1) ...
Setting up python-twisted-web (16.0.0-1) ...
Setting up python-ubuntu-sso-client (13.10-0ubuntu11) ...
Setting up ubuntu-sso-client (13.10-0ubuntu11) ...
Setting up oneconf-common (0.3.9) ...
Setting up python-oneconf (0.3.9) ...
Setting up python-debtagshw (2.0.1ubuntu6) ...
Setting up ureadahead (0.100.0-19) ...
Setting up perl-modules-5.22 (5.22.1-9) ...
Setting up libgdbm3:amd64 (1.8.3-13.1) ...
Setting up libperl5.22:amd64 (5.22.1-9) ...
Setting up perl (5.22.1-9) ...
Setting up libsub-identify-perl (0.12-1build1) ...
Setting up libautodie-perl (2.29-2) ...
Setting up bash-completion (1:2.1-4.2ubuntu1) ...
Setting up initramfs-tools-bin (0.122...

Bruce Pieterse (octoquad) wrote :
Bruce Pieterse (octoquad) wrote :
Download full text (4.3 KiB)

Looks like bluez is stuck:

octoquad@ugqa:~$ ps aux | grep dpkg
root 3672 0.0 0.3 21080 7920 ? Ss 13:06 0:00 /usr/bin/dpkg --force-overwrite --status-fd 71 --configure libc-bin:amd64 locales:all libcap-ng0:amd64 libdbus-1-3:amd64 libexpat1:amd64 dbus:amd64 libnih1:amd64 libnih-dbus1:amd64 libcgmanager0:amd64 libavahi-common-data:amd64 libavahi-common3:amd64 libavahi-client3:amd64 libcupsmime1:amd64 ucf:all libpaper1:amd64 libprocps4:amd64 initscripts:amd64 procps:amd64 openssl:amd64 ssl-cert:all bc:amd64 cups-daemon:amd64 cups-server-common:all libpng12-0:amd64 libjbig0:amd64 libtiff5:amd64 libcupsfilters1:amd64 libcupsimage2:amd64 libfreetype6:amd64 libpoppler58:amd64 libqpdf17:amd64 poppler-utils:amd64 cups-filters-core-drivers:amd64 cups-core-drivers:amd64 cups-common:all cups-client:amd64 cups-bsd:amd64 libcupscgi1:amd64 libcupsppdc1:amd64 libusb-1.0-0:amd64 libijs-0.35:amd64 libjbig2dec0:amd64 poppler-data:all libgs9-common:all libgs9:amd64 ghostscript:amd64 cups-ppdc:amd64 libfontembed1:amd64 liblouis-data:all libgomp1:amd64 libfftw3-double3:amd64 liblqr-1-0:amd64 libltdl7:amd64 libxdmcp6:amd64 libxcb1:amd64 libx11-data:all libx11-6:amd64 libxext6:amd64 libicu55:amd64 libxml2:amd64 imagemagick-common:all libmagickcore-6.q16-2:amd64 libmagickwand-6.q16-2:amd64 hicolor-icon-theme:all imagemagick-6.q16:amd64 imagemagick:amd64 cups-filters:amd64 cups:amd64 libgirepository-1.0-1:amd64 gir1.2-javascriptcoregtk-4.0:amd64 uno-libs3:amd64 ure:amd64 libreoffice-common:all libreoffice-style-galaxy:all libreoffice-style-elementary:all libreoffice-style-tango:all libclucene-core1v5:amd64 libclucene-contribs1v5:amd64 libboost-date-time1.58.0:amd64 libcmis-0.5-5v5:amd64 libeot0:amd64 libdrm2:amd64 libglapi-mesa:amd64 libx11-xcb1:amd64 libxcb-dri2-0:amd64 libxcb-dri3-0:amd64 libxcb-glx0:amd64 libxcb-present0:amd64 libxcb-sync1:amd64 libxdamage1:amd64 libxfixes3:amd64 libxshmfence1:amd64 libxxf86vm1:amd64 libdrm-amdgpu1:amd64 libpciaccess0:amd64 libdrm-intel1:amd64 libdrm-nouveau2:amd64 libdrm-radeon1:amd64 libelf1:amd64 libbsd0:amd64 libedit2:amd64 libllvm3.8:amd64 libgl1-mesa-dri:amd64 libgl1-mesa-glx:amd64 libglew1.13:amd64 libhunspell-1.3-0:amd64 librevenge-0.0-0:amd64 fonts-opensymbol:all libsuitesparseconfig4.4.6:amd64 libcolamd2.9.1:amd64 lp-solve:amd64 libdbus-glib-1-2:amd64 libexttextcat-data:all libexttextcat-2.0-0:amd64 libgraphite2-3:amd64 libharfbuzz-icu0:amd64 libhyphen0:amd64 x11-common:all libice6:amd64 liblangtag-common:all liblangtag1:amd64 libmythes-1.2-0:amd64 libneon27-gnutls:amd64 libnspr4:amd64 libnss3:amd64 libnss3-nssdb:all libsm6:amd64 libxrender1:amd64 libxslt1.1:amd64 libreoffice-core:amd64 libreoffice-base-core:amd64 libetonyek-0.1-1:amd64 libmwaw-0.3-3:amd64 libodfgen-0.1-1:amd64 libboost-iostreams1.58.0:amd64 liborcus-0.10-0v5:amd64 libwps-0.4-4:amd64 libreoffice-calc:amd64 libcdr-0.1-1:amd64 libfreehand-0.1-1:amd64 libmspub-0.1-1:amd64 libpagemaker-0.0-0:amd64 libvisio-0.1-1:amd64 libwpd-0.10-10:amd64 libwpg-0.3-3:amd64 libreoffice-draw:amd64 libgdk-pixbuf2.0-common:all libgdk-pixbuf2.0-0:amd64 libgtk2.0-common:all libxi6:amd64 shared-mime-info:amd64 libgtk2.0-0:amd64 libreoffice-gtk:amd64 libreoffice-...

Read more...

Bruce Pieterse (octoquad) wrote :
Download full text (30.5 KiB)

If you kill the bluez process and run sudo dpkg --reconfigure -a a bunch of packages get setup but will get stuck on bluez again:

octoquad@ugqa:~$ sudo dpkg --configure -a
Setting up fonts-sil-abyssinica (1.500-1) ...
Setting up libquadmath0:amd64 (5.3.1-12ubuntu4) ...
Setting up linux-sound-base (1.0.25+dfsg-0ubuntu5) ...
Setting up libgtk2.0-bin (2.24.30-1ubuntu1) ...
Setting up baobab (3.18.1-1ubuntu1) ...
Setting up gnome-themes-standard:amd64 (3.18.0-2ubuntu1) ...
Setting up libkpathsea6:amd64 (2015.20160222.37495-1) ...
Setting up libnet-domain-tld-perl (1.73-1) ...
Setting up libatomic1:amd64 (5.3.1-12ubuntu4) ...
Setting up gir1.2-mediaart-2.0:amd64 (1.9.0-2) ...
Setting up libtracker-miner-1.0-0:amd64 (1.6.1-2ubuntu1) ...
Setting up gir1.2-goa-1.0:amd64 (3.18.3-1ubuntu2) ...
Setting up ncurses-term (6.0+20160213-1ubuntu1) ...
Setting up libntdb1:amd64 (1.0-9) ...
Setting up gnome-accessibility-themes (3.18.0-2ubuntu1) ...
Setting up libsbc1:amd64 (1.3-1) ...
Setting up libgupnp-dlna-2.0-3 (0.10.4-1) ...
Setting up usbmuxd (1.1.0-2) ...
Setting up gstreamer1.0-clutter (2.0.16-1) ...
Setting up libnet-libidn-perl (0.12.ds-2build2) ...
Setting up manpages (4.04-2) ...
Setting up strace (4.11-1ubuntu3) ...
Setting up realmd (0.16.2-2) ...
Setting up libsecret-common (0.18.3-1) ...
Setting up libexiv2-14:amd64 (0.25-2.1) ...
Setting up gnome-software-common (3.19.93~git20160318.55deb24-0ubuntu1) ...
Setting up powermgmt-base (1.31+nmu1) ...
Setting up libdaemon0:amd64 (0.14-6) ...
Setting up libmeanwhile1:amd64 (1.0.2-7) ...
Setting up libpoppler-glib8:amd64 (0.41.0-0ubuntu1) ...
Setting up fonts-tlwg-kinnari-ttf (1:0.6.2-2.1) ...
Setting up gir1.2-notify-0.7 (0.7.6-2svn1) ...
Setting up libopenobex2 (1.7.1-4) ...
Setting up openoffice.org-hyphenation (0.9) ...
Setting up libwebpmux1:amd64 (0.4.4-1) ...
Setting up libestr0 (0.1.10-1) ...
Setting up libtagc0:amd64 (1.9.1-2.4ubuntu1) ...
Setting up python-urllib3 (1.13.1-2) ...
Setting up libpaper-utils (1.1.24+nmu4ubuntu1) ...
Setting up libcc1-0:amd64 (5.3.1-12ubuntu4) ...
Setting up less (481-2.1) ...
Setting up libwnck-3-common (3.14.1-1) ...
Setting up gir1.2-gtkclutter-1.0:amd64 (1.6.6-1) ...
Setting up fonts-tlwg-typo-ttf (1:0.6.2-2.1) ...
Setting up make (4.1-6) ...
Setting up xserver-common (2:1.18.1-1ubuntu4) ...
Setting up libqqwing2v5:amd64 (1.3.4-1) ...
Setting up apt-transport-https (1.2.7) ...
Setting up libgfbgraph-0.2-0:amd64 (0.2.3-1) ...
Setting up libpam-gnome-keyring:amd64 (3.18.3-0ubuntu2) ...
Setting up gcc-4.8-base:amd64 (4.8.5-4ubuntu2) ...
Setting up libdouble-conversion1v5:amd64 (2.0.1-3ubuntu2) ...
Setting up libxcb-render0:amd64 (1.11.1-1ubuntu1) ...
Setting up python-crypto (2.6.1-6build1) ...
Setting up libc6-dbg:amd64 (2.23-0ubuntu2) ...
Setting up libmbim-glib4:amd64 (1.12.2-2ubuntu1) ...
Setting up fonts-cantarell (0.0.21-1) ...
Setting up python-dbus-dev (1.2.0-3) ...
Setting up libegl1-mesa-drivers:amd64 (11.1.2-1ubuntu2) ...
Setting up firefox-locale-en (45.0.1+build1-0ubuntu1) ...
Setting up libisc160:amd64 (1:9.10.3.dfsg.P4-5) ...
Setting up libpam-cap:amd64 (1:2.24-12) ...
Setting up info (6.1.0.dfsg.1-5) ...
Setting up libapt-pkg-perl (0.1.29build7)...

gerry_the_hat (gerry-the-hat) wrote :

In order to recreate things I virtualized my laptop (backup and restore into a VM). So from now an, it should be much easier :-)

However, I'm bit struck. Performed the upgrade once more with same result: system is some kind of frozen. Compared the distupgrader's logs with my previous attempt and failed to find a common pattern. At least one thing I can say for sure: does not have someting to do with screensavers, as this was disabled at all, same goes for the power saver settings. I also checked the logs from Dave's attempt without finding something similar to mine...

Well, I'm willing to rerun an upgrade, but need assistance to increase the logging level for instance. Think we need more data to dig deeper.

Steve Langasek (vorlon) wrote :

I was able to reproduce this problem in a VM. Running X and compiz under gdb, I find that compiz is dying after the X server is killed with SIGKILL.

I also see this in /var/log/syslog:

Mar 25 22:31:40 trusty-upgrade kernel: [ 7167.438442] init: lightdm post-stop process (2433) killed by KILL signal

There is no evidence in dmesg that this is OOM related. So something appears to be inappropriately kill -9'ing lightdm on upgrade.

This is not the only process in /var/log/syslog showing as killed with SIGKILL.

The first of these happens in the logs at 22:31:39. At that time, /var/log/dpkg shows these packages being configured: udisks desktop-file-utils gvfs-common gvfs-libs gvfs-daemons gvfs psmisc python-gi.

/var/log/dist-upgrade/apt-term.log doesn't show anything particularly illuminating.

Attaching the contents of /var/log.

Steve Langasek (vorlon) wrote :

The only reference I find to SIGKILL is in invoke-rc.d, which uses it only in its force-stop command and only when pid1 is systemd. So I currently have no explanation for this behavior.

Steve Langasek (vorlon) wrote :

Context in /var/log/syslog shows:

Mar 25 22:31:39 trusty-upgrade udisksd[2392]: Acquired the name org.freedesktop.UDisks2 on the system message bus
Mar 25 22:31:39 trusty-upgrade kernel: [ 7166.057324] init: dbus main process (374) killed by KILL signal
Mar 25 22:31:39 trusty-upgrade kernel: [ 7166.057335] init: dbus main process ended, respawning

This points towards udisks2 being the culprit, since the other packages that were configured at 22:31:39 happened after udisks2.

Eleni Maria Stea (hikiko) wrote :

Here's some updated info on the "compiz crash" bug:

1- the "crash" only occurs in amd64 systems (in i386 I am able to successfully dist-upgrade)
2- it is not a "compiz crash". Xserver stops because it receives the SIGKILL signal => compiz, metacity and other wm stop as a consequence of that (https://paste.ubuntu.com/15537883/ <-gdb output)
3- after the crash the kernel hasn't created an initrd => on reboot we get a kernel panic
4- andrea noticed that the bug occurs after a udev restart

my syslog in case that systemd is sending the sigkill: http://pastebin.com/VJLsg8Ce

Eleni Maria Stea (hikiko) wrote :

just noticed that steve already wrote about sigkill etc :) sorry for the double info

Changed in compiz (Ubuntu):
status: Confirmed → Invalid
no longer affects: compiz (Ubuntu)
Bachsau (bachsau) wrote :

It is completely normal that X will exit on distribution upgrades. You should never Upgrade a Linux system while running X! The normal way is to update the sources, apt-get update, apt-get upgrade and reboot, followed by apt-get dist-upgrade.

On Wed, Mar 30, 2016 at 07:07:14PM -0000, Bachsau wrote:
> It is completely normal that X will exit on distribution upgrades. You
> should never Upgrade a Linux system while running X!

This is categorically false. We have supported upgrades without leaving the
desktop session for over a decade in Ubuntu.

Changed in systemd (Ubuntu):
status: New → In Progress
Changed in dbus (Ubuntu):
status: New → In Progress
Changed in systemd (Ubuntu):
assignee: nobody → Martin Pitt (pitti)
Changed in dbus (Ubuntu):
assignee: nobody → Mathieu Trudel-Lapierre (cyphermox)
Changed in ubuntu-release-upgrader (Ubuntu):
status: In Progress → Invalid
assignee: Mathieu Trudel-Lapierre (cyphermox) → nobody
Changed in dbus (Ubuntu):
importance: Undecided → Critical
Changed in systemd (Ubuntu):
importance: Undecided → Critical
Martin Pitt (pitti) wrote :

Mathieu found the culprit, many thanks! This is due to some cleanup code in udevd which wreaks havoc when not running under systemd, as it assumes that it runs in its own cgroup (normally /system.slice/systemd-udevd.service). src/udev/udevd.c on_post() has:

                       } else if (manager->cgroup)
                                /* cleanup possible left-over processes in our cgroup */
                                cg_kill(SYSTEMD_CGROUP_CONTROLLER, manager->cgroup, SIGKILL, false, true, NULL);

But under upstart processes don't run in cgroups, and hence manager->cgroup is (almost surely) "/" and thus it kills all running processes. We need to disarm this to only clean up if the cgroup is not empty.

This can be reproduced fairly easily:

  - Boot a trusty VM (cloud-image/autopkgtest image is fine)
  - Install udisks2
  - Change apt sources to xenial, install udev from xenial
  - apt-get install --reinstall udisks2

The last command triggers udev rules and causes most processes to get killed and restarted, which can be seen nicely in their startup time and in /var/log/syslog.

Changed in systemd (Ubuntu):
milestone: none → ubuntu-16.04

There's also the issue with dbus's helper which is what causes compiz crashes taking down the session when screensaver fires (which tends to break upgrade even before udev / udisks2 clash). I'm testing a fix for that now.

Changed in dbus (Ubuntu):
milestone: none → ubuntu-16.04
Martin Pitt (pitti) wrote :
Changed in systemd (Ubuntu):
status: In Progress → Fix Committed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package systemd - 229-3ubuntu2

---------------
systemd (229-3ubuntu2) xenial; urgency=medium

  * udev.postinst: Quiesce error message if /sys/class/net/eno* does not
    exist. (LP: #1560112)
  * Fix assertion crash when processing a (broken) device without a sysfs
    path. (Closes: #819290, LP: #1560695)
  * Fix crash when shutdown is issued from a non-tty. (LP: #1553040)
  * networkd: Stay running while any non-loopback interface is up.
    (Closes: #819414)
  * udev: Don't kill peer processes if we don't run in a cgroup. This happens
    when running under/upgrading from upstart. udevd and everything
    else run in the root hierarchy on all controllers then, and the alleged
    cleanup of "our" cgroup becomes a system-wide killing spree.
    (LP: #1555237)

 -- Martin Pitt <email address hidden> Thu, 31 Mar 2016 23:11:19 +0200

Changed in systemd (Ubuntu):
status: Fix Committed → Fix Released
Tim (darkxst) wrote :

On 31/03/16 08:49, Steve Langasek wrote:
> This is categorically false. We have supported upgrades without leaving the
> desktop session for over a decade in Ubuntu.
I'm pretty sure we (Ubuntu GNOME) will loose the auth channel to gdm after upgrade, which will break the session next time the screen locks, but
should cause epic upgrade failures like this bug.

Tim (darkxst) wrote :

^ shouldn't cause epic upgrade failures

Erick Brunzell (lbsolost) wrote :

@ Tim Lunn, I typically turn off screen lock and set blank screen to never before beginning my upgrade tests but I suppose this time I should leave screen lock on and just increase blank screen to 30 minutes or so. Then if things seem even remotely buggy I'll compare the results between Ubuntu and Ubuntu GNOME release upgrades, and file a new bug report as needed.

I should have time to do that this weekend so we'll have time to address any gdm or GNOME specific issues prior to release. If you have any other thoughts about performing the next round of release-upgrade tests please send me a PM.

no longer affects: ubuntu-release-upgrader (Ubuntu)

Mathew, please, pretty please stop removing tasks, even if they're marked Invalid.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package dbus - 1.10.6-1ubuntu3

---------------
dbus (1.10.6-1ubuntu3) xenial; urgency=medium

  * debian/dbus.preinst: divert the dbus-daemon-launch-helper if upgrading
    from < 1.9.4-2~. This will make sure we keep the setuid bit during upgrade.
    (LP: #1555237)
  * debian/dbus.postinst: remove diversion.

 -- Mathieu Trudel-Lapierre <email address hidden> Thu, 31 Mar 2016 15:07:46 -0400

Changed in dbus (Ubuntu):
status: In Progress → Fix Released
Kev Bowring (flocculant) wrote :

ftr - got an upgraded from 14.04 system here now (vanilla virtual - but even so ...)

Tim (darkxst) wrote :

Upgrades are working for Ubuntu GNOME, but screensaver issues exist, see Bug 1565177 w/ proposed fix

fernando biondi (fbiondip) wrote :

painful dist- upgrade as OFTEN, files and programs missing after various futile attempts . when will come the day for no-issue Ubuntu upgrade?

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers