Failed unmounting Mount unit for nvidia support in snappy

[hectorsales]

Hello after upgrade nvidia drivers(proposed repo) on my laptop..i can not login successful (loop at login). I try CTRL+ALT+F1 and purge nvidia-361..but this occurs :

Job for var-lib-snapd-lib-gl.mount failed. See "systemctl status var-lib-snapd-lib-gl.mount" and "journalctl -xe" for details.

..after type:

$ journalct -xe

-- Unit var-lib-snapd-lib-gl.mount has begun shutting down.
jun 04 00:26:17 Unity-X550JF umount[4139]: umount: /var/lib/snapd/lib/gl: target is busy
jun 04 00:26:17 Unity-X550JF umount[4139]: (In some cases useful info about processes that
jun 04 00:26:17 Unity-X550JF umount[4139]: use the device is found by lsof(8) or fuser(1).)
jun 04 00:26:17 Unity-X550JF systemd[1]: var-lib-snapd-lib-gl.mount: Mount process exited, code=exited status=32
jun 04 00:26:17 Unity-X550JF systemd[1]: Failed unmounting Mount unit for nvidia support in snappy.
-- Subject: Unit var-lib-snapd-lib-gl.mount has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit var-lib-snapd-lib-gl.mount has finished shutting down.
jun 04 00:26:17 Unity-X550JF sudo[4094]: pam_unix(sudo:session): session closed for user root

Regards

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Bart Willemsen (b-willemsen8)]

Can someone please tell how to fix this? I can't login to my system now and I need to get work done.

[disz (d-schlabing)]

I was able to downgrade the nvidia packages by first stopping sddm via:

sudo systemctl stop sddm.service

I am running kubuntu though. If you're on Ubuntu it is not sddm but lightdm, I guess.

Then:

sudo apt install {nvidia-opencl-icd-361,nvidia-352,nvidia-361}=361.42-0ubuntu2

And reboot!

[Francois Thirioux (fthx)]

I'm not an Ubuntu noob but this combination of Nvidia 361.45 and snapd 2.06 upgrades is very nasty.

I was not able to downgrade or remove the 361.45 packages since the bug description's trick happens. Even recovery mode was not sufficient. I finally found that booting with upstart instead of systemd allows to downgrade/remove when upstart lead the boot to a console.

[Lilian A. Moraru (lilianmoraru)]

Same issue here on Ubuntu 16.04 LTS(Unity).
Could not login. What I did is stop "lightdm", after which "var-lib-snapd-lib-gl.mount", after that I purged "nvidia*", removed "snapd" and removed the folder "/var/lib/snapd", in the hopes that it won't start this problematic service any more.
I restarted and now it could login but compiz does not load(windows without frames and --replace does not work).
If I try to install nvidia, it does not allow me because it tries to remove "libcuda1-364" which fails to be removed because it can't find "var-lib-snapd-lib-gl.mount", tried forcing and everything, don't know what to do with it any more...
The biggest issue is that this is my main and only machine in the house + it is a full encryption system, I can't just start installing Ubuntu all over again, I have a lot of data...

I said to myself I won't ever install Ubuntu as the only system since I always get hit with these issue within' 1-2 months of normal usage(every Ubuntu version I had breaks at some point after an update - at work we have 12.04.5 and one day it broke a few hundred of dev machines with an update to the Linux kernel, luckily you can switch to an older version in grub). I thought 16.04 LTS would be stable at this point and risked...

[Matteo Iervasi (matteoiervasi)]

I solved this issue by logging in TTY1, purging snapd and everything nvidia related (sudo apt purge snapd nvidia* && sudo apt autoremove --purge, libcuda1-361 couldn't be removed but that's not a problem), then I ran the following script which resets all snaps, https://github.com/zyga/devtools/blob/master/reset-state (from the TTY1 use "curl https://raw.githubusercontent.com/zyga/devtools/master/reset-state >> reset.sh && chmod +x reset.sh && sudo ./reset.sh").
After all of these I reinstalled nvidia drivers (sudo apt install nvidia-361) and created the xorg conf (sudo nvidia-xconvig).
Hope this helped :)

[Oliver Grawert (ogra)]

Please note that the proposed archive is explicitly for testing package upgrades before they go into xenial-updates (this one seemingly failed). If you do not want to participate in testing,do not enable the proposed repository in your software sources (note that it is off by default).

[Oliver Grawert (ogra)]

The status for this SRU test is tracked in bug #1588192

[Lilian A. Moraru (lilianmoraru)]

I managed to remove "libcuda" and every package that was depending on ...snapd...mount by modifying "/var/lib/dpkg/info/[the package].prerm" and putting "exit 0" right after the shebang.
I removed nvidia, tried reinstalling different versions and configuring in different ways 4 times... and it didn't work(can login but no window frame).
Tried to install nouveau instead and still no window frame.
Tried reinstalling ubuntu-desktop, unity, compiz... still no luck.
unity/compiz --replace. -- no luck.

I will try to uninstall nouveau and attempt Matteo's solution.

[Lilian A. Moraru (lilianmoraru)]

Same behavior with Matteo's solution.
Logs in but there is no compiz(no Unity and if I open a terminal, it does not have a frame)...

[Matteo Iervasi (matteoiervasi)]

If now you're able to log-in but there's no frame, it usually means it's a compiz problem :D
Back with Ubuntu 10.04-12.04, when I played very frequently with Compiz settings, I messed up the system many times, and the solution has always been to purge everything related to compiz/unity (even the settings in the home directory, with rm -rf ~/.compiz-1 ~/.config/compiz-1 ) and then reinstalling it (maybe with the ubuntu-desktop metapackage).

Also, make sure nvidia's drivers has been correctly installed and they has been loaded (look at the output of lshw in the display section)

[Mateusz Stachowski (stachowski-mateusz)]

It should be noted that this problem only affects people using 361 drivers.

Those of us on Legacy 340 drivers have working Krita and desktop session.

[Lilian A. Moraru (lilianmoraru)]

Mateusz, I have the same issue with 364 and 367, regardless.
I guess 36*?

Thanks Matteo for the suggestion, I will try it and report the results.
Btw, in the meantime I installed "lubuntu-desktop" and it works alright(not the solution I would like but if it helps other people...).

[Lilian A. Moraru (lilianmoraru)]

It didn't work. Again could not login after doing that...
Still works logging in LXDE.

[Marcin Sągol (soee)]

I can confirm this problem on Kubnut 16.10, nvidia-367 driver. Stopping sddm and removing driver helped.

[Marcin Sągol (soee)]

I shuold add that after the update, sddm stopped working on Kubuntu 16.10 - only black screen. Without stopping sddm i couldn't uninstall the driver or change version.

so if someone want to have sddm login screen back:

sudo systemctl stop sddm
sudo apt remove --purge nvidia*
sudo systemctl start sddm

and you are back on login screen,

[Rik Mills (rikmills)]

Also hit this bug this morning. Oddly when installing the gpu ppa update to fix/revert the problem, rather than with the original affected package.

preventing any action of /var/lib/dpkg/info/nvidia-opencl-icd-364.prerm allowed old nvidia packages to be fully purged and updated/fixed ones installed

[Bart Willemsen (b-willemsen8)]

I was able to uninstall the driver by first stopping lightdm with "sudo systemctl stop lightdm". After reverting back to Nouveau I was able to log in, but Unity didn't start. (no window borders or launcher, etc.) I fixed this by doing the following:

sudo dconf reset -f /org/compiz/
sudo setsid unity

This immediately caused Unity to pop back into place! But I did a reboot just to be sure and everything is working great again now. :)

[Michael Marley (mamarley)]

For anyone encountering this bug, a workaround to allow the removal of the broken package is to first stop your X display manager, then use remove the packages manually on the command line, and finally reboot. This will switch you back to Nouveau but at least give you a working system from which you can reinstall the older NVIDIA package that is not afflicted by the bug.

[Michael Vogt (mvo)]

This need some more investigation (why unity fails to start) but https://github.com/snapcore/snapd/pull/1271 should at least prevent the "mount point busy" error.

[Lilian A. Moraru (lilianmoraru)]

Finally, Bart's(#18) suggestion worked.
I get a lot of errors from the removed nvidia packages but I guess I can live with that...

Now I only have to figure out how to remove Lubuntu without destroying my system.

[3vi1 (launchpad-net-eternaldusk)]

> For anyone encountering this bug, a workaround to allow the removal of the broken package is to first stop your X display manager, then use remove the packages manually on the command line, and finally reboot.

That did not work in my case - even booting into recovery mode, never starting X.

I had to go edit two of the prerm scripts so that they didn't try to stop var-lib-snapd-lib-gl.mount. Otherwise, that failure would block libcuda1* and nvidia-opencl-icd*.

[Adrien Béraud (aberaud)]

3vi1 solution at #22 worked for me.

I had to edit
/var/lib/dpkg/info/nvidia*.prerm
and
/var/lib/dpkg/info/libcuda*.prerm
and removed the automatically added section (last 3-4 lines).

[Lilian A. Moraru (lilianmoraru)]

@Andrie, you can just put "exit 0" right before the shebang.

[Abhishek Bhatia (abhigenie92)]

I can't remove the nvidia packages at first to reinstall later. I am using Kubuntu 16.04.

abhishek ~ $ sudo apt-get purge libcuda1-367
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
  bbswitch-dkms fonts-roboto-hinted lib32gcc1 libechonest2.3 libgsasl7 libjansson4 libjdns2 libjreen1
  liblucene++0v5 libntlm0 libqjdns-qt4-2 libqtkeychain0 libquazip1 libxnvctrl0 screen-resolution-extra
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED:
  libcuda1-367*
0 upgraded, 0 newly installed, 1 to remove and 1 not upgraded.
After this operation, 35.6 MB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 294457 files and directories currently installed.)
Removing libcuda1-367 (367.18-0ubuntu0~gpu16.04.2) ...
Failed to stop var-lib-snapd-lib-gl.mount: Unit var-lib-snapd-lib-gl.mount not loaded.
dpkg: error processing package libcuda1-367 (--purge):
 subprocess installed pre-removal script returned error exit status 5
Failed to get unit file state for var-lib-snapd-lib-gl.mount: No such file or directory
var-lib-snapd-lib-gl.mount is a disabled or a static unit, not starting it.
Processing triggers for libc-bin (2.23-0ubuntu3) ...
Errors were encountered while processing:
 libcuda1-367
E: Sub-process /usr/bin/dpkg returned an error code (1)
abhishek ~ $ sudo apt-get purge nvidia-opencl-icd-367
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
  bbswitch-dkms fonts-roboto-hinted lib32gcc1 libechonest2.3 libgsasl7 libjansson4 libjdns2 libjreen1
  liblucene++0v5 libntlm0 libqjdns-qt4-2 libqtkeychain0 libquazip1 libxnvctrl0 screen-resolution-extra
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED:
  nvidia-opencl-icd-367*
0 upgraded, 0 newly installed, 1 to remove and 1 not upgraded.
After this operation, 36.4 MB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 294457 files and directories currently installed.)
Removing nvidia-opencl-icd-367 (367.18-0ubuntu0~gpu16.04.2) ...
Failed to stop var-lib-snapd-lib-gl.mount: Unit var-lib-snapd-lib-gl.mount not loaded.
dpkg: error processing package nvidia-opencl-icd-367 (--purge):
 subprocess installed pre-removal script returned error exit status 5
Failed to get unit file state for var-lib-snapd-lib-gl.mount: No such file or directory
var-lib-snapd-lib-gl.mount is a disabled or a static unit, not starting it.
Processing triggers for libc-bin (2.23-0ubuntu3) ...
Errors were encountered while processing:
 nvidia-opencl-icd-367
E: Sub-process /usr/bin/dpkg returned an error code (1)
abhishek ~ $

[Rik Mills (rikmills)]

@Abhishek

As said in comment 23, if you edit

/var/lib/dpkg/info/libcuda1-367.prerm

and

/var/lib/dpkg/info/nvidia-opencl-icd-367.prerm

to insert 'exit 0' before the active script lines, then those errors should be bypassed allowing you to uninstall the faulty packages.

[Brian Murray (brian-murray)]

Hello hectorsales, or anyone else affected,

Accepted snapd into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/snapd/2.0.7 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Lilian A. Moraru (lilianmoraru)]

@brian-murray I see nobody came with a response.
Personally I cannot test this since I banged my head of a wall to fix this issue and now I don't want to break my system and get into that hell all over again just to test the issue(I am not even sure what are exactly the steps to test this. Do you need this package before the nVidia update? Is it ok that you removed nVidia and installed the package?...).
My guess is that the other guys don't want that either.

This is not the kind of a bug that people can live with and that's why they don't already have a fix for it...

Best chance is that people that didn't update yet hit the issue... and don't want to fix it with the suggestions in here...

[Oliver Grawert (ogra)]

@lilianmoraru since the (broken and fixed) packages went into the xenial-proposed repo, only people that actively opted in to test pre-upgrades (including potentially broken ones) to give feedback to the developers will actually get this update.
xenial-proposed is nothing that anyone should have enabled permanently, packages get tested, fixed and verified there before they move to xenial-updates.
people using a standard ubuntu install where proposed is not enabled will never hit this issue.

[Francois Thirioux (fthx)]

As Lilian, I'll rather wait to test again since in Yakkety nvidia package is also updated, not only snapd :

nvidia-graphics-drivers-361 (361.45.11-0ubuntu3) yakkety; urgency=medium

  * debian/rules, debian/templates/control.in,
    debian/templates/nvidia-graphics-drivers.dirs.in,
    debian/templates/nvidia-graphics-drivers.install.in,
    debian/templates/nvidia-graphics-drivers.postinst.in,
    debian/templates/var-lib-snapd-lib-gl.mount.in,
    - Revert bind mount unit for snappy, as it causes a regression. <<<****************************

 -- Alberto Milone <email address hidden> Mon, 06 Jun 2016 15:53:32 +0200

[Michael Vogt (mvo)]

I can no longer reproduce the unmount issue with the snapd from -proposed.

[Launchpad Janitor (janitor)]

This bug was fixed in the package snapd - 2.0.8

---------------
snapd (2.0.8) xenial; urgency=medium

  * New upstream release: LP: #1589534
    - debian: make `snap refresh` times more random (LP: #1537793)
    - cmd: ExecInCoreSnap looks in "core" snap first, and only in
      "ubuntu-core" snap if rev>125.
    - cmd/snap: have 'snap list' display helper message on stderr
      (LP: #1587445)
    - snap: make app names more restrictive.

 -- Michael Vogt <email address hidden> Wed, 08 Jun 2016 07:56:58 +0200

[David Daynard (nardholio)]

This issue is confusing me. I'm on yakkety running Cinammon and the nvidia drivers will not update due to this error. I do not have snapd installed. Why is a .deb file trying to use Snap? I tried installing snapd as a test and it still errors out on this. I ran the Snap reset script (which did nothing since I don't have any Snaps installed) and it still didn't work. Should I open this as a separate bug?

[Launchpad Janitor (janitor)]

This bug was fixed in the package snapd - 2.11+16.10

---------------
snapd (2.11+16.10) yakkety; urgency=medium

  * New upstream release: LP: #1605303
    - increase version number to reflect the nature of the update
      better
    - store, daemon, client, cmd/snap, docs/rest.md: adieu search
      grammar
    - debian: move snapd.refresh.timer into timers.target
    - snapstate: add daemon-reload to fix autopkgtest on yakkety
    - Interfaces: hardware-observe
    - snap: rework the output after a snap operation
    - daemon, cmd/snap: refresh --devmode
    - store, daemon, client, cmd/snap: implement `snap find --private`
    - tests: add network-observe interface spread test
    - interfaces/builtin: allow getsockopt for connected x11 plugs
    - osutil: check for nogrup instead of adm
    - store: small cleanups (more needed)
    - snap/squashfs: fix test not to hardcode snap size
    - client,cmd/snap: cleanup cmd/snap test suite, add extra args
      testThis cleans up the cmd/snap test suite:
    - wrappers: map "never" restart condition to "no."
    - wrappers: run update-desktop-database after add/remove of desktop
      files
    - release: work around elementary mistake
    - many: remove all traces of channel from the buying codepath
    - store: kill setUbuntuStoreHeaders
    - docs: add payment methods documentation
    - many: present user with a choice of payment backends
    - asserts: add cross checks for snap asserts
    - cmd/snap,cmd/snap-exec: support running hooks via snap-exec.
    - tests: improve snap run symlink tests
    - tests: add content sharing interface spread test
    - store & many: a mechanical branch shortening store names
    - snappy: remove old snappy pkg
    - overlord/snapstate: kill flagscompat
    - overlord/snapstate, daemon, client, cmd/snap: devmode override
      (aka confined)
    - tests: extend refresh test to talk to the staging and production
      stores
    - asserts,daemon: cross checks for account and account-key
      assertions
    - client: existing JSON fixtures uses tabs for indentation
    - snap-exec: add proper integration test for snap-exec
    - spread.yaml, tests: replace hello-world with test-snapd-tools
    - tests: add locale-control interface spread test
    - tests: add mount-observe interface spread test
    - tests: add system-observe interface spread test
    - many: add AuthContext to mediate user updates to the state
    - store/auth: add helper for the macaroon refresh endpoint
    - cmd: add buy command
    - overlord: switch snapstate.Update to use ListRefresh (aka
      /snaps/metadata)
    - snap-exec: fix silly off-by-one error
    - tests: stop using hello-world.echo in the tests
    - tests: add env command to test-snapd-tools
    - classic: remove (most of) "classic" mode, this is implemented as a
      snap now
    - many: remove snapstate.Candidate and other cleanups
    - many: removed authenticator, store gets a user instead
    - asserts: fix minor doc comment typo
    - snap: ensure unknown arguments to `snap run` are ignored
    - overlord/auth: add Device/SetDevice to persist device identity in
      state
    - overlord: make SyncBoot work aga...