| 2018-11-16 16:59:13 |
Balint Reczey |
bug |
|
|
added bug |
| 2018-11-19 13:05:46 |
Julian Andres Klode |
affects |
python-apt (Ubuntu) |
unattended-upgrades (Ubuntu) |
|
| 2018-11-19 22:29:41 |
Balint Reczey |
unattended-upgrades (Ubuntu): status |
New |
In Progress |
|
| 2018-11-19 22:29:44 |
Balint Reczey |
unattended-upgrades (Ubuntu): assignee |
|
Balint Reczey (rbalint) |
|
| 2018-11-19 22:29:50 |
Balint Reczey |
unattended-upgrades (Ubuntu): importance |
Undecided |
High |
|
| 2018-11-26 12:20:24 |
Balint Reczey |
description |
While triaging LP: #1803587 I found that originally autoremovable packages were handled as newly autoremovable ones by unattended-upgrades due to not finding all of them at the beginning of u-u's run.
The root cause seems to be cache.clear() resetting pkg.is_auto_removable to False under some circumstances I haven't fully narrowed down.
The reproduction is as follows:
Set up a Bionic (or later) system with autoremovable packages, packages upgradable from -security and blacklist at least one upgradable package and apply the following patch to u-u:
root@bb-1803587:~# diff -Naur /usr/bin/unattended-upgrade.orig /usr/bin/unattended-upgrade
--- /usr/bin/unattended-upgrade.orig 2018-11-16 16:17:58.522583254 +0000
+++ /usr/bin/unattended-upgrade 2018-11-16 16:36:12.226675870 +0000
@@ -948,7 +948,9 @@
def rewind_cache(cache, pkgs_to_upgrade):
# type: (apt.Cache, List[apt.Package]) -> None
""" set the cache back to the state with packages_to_upgrade """
+ print([pkg.name for pkg in cache if pkg.is_auto_removable])
cache.clear()
+ print([pkg.name for pkg in cache if pkg.is_auto_removable])
for pkg2 in pkgs_to_upgrade:
pkg2.mark_install(from_user=not pkg2.is_auto_installed)
if cache.broken_count > 0:
Run u-u to observe cache.clear() resetting the list of autoremovable packages:
~# /usr/bin/unattended-upgrade --dry-run --verbose --debug
Initial blacklisted packages: systemd
Initial whitelisted packages:
Starting unattended upgrades script
Allowed origins are: o=Ubuntu,a=bionic, o=Ubuntu,a=bionic-security, o=UbuntuESM,a=bionic
Using (^linux-image|^linux-headers|^linux-image-extra|^linux-modules|^linux-modules-extra|^linux-signed-image|^kfreebsd-image|^kfreebsd-headers|^gnumach-image|^.*-modules|^.*-kernel|^linux-backports-modules-.*|^linux-modules-.*|^linux-tools|^linux-cloud-tools) regexp to find kernel packages
Using (^linux-image.*4.15.0-38-generic|^linux-headers.*4.15.0-38-generic|^linux-image-extra.*4.15.0-38-generic|^linux-modules.*4.15.0-38-generic|^linux-modules-extra.*4.15.0-38-generic|^linux-signed-image.*4.15.0-38-generic|^kfreebsd-image.*4.15.0-38-generic|^kfreebsd-headers.*4.15.0-38-generic|^gnumach-image.*4.15.0-38-generic|4.15.0-38-generic.*-modules|4.15.0-38-generic.*-kernel|^linux-backports-modules-.*.*4.15.0-38-generic|^linux-modules-.*.*4.15.0-38-generic|^linux-tools.*4.15.0-38-generic|^linux-cloud-tools.*4.15.0-38-generic) regexp to find running kernel packages
Checking: apport ([<Origin component:'main' archive:'bionic-updates' origin:'Ubuntu' label:'Ubuntu' site:'archive.ubuntu.com' isTrusted:True>])
adjusting candidate version: apport=2.20.9-0ubuntu7.1
...
Checking: libnss-systemd ([<Origin component:'main' archive:'bionic-updates' origin:'Ubuntu' label:'Ubuntu' site:'archive.ubuntu.com' isTrusted:True>, <Origin component:'main' archive:'bionic-security' origin:'Ubuntu' label:'Ubuntu' site:'security.ubuntu.com' isTrusted:True>])
skipping blacklisted package systemd
pkg systemd package has been blacklisted
sanity check failed
['libfreetype6']
[]
... |
[Impact]
* Originally autoremovable packages can be removed as newly autoremovable ones by unattended-upgrades
* This can surprise users potentially removing packages which are needed for the system's operation.
[Test Case]
* WIP, see Original Bug Test
[Regression Potential]
* Unattended-upgrades may use more CPU-time for operation but I did not observe a significant increase. Autopkgtest measures u-u's performance thus if this regression occurs, we can observe it easily.
* Due to the code changes u-u may still remove already autoremovable packages or fail to remove newly autoremovable ones in default configuration, but since the code became simpler with the change by eliminating an optimization this regression is unlikely to take place.
[Original Bug Text]
While triaging LP: #1803587 I found that originally autoremovable packages were handled as newly autoremovable ones by unattended-upgrades due to not finding all of them at the beginning of u-u's run.
The root cause seems to be cache.clear() resetting pkg.is_auto_removable to False under some circumstances I haven't fully narrowed down.
Set up a Bionic (or later) system with autoremovable packages, packages upgradable from -security and blacklist at least one upgradable package and apply the following patch to u-u:
root@bb-1803587:~# diff -Naur /usr/bin/unattended-upgrade.orig /usr/bin/unattended-upgrade
--- /usr/bin/unattended-upgrade.orig 2018-11-16 16:17:58.522583254 +0000
+++ /usr/bin/unattended-upgrade 2018-11-16 16:36:12.226675870 +0000
@@ -948,7 +948,9 @@
def rewind_cache(cache, pkgs_to_upgrade):
# type: (apt.Cache, List[apt.Package]) -> None
""" set the cache back to the state with packages_to_upgrade """
+ print([pkg.name for pkg in cache if pkg.is_auto_removable])
cache.clear()
+ print([pkg.name for pkg in cache if pkg.is_auto_removable])
for pkg2 in pkgs_to_upgrade:
pkg2.mark_install(from_user=not pkg2.is_auto_installed)
if cache.broken_count > 0:
Run u-u to observe cache.clear() resetting the list of autoremovable packages:
~# /usr/bin/unattended-upgrade --dry-run --verbose --debug
Initial blacklisted packages: systemd
Initial whitelisted packages:
Starting unattended upgrades script
Allowed origins are: o=Ubuntu,a=bionic, o=Ubuntu,a=bionic-security, o=UbuntuESM,a=bionic
Using (^linux-image|^linux-headers|^linux-image-extra|^linux-modules|^linux-modules-extra|^linux-signed-image|^kfreebsd-image|^kfreebsd-headers|^gnumach-image|^.*-modules|^.*-kernel|^linux-backports-modules-.*|^linux-modules-.*|^linux-tools|^linux-cloud-tools) regexp to find kernel packages
Using (^linux-image.*4.15.0-38-generic|^linux-headers.*4.15.0-38-generic|^linux-image-extra.*4.15.0-38-generic|^linux-modules.*4.15.0-38-generic|^linux-modules-extra.*4.15.0-38-generic|^linux-signed-image.*4.15.0-38-generic|^kfreebsd-image.*4.15.0-38-generic|^kfreebsd-headers.*4.15.0-38-generic|^gnumach-image.*4.15.0-38-generic|4.15.0-38-generic.*-modules|4.15.0-38-generic.*-kernel|^linux-backports-modules-.*.*4.15.0-38-generic|^linux-modules-.*.*4.15.0-38-generic|^linux-tools.*4.15.0-38-generic|^linux-cloud-tools.*4.15.0-38-generic) regexp to find running kernel packages
Checking: apport ([<Origin component:'main' archive:'bionic-updates' origin:'Ubuntu' label:'Ubuntu' site:'archive.ubuntu.com' isTrusted:True>])
adjusting candidate version: apport=2.20.9-0ubuntu7.1
...
Checking: libnss-systemd ([<Origin component:'main' archive:'bionic-updates' origin:'Ubuntu' label:'Ubuntu' site:'archive.ubuntu.com' isTrusted:True>, <Origin component:'main' archive:'bionic-security' origin:'Ubuntu' label:'Ubuntu' site:'security.ubuntu.com' isTrusted:True>])
skipping blacklisted package systemd
pkg systemd package has been blacklisted
sanity check failed
['libfreetype6']
[]
... |
|
| 2018-11-26 13:42:57 |
Launchpad Janitor |
unattended-upgrades (Ubuntu): status |
In Progress |
Fix Released |
|
| 2018-11-26 17:19:39 |
Łukasz Zemczak |
unattended-upgrades (Ubuntu Cosmic): status |
New |
Fix Committed |
|
| 2018-11-26 17:19:41 |
Łukasz Zemczak |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2018-11-26 17:19:45 |
Łukasz Zemczak |
bug |
|
|
added subscriber SRU Verification |
| 2018-11-26 17:19:48 |
Łukasz Zemczak |
tags |
|
verification-needed verification-needed-cosmic |
|
| 2018-11-26 17:24:20 |
Łukasz Zemczak |
unattended-upgrades (Ubuntu Bionic): status |
New |
Fix Committed |
|
| 2018-11-26 17:24:24 |
Łukasz Zemczak |
tags |
verification-needed verification-needed-cosmic |
verification-needed verification-needed-bionic verification-needed-cosmic |
|
| 2018-11-27 20:59:53 |
Balint Reczey |
description |
[Impact]
* Originally autoremovable packages can be removed as newly autoremovable ones by unattended-upgrades
* This can surprise users potentially removing packages which are needed for the system's operation.
[Test Case]
* WIP, see Original Bug Test
[Regression Potential]
* Unattended-upgrades may use more CPU-time for operation but I did not observe a significant increase. Autopkgtest measures u-u's performance thus if this regression occurs, we can observe it easily.
* Due to the code changes u-u may still remove already autoremovable packages or fail to remove newly autoremovable ones in default configuration, but since the code became simpler with the change by eliminating an optimization this regression is unlikely to take place.
[Original Bug Text]
While triaging LP: #1803587 I found that originally autoremovable packages were handled as newly autoremovable ones by unattended-upgrades due to not finding all of them at the beginning of u-u's run.
The root cause seems to be cache.clear() resetting pkg.is_auto_removable to False under some circumstances I haven't fully narrowed down.
Set up a Bionic (or later) system with autoremovable packages, packages upgradable from -security and blacklist at least one upgradable package and apply the following patch to u-u:
root@bb-1803587:~# diff -Naur /usr/bin/unattended-upgrade.orig /usr/bin/unattended-upgrade
--- /usr/bin/unattended-upgrade.orig 2018-11-16 16:17:58.522583254 +0000
+++ /usr/bin/unattended-upgrade 2018-11-16 16:36:12.226675870 +0000
@@ -948,7 +948,9 @@
def rewind_cache(cache, pkgs_to_upgrade):
# type: (apt.Cache, List[apt.Package]) -> None
""" set the cache back to the state with packages_to_upgrade """
+ print([pkg.name for pkg in cache if pkg.is_auto_removable])
cache.clear()
+ print([pkg.name for pkg in cache if pkg.is_auto_removable])
for pkg2 in pkgs_to_upgrade:
pkg2.mark_install(from_user=not pkg2.is_auto_installed)
if cache.broken_count > 0:
Run u-u to observe cache.clear() resetting the list of autoremovable packages:
~# /usr/bin/unattended-upgrade --dry-run --verbose --debug
Initial blacklisted packages: systemd
Initial whitelisted packages:
Starting unattended upgrades script
Allowed origins are: o=Ubuntu,a=bionic, o=Ubuntu,a=bionic-security, o=UbuntuESM,a=bionic
Using (^linux-image|^linux-headers|^linux-image-extra|^linux-modules|^linux-modules-extra|^linux-signed-image|^kfreebsd-image|^kfreebsd-headers|^gnumach-image|^.*-modules|^.*-kernel|^linux-backports-modules-.*|^linux-modules-.*|^linux-tools|^linux-cloud-tools) regexp to find kernel packages
Using (^linux-image.*4.15.0-38-generic|^linux-headers.*4.15.0-38-generic|^linux-image-extra.*4.15.0-38-generic|^linux-modules.*4.15.0-38-generic|^linux-modules-extra.*4.15.0-38-generic|^linux-signed-image.*4.15.0-38-generic|^kfreebsd-image.*4.15.0-38-generic|^kfreebsd-headers.*4.15.0-38-generic|^gnumach-image.*4.15.0-38-generic|4.15.0-38-generic.*-modules|4.15.0-38-generic.*-kernel|^linux-backports-modules-.*.*4.15.0-38-generic|^linux-modules-.*.*4.15.0-38-generic|^linux-tools.*4.15.0-38-generic|^linux-cloud-tools.*4.15.0-38-generic) regexp to find running kernel packages
Checking: apport ([<Origin component:'main' archive:'bionic-updates' origin:'Ubuntu' label:'Ubuntu' site:'archive.ubuntu.com' isTrusted:True>])
adjusting candidate version: apport=2.20.9-0ubuntu7.1
...
Checking: libnss-systemd ([<Origin component:'main' archive:'bionic-updates' origin:'Ubuntu' label:'Ubuntu' site:'archive.ubuntu.com' isTrusted:True>, <Origin component:'main' archive:'bionic-security' origin:'Ubuntu' label:'Ubuntu' site:'security.ubuntu.com' isTrusted:True>])
skipping blacklisted package systemd
pkg systemd package has been blacklisted
sanity check failed
['libfreetype6']
[]
... |
[Impact]
* Originally autoremovable packages can be removed as newly autoremovable ones by unattended-upgrades
* This can surprise users potentially removing packages which are needed for the system's operation.
[Test Case]
* The buggy u-u version will remove the previously autoremovable zsh at the end, the fixed one does not.
$ sudo debootstrap bionic uu-test-bionic-1803749
...
$ sudo chroot uu-test-bionic-1803749
# apt install unattended-upgrades zsh
...
# echo "Unattended-Upgrade::Package-Blacklist {"libs";}" > /etc/apt/apt.conf.d/51unattended-upgrades
# echo "deb http://archive.ubuntu.com/ubuntu bionic-security main" >> /etc/apt/sources.list
# echo "deb http://archive.ubuntu.com/ubuntu bionic-updates main" >> /etc/apt/sources.list
# apt update -qq
# unattended-upgrades --dry-run --verbose --debug
...
All upgrades installed
marking zsh for removal
marking zsh-common for removal
Packages that were successfully auto-removed:
Packages that are kept back:
InstCount=0 DelCount=0 BrokenCount=0
#
[Regression Potential]
* Unattended-upgrades may use more CPU-time for operation but I did not observe a significant increase. Autopkgtest measures u-u's performance thus if this regression occurs, we can observe it easily.
* Due to the code changes u-u may still remove already autoremovable packages or fail to remove newly autoremovable ones in default configuration, but since the code became simpler with the change by eliminating an optimization this regression is unlikely to take place.
[Original Bug Text]
While triaging LP: #1803587 I found that originally autoremovable packages were handled as newly autoremovable ones by unattended-upgrades due to not finding all of them at the beginning of u-u's run.
The root cause seems to be cache.clear() resetting pkg.is_auto_removable to False under some circumstances I haven't fully narrowed down.
Set up a Bionic (or later) system with autoremovable packages, packages upgradable from -security and blacklist at least one upgradable package and apply the following patch to u-u:
root@bb-1803587:~# diff -Naur /usr/bin/unattended-upgrade.orig /usr/bin/unattended-upgrade
--- /usr/bin/unattended-upgrade.orig 2018-11-16 16:17:58.522583254 +0000
+++ /usr/bin/unattended-upgrade 2018-11-16 16:36:12.226675870 +0000
@@ -948,7 +948,9 @@
def rewind_cache(cache, pkgs_to_upgrade):
# type: (apt.Cache, List[apt.Package]) -> None
""" set the cache back to the state with packages_to_upgrade """
+ print([pkg.name for pkg in cache if pkg.is_auto_removable])
cache.clear()
+ print([pkg.name for pkg in cache if pkg.is_auto_removable])
for pkg2 in pkgs_to_upgrade:
pkg2.mark_install(from_user=not pkg2.is_auto_installed)
if cache.broken_count > 0:
Run u-u to observe cache.clear() resetting the list of autoremovable packages:
~# /usr/bin/unattended-upgrade --dry-run --verbose --debug
Initial blacklisted packages: systemd
Initial whitelisted packages:
Starting unattended upgrades script
Allowed origins are: o=Ubuntu,a=bionic, o=Ubuntu,a=bionic-security, o=UbuntuESM,a=bionic
Using (^linux-image|^linux-headers|^linux-image-extra|^linux-modules|^linux-modules-extra|^linux-signed-image|^kfreebsd-image|^kfreebsd-headers|^gnumach-image|^.*-modules|^.*-kernel|^linux-backports-modules-.*|^linux-modules-.*|^linux-tools|^linux-cloud-tools) regexp to find kernel packages
Using (^linux-image.*4.15.0-38-generic|^linux-headers.*4.15.0-38-generic|^linux-image-extra.*4.15.0-38-generic|^linux-modules.*4.15.0-38-generic|^linux-modules-extra.*4.15.0-38-generic|^linux-signed-image.*4.15.0-38-generic|^kfreebsd-image.*4.15.0-38-generic|^kfreebsd-headers.*4.15.0-38-generic|^gnumach-image.*4.15.0-38-generic|4.15.0-38-generic.*-modules|4.15.0-38-generic.*-kernel|^linux-backports-modules-.*.*4.15.0-38-generic|^linux-modules-.*.*4.15.0-38-generic|^linux-tools.*4.15.0-38-generic|^linux-cloud-tools.*4.15.0-38-generic) regexp to find running kernel packages
Checking: apport ([<Origin component:'main' archive:'bionic-updates' origin:'Ubuntu' label:'Ubuntu' site:'archive.ubuntu.com' isTrusted:True>])
adjusting candidate version: apport=2.20.9-0ubuntu7.1
...
Checking: libnss-systemd ([<Origin component:'main' archive:'bionic-updates' origin:'Ubuntu' label:'Ubuntu' site:'archive.ubuntu.com' isTrusted:True>, <Origin component:'main' archive:'bionic-security' origin:'Ubuntu' label:'Ubuntu' site:'security.ubuntu.com' isTrusted:True>])
skipping blacklisted package systemd
pkg systemd package has been blacklisted
sanity check failed
['libfreetype6']
[]
... |
|
| 2018-11-27 21:06:00 |
Balint Reczey |
tags |
verification-needed verification-needed-bionic verification-needed-cosmic |
verification-done-bionic verification-needed verification-needed-cosmic |
|
| 2018-11-27 21:10:21 |
Balint Reczey |
description |
[Impact]
* Originally autoremovable packages can be removed as newly autoremovable ones by unattended-upgrades
* This can surprise users potentially removing packages which are needed for the system's operation.
[Test Case]
* The buggy u-u version will remove the previously autoremovable zsh at the end, the fixed one does not.
$ sudo debootstrap bionic uu-test-bionic-1803749
...
$ sudo chroot uu-test-bionic-1803749
# apt install unattended-upgrades zsh
...
# echo "Unattended-Upgrade::Package-Blacklist {"libs";}" > /etc/apt/apt.conf.d/51unattended-upgrades
# echo "deb http://archive.ubuntu.com/ubuntu bionic-security main" >> /etc/apt/sources.list
# echo "deb http://archive.ubuntu.com/ubuntu bionic-updates main" >> /etc/apt/sources.list
# apt update -qq
# unattended-upgrades --dry-run --verbose --debug
...
All upgrades installed
marking zsh for removal
marking zsh-common for removal
Packages that were successfully auto-removed:
Packages that are kept back:
InstCount=0 DelCount=0 BrokenCount=0
#
[Regression Potential]
* Unattended-upgrades may use more CPU-time for operation but I did not observe a significant increase. Autopkgtest measures u-u's performance thus if this regression occurs, we can observe it easily.
* Due to the code changes u-u may still remove already autoremovable packages or fail to remove newly autoremovable ones in default configuration, but since the code became simpler with the change by eliminating an optimization this regression is unlikely to take place.
[Original Bug Text]
While triaging LP: #1803587 I found that originally autoremovable packages were handled as newly autoremovable ones by unattended-upgrades due to not finding all of them at the beginning of u-u's run.
The root cause seems to be cache.clear() resetting pkg.is_auto_removable to False under some circumstances I haven't fully narrowed down.
Set up a Bionic (or later) system with autoremovable packages, packages upgradable from -security and blacklist at least one upgradable package and apply the following patch to u-u:
root@bb-1803587:~# diff -Naur /usr/bin/unattended-upgrade.orig /usr/bin/unattended-upgrade
--- /usr/bin/unattended-upgrade.orig 2018-11-16 16:17:58.522583254 +0000
+++ /usr/bin/unattended-upgrade 2018-11-16 16:36:12.226675870 +0000
@@ -948,7 +948,9 @@
def rewind_cache(cache, pkgs_to_upgrade):
# type: (apt.Cache, List[apt.Package]) -> None
""" set the cache back to the state with packages_to_upgrade """
+ print([pkg.name for pkg in cache if pkg.is_auto_removable])
cache.clear()
+ print([pkg.name for pkg in cache if pkg.is_auto_removable])
for pkg2 in pkgs_to_upgrade:
pkg2.mark_install(from_user=not pkg2.is_auto_installed)
if cache.broken_count > 0:
Run u-u to observe cache.clear() resetting the list of autoremovable packages:
~# /usr/bin/unattended-upgrade --dry-run --verbose --debug
Initial blacklisted packages: systemd
Initial whitelisted packages:
Starting unattended upgrades script
Allowed origins are: o=Ubuntu,a=bionic, o=Ubuntu,a=bionic-security, o=UbuntuESM,a=bionic
Using (^linux-image|^linux-headers|^linux-image-extra|^linux-modules|^linux-modules-extra|^linux-signed-image|^kfreebsd-image|^kfreebsd-headers|^gnumach-image|^.*-modules|^.*-kernel|^linux-backports-modules-.*|^linux-modules-.*|^linux-tools|^linux-cloud-tools) regexp to find kernel packages
Using (^linux-image.*4.15.0-38-generic|^linux-headers.*4.15.0-38-generic|^linux-image-extra.*4.15.0-38-generic|^linux-modules.*4.15.0-38-generic|^linux-modules-extra.*4.15.0-38-generic|^linux-signed-image.*4.15.0-38-generic|^kfreebsd-image.*4.15.0-38-generic|^kfreebsd-headers.*4.15.0-38-generic|^gnumach-image.*4.15.0-38-generic|4.15.0-38-generic.*-modules|4.15.0-38-generic.*-kernel|^linux-backports-modules-.*.*4.15.0-38-generic|^linux-modules-.*.*4.15.0-38-generic|^linux-tools.*4.15.0-38-generic|^linux-cloud-tools.*4.15.0-38-generic) regexp to find running kernel packages
Checking: apport ([<Origin component:'main' archive:'bionic-updates' origin:'Ubuntu' label:'Ubuntu' site:'archive.ubuntu.com' isTrusted:True>])
adjusting candidate version: apport=2.20.9-0ubuntu7.1
...
Checking: libnss-systemd ([<Origin component:'main' archive:'bionic-updates' origin:'Ubuntu' label:'Ubuntu' site:'archive.ubuntu.com' isTrusted:True>, <Origin component:'main' archive:'bionic-security' origin:'Ubuntu' label:'Ubuntu' site:'security.ubuntu.com' isTrusted:True>])
skipping blacklisted package systemd
pkg systemd package has been blacklisted
sanity check failed
['libfreetype6']
[]
... |
[Impact]
* Originally autoremovable packages can be removed as newly autoremovable ones by unattended-upgrades
* This can surprise users potentially removing packages which are needed for the system's operation.
[Test Case]
* The buggy u-u version will remove the previously autoremovable zsh at the end, the fixed one does not.
$ sudo debootstrap bionic uu-test-bionic-1803749
...
$ sudo chroot uu-test-bionic-1803749
# apt install unattended-upgrades zsh
# apt-mark auto zsh
...
# echo "Unattended-Upgrade::Package-Blacklist {"libs";}" > /etc/apt/apt.conf.d/51unattended-upgrades
# echo "deb http://archive.ubuntu.com/ubuntu bionic-security main" >> /etc/apt/sources.list
# echo "deb http://archive.ubuntu.com/ubuntu bionic-updates main" >> /etc/apt/sources.list
# apt update -qq
# unattended-upgrades --dry-run --verbose --debug
...
All upgrades installed
marking zsh for removal
marking zsh-common for removal
Packages that were successfully auto-removed:
Packages that are kept back:
InstCount=0 DelCount=0 BrokenCount=0
#
[Regression Potential]
* Unattended-upgrades may use more CPU-time for operation but I did not observe a significant increase. Autopkgtest measures u-u's performance thus if this regression occurs, we can observe it easily.
* Due to the code changes u-u may still remove already autoremovable packages or fail to remove newly autoremovable ones in default configuration, but since the code became simpler with the change by eliminating an optimization this regression is unlikely to take place.
[Original Bug Text]
While triaging LP: #1803587 I found that originally autoremovable packages were handled as newly autoremovable ones by unattended-upgrades due to not finding all of them at the beginning of u-u's run.
The root cause seems to be cache.clear() resetting pkg.is_auto_removable to False under some circumstances I haven't fully narrowed down.
Set up a Bionic (or later) system with autoremovable packages, packages upgradable from -security and blacklist at least one upgradable package and apply the following patch to u-u:
root@bb-1803587:~# diff -Naur /usr/bin/unattended-upgrade.orig /usr/bin/unattended-upgrade
--- /usr/bin/unattended-upgrade.orig 2018-11-16 16:17:58.522583254 +0000
+++ /usr/bin/unattended-upgrade 2018-11-16 16:36:12.226675870 +0000
@@ -948,7 +948,9 @@
def rewind_cache(cache, pkgs_to_upgrade):
# type: (apt.Cache, List[apt.Package]) -> None
""" set the cache back to the state with packages_to_upgrade """
+ print([pkg.name for pkg in cache if pkg.is_auto_removable])
cache.clear()
+ print([pkg.name for pkg in cache if pkg.is_auto_removable])
for pkg2 in pkgs_to_upgrade:
pkg2.mark_install(from_user=not pkg2.is_auto_installed)
if cache.broken_count > 0:
Run u-u to observe cache.clear() resetting the list of autoremovable packages:
~# /usr/bin/unattended-upgrade --dry-run --verbose --debug
Initial blacklisted packages: systemd
Initial whitelisted packages:
Starting unattended upgrades script
Allowed origins are: o=Ubuntu,a=bionic, o=Ubuntu,a=bionic-security, o=UbuntuESM,a=bionic
Using (^linux-image|^linux-headers|^linux-image-extra|^linux-modules|^linux-modules-extra|^linux-signed-image|^kfreebsd-image|^kfreebsd-headers|^gnumach-image|^.*-modules|^.*-kernel|^linux-backports-modules-.*|^linux-modules-.*|^linux-tools|^linux-cloud-tools) regexp to find kernel packages
Using (^linux-image.*4.15.0-38-generic|^linux-headers.*4.15.0-38-generic|^linux-image-extra.*4.15.0-38-generic|^linux-modules.*4.15.0-38-generic|^linux-modules-extra.*4.15.0-38-generic|^linux-signed-image.*4.15.0-38-generic|^kfreebsd-image.*4.15.0-38-generic|^kfreebsd-headers.*4.15.0-38-generic|^gnumach-image.*4.15.0-38-generic|4.15.0-38-generic.*-modules|4.15.0-38-generic.*-kernel|^linux-backports-modules-.*.*4.15.0-38-generic|^linux-modules-.*.*4.15.0-38-generic|^linux-tools.*4.15.0-38-generic|^linux-cloud-tools.*4.15.0-38-generic) regexp to find running kernel packages
Checking: apport ([<Origin component:'main' archive:'bionic-updates' origin:'Ubuntu' label:'Ubuntu' site:'archive.ubuntu.com' isTrusted:True>])
adjusting candidate version: apport=2.20.9-0ubuntu7.1
...
Checking: libnss-systemd ([<Origin component:'main' archive:'bionic-updates' origin:'Ubuntu' label:'Ubuntu' site:'archive.ubuntu.com' isTrusted:True>, <Origin component:'main' archive:'bionic-security' origin:'Ubuntu' label:'Ubuntu' site:'security.ubuntu.com' isTrusted:True>])
skipping blacklisted package systemd
pkg systemd package has been blacklisted
sanity check failed
['libfreetype6']
[]
... |
|
| 2018-11-27 21:14:33 |
Balint Reczey |
tags |
verification-done-bionic verification-needed verification-needed-cosmic |
verification-done verification-done-bionic verification-done-cosmic |
|
| 2018-12-03 15:13:09 |
Launchpad Janitor |
unattended-upgrades (Ubuntu Cosmic): status |
Fix Committed |
Fix Released |
|
| 2018-12-03 15:13:23 |
Łukasz Zemczak |
removed subscriber Ubuntu Stable Release Updates Team |
|
|
|
| 2018-12-03 15:20:08 |
Launchpad Janitor |
unattended-upgrades (Ubuntu Bionic): status |
Fix Committed |
Fix Released |
|
| 2018-12-03 17:29:19 |
Brian Murray |
nominated for series |
|
Ubuntu Xenial |
|
| 2018-12-03 17:29:19 |
Brian Murray |
bug task added |
|
unattended-upgrades (Ubuntu Xenial) |
|
| 2018-12-03 19:21:02 |
Brian Murray |
unattended-upgrades (Ubuntu Xenial): status |
New |
Fix Committed |
|
| 2018-12-03 19:21:05 |
Brian Murray |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2018-12-03 19:21:10 |
Brian Murray |
tags |
verification-done verification-done-bionic verification-done-cosmic |
verification-done-bionic verification-done-cosmic verification-needed verification-needed-xenial |
|
| 2018-12-28 20:58:41 |
Mathew Hodson |
unattended-upgrades (Ubuntu Xenial): importance |
Undecided |
High |
|
| 2018-12-28 20:58:43 |
Mathew Hodson |
unattended-upgrades (Ubuntu Bionic): importance |
Undecided |
High |
|
| 2018-12-28 20:58:46 |
Mathew Hodson |
unattended-upgrades (Ubuntu Cosmic): importance |
Undecided |
High |
|
| 2019-02-08 14:53:06 |
Balint Reczey |
tags |
verification-done-bionic verification-done-cosmic verification-needed verification-needed-xenial |
verification-done verification-done-bionic verification-done-cosmic verification-done-xenial |
|
| 2019-02-28 13:01:12 |
Łukasz Zemczak |
tags |
verification-done verification-done-bionic verification-done-cosmic verification-done-xenial |
verification-done-bionic verification-done-cosmic verification-needed verification-needed-xenial |
|
| 2019-03-12 17:59:20 |
Balint Reczey |
tags |
verification-done-bionic verification-done-cosmic verification-needed verification-needed-xenial |
verification-done verification-done-bionic verification-done-cosmic verification-done-xenial |
|
| 2019-04-25 10:50:00 |
Launchpad Janitor |
unattended-upgrades (Ubuntu Xenial): status |
Fix Committed |
Fix Released |
|
