Apt updates that are uniformly spread across all timezones, with predictable application windows

Bug #1686470 reported by Dimitri John Ledkov
72
This bug affects 17 people
Affects Status Importance Assigned to Milestone
apt (Debian)
Fix Released
Unknown
apt (Ubuntu)
Fix Released
High
Dimitri John Ledkov
Xenial
Fix Released
High
Dimitri John Ledkov
Yakkety
Won't Fix
High
Dimitri John Ledkov
Zesty
Fix Released
High
Dimitri John Ledkov
Artful
Fix Released
High
Dimitri John Ledkov
unattended-upgrades (Debian)
Fix Released
Unknown
unattended-upgrades (Ubuntu)
Fix Released
High
Dimitri John Ledkov
Xenial
Fix Released
High
Steve Langasek
Yakkety
Won't Fix
High
Dimitri John Ledkov
Zesty
Fix Released
High
Steve Langasek
Artful
Fix Released
High
Dimitri John Ledkov

Bug Description

[ Impact ]

 * unattended-upgrades are enabled by default in Ubuntu 16.04 and later

 * Currently the following three things happen as a monolithic event:
   - metadata updates: apt update
   - download of updates: apt upgrade --download-only
   - application of updates: apt upgrade

 * For the long running instances, all of the above happens at random
   times throughout the day.

 * If systems were poweredoff / suspended, this happens on boot / resume

 * End-users would like to have predictable timing, and control over when
   the updates happen.

Considering all of the above, the following new behavior is proposed which should address all concerns in question. It combines all the desired properties from both end-user and mirror perspectives.

[ Proposed Default Behavior ]

 * Decouple unattended-upgrades application, from apt update

 * apt update:
   - shall be a systemd timer based unit, triggered every 12h with a
     random delay of 12h, therefore executed randomly twice a day.
   - if unattened-upgrades (default on), or download-upgreadaeble-packages
     are enabled, it should result in updates being downloaded aka
     `apt upgrade --download-only`

 * unattended-upgrades:
   - shall be a separate systemd timer based unit triggered at 6am local
     time with a random delay of 1h, therefore executed between 6am and
     7am local time.

 * On boot / resume:
   - if we have missed one, or more, apt update timers,
     apt update / download upgrades / unattended-upgrade will happen in
     sequence. This may result in mirror spikes, but we do want to secure
     cold/stale-booted systems as soon as possible.

[Test Case]

 * Run system for more than 24h, and check that apt updates were
   automatically executed twice.

 * Check that unattended upgrades were triggered to be applied at
   6am..7am window, if any.

 * Poweroff the machine over the period when apt-get update was
   scheduled, poweron and observe that apt-get update / download /
   unattended upgrade are all performed on boot.

 * Downgrade systemd to the release version of the package (from
   -security). Remove apt periodic stamp files rm /var/lib/apt/periodic/*.
   Then run 'sudo systemctl start apt-daily.service'.
   Confirm that the systemd package is downloaded, but not upgraded.

[Regression Potential]

 * The newly proposed behavior is a mix of Pre-xenial behavior of "do
   everything at 6am..6:30am window" and the xenial+ behavior of "do
   everything at random times throughout the day". If there are specific
   deployments that rely on the previous types of behaviour they will be
   able to adjust manually the systemd timers with the overrides to be
   executed exactly as they wish; or match the .0 release behaviour that
   they prefer.

 * If timers behavior is coded wrongly the proposed behaviour might not be
   executed as intended, thus requiring further SRUs to bring us in-line
   with the great expectations.

[Other Info]

  * Related bug reports and history:
    - bug #1615482
    - bug #1554848

Changed in apt (Ubuntu Xenial):
status: New → Triaged
Changed in apt (Ubuntu Yakkety):
status: New → Triaged
Changed in apt (Ubuntu Zesty):
status: New → Triaged
Changed in apt (Ubuntu Artful):
status: New → Triaged
Changed in apt (Ubuntu Xenial):
importance: Undecided → High
Changed in apt (Ubuntu Yakkety):
importance: Undecided → High
Changed in apt (Ubuntu Zesty):
importance: Undecided → High
Changed in apt (Ubuntu Artful):
importance: Undecided → High
description: updated
description: updated
Changed in unattended-upgrades (Ubuntu Xenial):
status: New → Triaged
Changed in unattended-upgrades (Ubuntu Yakkety):
status: New → Triaged
Changed in unattended-upgrades (Ubuntu Zesty):
status: New → Triaged
Changed in unattended-upgrades (Ubuntu Artful):
status: New → Triaged
Changed in unattended-upgrades (Ubuntu Xenial):
importance: Undecided → High
Changed in unattended-upgrades (Ubuntu Yakkety):
importance: Undecided → High
Changed in unattended-upgrades (Ubuntu Zesty):
importance: Undecided → High
Changed in unattended-upgrades (Ubuntu Artful):
importance: Undecided → High
Revision history for this message
Julian Andres Klode (juliank) wrote :

I propose we simply split up the apt daily script and run that in two timers (one for fetching, the other for potentially destructive stuff like u-u and clean).

/usr/lib/apt/apt.systemd.daily.common.sh
  The shell functions used by both scripts (top part of script), plus the variables
  (so right until the # update package lists part).

/usr/lib/apt/apt.systemd.daily
  The middle part of the current script, until u-u

/usr/lib/apt/apt.systemd.destructive-daily

  The final part of the script handling u-u, clean, and autoclean

Revision history for this message
Julian Andres Klode (juliank) wrote :

I just do an if now instead of three files, and here is the implementation:

https://github.com/Debian/apt/compare/master...julian-klode:lp1686470?diff=split&expand=1&name=lp1686470

Revision history for this message
Julian Andres Klode (juliank) wrote :

(be?field.comment=(be

Revision history for this message
Julian Andres Klode (juliank) wrote :

Uploaded 1.4.1ubuntu1 to artful which introduces the apt-daily-upgrade timer and service matching the proposed default behavior.

Changed in apt (Ubuntu Artful):
status: Triaged → Fix Committed
Revision history for this message
Julian Andres Klode (juliank) wrote :

Hmm, seems I forgot to change the downloading to run when unattended-upgrades is configured. But this seems like a bad choice anyway with the current implementation.

Maybe let's do something like

    if which unattended-upgrade >/dev/null 2>&1 && check_stamp $DOWNLOAD_UPGRADEABLE_STAMP $UnattendedUpgradeInterval; then
 if unattended-upgrade -d $XUUPOPT; then
     update_stamp $UPGRADE_STAMP
     debug_echo "unattended-upgrade -d (success)"
 else
     debug_echo "unattended-upgrade -d (error)"
 fi
    else
 debug_echo "unattended-upgrade -d (not run)"
    fi

That is, don't download the upgrades with apt -d dist-upgrade, use unattended-upgrade -d. This uses the same stamp file as the current list-upgrade option, but compares it against the unattended-upgrades section, and contains the unattended-upgrade body, with -d appended.

Revision history for this message
Julian Andres Klode (juliank) wrote :

Fixed in ubuntu2.

Revision history for this message
Steve Langasek (vorlon) wrote :

Why should apt update be run twice a day if the updates are only applied to the system once a day? This would be:
- a no-op for any metadata that hasn't changed in the preceding 12h
- a wasteful, redundant download for metadata that *has* changed in the preceding 12h
- (in rare cases) very wasteful if two SRUs are published back-to-back in the same day, since the first will be downloaded but not applied.

If the purpose is to ensure the updates being applied are never more than 12 hours out of date, that can be achieved with a single apt update scheduled in the 12h before 6am machine time.

If the purpose is to ensure downloads are spread more evenly around the clock, that is better achieved by doing one apt update per day, with a 24h smear, instead of two per day. (And btw, we probably want locking between the two jobs, so that we don't try to run an 'apt update' /while/ unattended-upgrades are being applied, but instead wait for unattended-upgrades to finish)

If your goal is to try to achieve some sort of balance between these goals, that at least makes sense why the design would be this way; but I think overall it's better to just do a single 24h smear and not try to compromise in a way that increases overall load on the mirrors with no major benefit to the users.

Can you please also document this design on wiki.ubuntu.com, so that we have better history tracking of any further changes (vs. edits to a bug description) and can more easily refer to it rather than pointing to an (at some point) ancient bug?

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

Justifications for twice a day:

 - to get motd updated & get upgrade popups on the desktop.
 - to have a better chance of having critical security fixes downloaded, before the next upgrade window.
 - in case of intermittent network losses, having two download attempts is better than one.
 - machines that are off overnight (office workers), with twice daily download have a better chance of downloading updates during office hours; instead of effectively always on boot.
 - i like it

Revision history for this message
Julian Andres Klode (juliank) wrote :

APT maintains a multitude of internal time stamp files preventing updates occuring more than once per day.

tags: added: patch
Revision history for this message
Julian Andres Klode (juliank) wrote :

I think the "locking" is already done by systemd. When units specify an ordering, it seems that one unit will only start when the other has shut down:

"Given two units with any ordering dependency between them, if one unit is shut down and the other is started up, the shutdown is ordered before the start-up"

Revision history for this message
Steve Langasek (vorlon) wrote : Re: [Bug 1686470] Re: Apt updates that are uniformally spread across all timezones, with predictable application windows

On Wed, Apr 26, 2017 at 08:39:36PM -0000, Julian Andres Klode wrote:
> I think the "locking" is already done by systemd. When units specify an
> ordering, it seems that one unit will only start when the other has shut
> down:

> "Given two units with any ordering dependency between them, if one unit
> is shut down and the other is started up, the shutdown is ordered before
> the start-up"

That language sounds very general; it's probably worth having someone
explicitly check this for our case before committing to rely on systemd :)

summary: - Apt updates that are uniformally spread across all timezones, with
+ Apt updates that are uniformly spread across all timezones, with
predictable application windows
Revision history for this message
Julian Andres Klode (juliank) wrote :

pabs just mentioned in #debian-apt that we could probably drop the ConditionACPower for the downloading service. I'm not sure why that was originally added, if only to protect upgrades or to save power when downloading, if anyone has an opinion, let me know.

Revision history for this message
Julian Andres Klode (juliank) wrote :

Re systemd conflict handling, I can ask systemd ML tomorrow.

Revision history for this message
Steve Langasek (vorlon) wrote : Re: [Bug 1686470] Re: Apt updates that are uniformly spread across all timezones, with predictable application windows

On Wed, Apr 26, 2017 at 11:20:47PM -0000, Julian Andres Klode wrote:
> pabs just mentioned in #debian-apt that we could probably drop the
> ConditionACPower for the downloading service. I'm not sure why that was
> originally added, if only to protect upgrades or to save power when
> downloading, if anyone has an opinion, let me know.

Agreed, it should be ok to drop this. I believe the original rationale is
that you don't want to *apply* an upgrade while on battery, since running
out of power during the upgrade is Very Bad. But downloads should be ok.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apt - 1.4.1ubuntu2

---------------
apt (1.4.1ubuntu2) artful; urgency=medium

  * Run unattended-upgrade -d in download part (LP: #1686470)

apt (1.4.1ubuntu1) artful; urgency=medium

  * Split apt-daily timer into two (LP: #1686470)

 -- Julian Andres Klode <email address hidden> Wed, 26 Apr 2017 21:41:05 +0200

Changed in apt (Ubuntu Artful):
status: Fix Committed → Fix Released
Revision history for this message
Julian Andres Klode (juliank) wrote :

I asked the systemd mailing list for some assistance on keeping the units from interfering with each other:

https://lists.freedesktop.org/archives/systemd-devel/2017-April/038772.html

Maybe they have a good solution to that issue.

Changed in apt (Debian):
status: Unknown → New
Revision history for this message
Julian Andres Klode (juliank) wrote :

OK, so 1.4.4 and friends actually enable debugging mode of unattended-upgrade instead of a download-only mode. --dry-run is documented as only doing downloads, but it also simulates the install and does verbose logging, so that's not useful.

In 1.4.6, I modified apt.systemd.daily to check unattended-upgrade --help for "download-only", and use that if it exists. This means that with 1.4.6, everything should work once this is implemented in unattended-upgrades.

I attached an early patch for unattended-upgrades (against the upstream version), there might be issues with it other than missing translations for --help.

Changed in unattended-upgrades (Debian):
status: Unknown → New
Changed in apt (Debian):
status: New → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Hello! I see uploads with fixes to this bug in various stable release queues - seeing that there was still some discussion after the uploads were done, I wanted to ask - what is the state of the fixes that are uploaded to the SRU queues? Are those fixes that are good to go or is this still in progress?

Revision history for this message
Julian Andres Klode (juliank) wrote :

Ahem, yes, sorry, I have to update those with the fix from 1.4.6. We also need updates for unattended upgrade that adds a --download-only option.

Revision history for this message
Julian Andres Klode (juliank) wrote :

Fixed updates are on their way, currently finishing CI tests.

Revision history for this message
Julian Andres Klode (juliank) wrote :

Uploaded 1.4.6~17.04.1, 1.3.9, and 1.2.24

Changed in apt (Ubuntu Xenial):
status: Triaged → In Progress
Changed in apt (Ubuntu Yakkety):
status: Triaged → In Progress
Changed in apt (Ubuntu Zesty):
status: Triaged → In Progress
Changed in unattended-upgrades (Ubuntu Artful):
assignee: nobody → Dimitri John Ledkov (xnox)
Changed in unattended-upgrades (Ubuntu Zesty):
assignee: nobody → Dimitri John Ledkov (xnox)
Changed in unattended-upgrades (Ubuntu Yakkety):
assignee: nobody → Dimitri John Ledkov (xnox)
Changed in unattended-upgrades (Ubuntu Xenial):
assignee: nobody → Dimitri John Ledkov (xnox)
Changed in apt (Ubuntu Artful):
assignee: nobody → Dimitri John Ledkov (xnox)
Changed in apt (Ubuntu Zesty):
assignee: nobody → Dimitri John Ledkov (xnox)
Changed in apt (Ubuntu Yakkety):
assignee: nobody → Dimitri John Ledkov (xnox)
Changed in apt (Ubuntu Xenial):
assignee: nobody → Dimitri John Ledkov (xnox)
Changed in unattended-upgrades (Debian):
status: New → Fix Committed
Revision history for this message
Adam Conrad (adconrad) wrote : Please test proposed package

Hello Dimitri, or anyone else affected,

Accepted apt into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apt/1.2.24 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in apt (Ubuntu Xenial):
status: In Progress → Fix Committed
Changed in apt (Ubuntu Yakkety):
status: In Progress → Won't Fix
tags: added: verification-needed verification-needed-xenial
Changed in unattended-upgrades (Ubuntu Yakkety):
status: Triaged → Won't Fix
Revision history for this message
Adam Conrad (adconrad) wrote :

Hello Dimitri, or anyone else affected,

Accepted apt into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apt/1.4.6~17.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in apt (Ubuntu Zesty):
status: In Progress → Fix Committed
Revision history for this message
Dimitri John Ledkov (xnox) wrote :

Starting with:
ii apt 1.2.20 amd64 commandline package manager

The following timers are present:
# systemctl list-timers
NEXT LEFT LAST PASSED UNIT ACTIVATES
Fri 2017-07-28 09:42:31 UTC 14min left n/a n/a systemd-tmpfiles-clean.timer systemd-tmpfiles-clean.service
Fri 2017-07-28 12:52:15 UTC 3h 24min left n/a n/a snapd.refresh.timer snapd.refresh.service
Sat 2017-07-29 03:09:15 UTC 17h left n/a n/a apt-daily.timer apt-daily.service

The apt-daily.service is going to run, unexpectadly, at 3:09am.

Upgrading just apt:
ii apt 1.2.24 amd64 commandline package manager

# systemctl list-timers
NEXT LEFT LAST PASSED UNIT ACTIVATES
Fri 2017-07-28 09:42:31 UTC 12min left n/a n/a systemd-tmpfiles-clean.timer systemd-tmpfiles-clean.service
Fri 2017-07-28 10:03:19 UTC 33min left n/a n/a snapd.refresh.timer snapd.refresh.service
Fri 2017-07-28 20:18:24 UTC 10h left Fri 2017-07-28 09:27:35 UTC 1min 57s ago apt-daily.timer apt-daily.service
Sat 2017-07-29 06:30:36 UTC 21h left n/a n/a apt-daily-upgrade.timer apt-daily-upgrade.service

Now there is timer to do a refresh at a random time, yet there is a predictable apt-daily-upgrade to apply the timer between 6 and 7am.

All of this looks good.

I will leave this container around, and will check if upgrades were applied tomorrow.

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

Similarly starting a zesty container for the same test case:
ii apt 1.4 amd64

upgrading to:
ii apt 1.4.6~17.04.1 amd64

Setting up apt (1.4.6~17.04.1) ...
Created symlink /etc/systemd/system/timers.target.wants/apt-daily-upgrade.timer → /lib/systemd/system/apt-daily-upgrade.timer.

However, this unit did not get activated.
# systemctl status apt-daily-upgrade.timer
● apt-daily-upgrade.timer - Daily apt upgrade and clean activities
   Loaded: loaded (/lib/systemd/system/apt-daily-upgrade.timer; enabled; vendor preset: enabled)
   Active: inactive (dead)

After rebooting the unit is active:
# systemctl list-timers
NEXT LEFT LAST PASSED UNIT ACTIVATES
Fri 2017-07-28 09:55:34 UTC 14min left n/a n/a systemd-tmpfiles-clean.timer systemd-tmpfiles-clean.service
Fri 2017-07-28 10:32:53 UTC 52min left Fri 2017-07-28 09:36:24 UTC 4min 22s ago motd-news.timer motd-news.service
Fri 2017-07-28 13:44:26 UTC 4h 3min left Fri 2017-07-28 09:36:24 UTC 4min 22s ago snapd.refresh.timer snapd.refresh.service
Fri 2017-07-28 18:04:19 UTC 8h left Fri 2017-07-28 09:36:24 UTC 4min 22s ago apt-daily.timer apt-daily.service
Sat 2017-07-29 06:52:52 UTC 21h left n/a n/a apt-daily-upgrade.timer apt-daily-upgrade.service

I will leave this zesty instance running, but it may mean a fix up on zesty is needed to activate the apt-daily-upgrade.timer in postinst.

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

In zesty, postinst has:
deb-systemd-invoke $_dh_action apt-daily-upgrade.timer apt-daily.timer >/dev/null || true

maybe my instance was still booting hence this did not work. Will troubleshoot zesty more.

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

Leaving instances running shows that both xenial and zesty got unattended upgrade logs / attempts around 6:30am every day. Marking xenial as good. And will redo zesty upgrade test again to make sure units are started for the upgrade timer.

tags: added: verification-done-xenial
removed: verification-needed-xenial
tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apt - 1.2.24

---------------
apt (1.2.24) xenial; urgency=medium

  * Microrelease covering fixes of 1.4.6
  * Fix parsing of or groups in build-deps with ignored packages (LP: #1694697)
  * apt.systemd.daily: Use unattended-ugrade --download-only if available.
    Instead of passing -d, which enables a debugging mode; check if
    unattended-upgrade supports an option --download-only (which is yet
    to be implemented) and use that (Closes: #863859)

apt (1.2.23) xenial; urgency=medium

  * Microrelease covering fixes of 1.4.4

  [ Alan Jenkins ]
  * apt.systemd.daily: fix error from locking code (Closes: #862567)

apt (1.2.22) xenial; urgency=medium

  [ Julian Andres Klode ]
  * Run unattended-upgrade -d in download part
  * apt.systemd.daily: Add locking
  * Split apt-daily timer into two (LP: #1686470)

  [ Matt Kraai ]
  * bash-completion: Fix spelling of autoclean (Closes: #861846)

apt (1.2.21) xenial; urgency=medium

  * Microrelease covering fixes of 1.4 and 1.4.1

  [ Julian Andres Klode ]
  * Ignore \.ucf-[a-z]+$ like we do for \.dpkg-[a-z]+$
  * systemd: Rework timing and add After=network-online (was LP #1615482)

  [ David Kalnischkies ]
  * Fix and avoid quoting in CommandLine::AsString (LP: #1672710)

  [ Unit 193 ]
  * apt-ftparchive: Support '.ddeb' dbgsym packages

 -- Julian Andres Klode <email address hidden> Mon, 19 Jun 2017 13:58:04 +0200

Changed in apt (Ubuntu Xenial):
status: Fix Committed → Fix Released
Revision history for this message
Adam Conrad (adconrad) wrote : Update Released

The verification of the Stable Release Update for apt has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Steve Langasek (vorlon) wrote :

Landing the apt piece of this without the unattended-upgrades piece has regressed the around-the-clock distribution of unattended-upgrades downloads (which is enabled by default for security updates in 16.04).

The plan is to follow up with unattended-upgrades SRU today to get this all landed before 16.04.3 so that the point release images are in a coherent state.

apt must also declare a versioned Breaks: against versions of unattended-upgrades that don't implement --download-only. Otherwise, the infrastructure consequences of a future apt update published to the security pocket without a matching unattended-upgrades update would be severe.

Steve Langasek (vorlon)
description: updated
Changed in unattended-upgrades (Ubuntu Xenial):
assignee: Dimitri John Ledkov (xnox) → Steve Langasek (vorlon)
status: Triaged → In Progress
Changed in unattended-upgrades (Ubuntu Zesty):
assignee: Dimitri John Ledkov (xnox) → Steve Langasek (vorlon)
status: Triaged → In Progress
description: updated
Revision history for this message
Adam Conrad (adconrad) wrote : Please test proposed package

Hello Dimitri, or anyone else affected,

Accepted unattended-upgrades into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/unattended-upgrades/0.90ubuntu0.7 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in unattended-upgrades (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-xenial
removed: verification-done verification-done-xenial
Changed in unattended-upgrades (Ubuntu Artful):
status: Triaged → Fix Committed
Changed in unattended-upgrades (Ubuntu Zesty):
status: In Progress → Fix Committed
tags: added: verification-needed-zesty
Revision history for this message
Adam Conrad (adconrad) wrote :

Hello Dimitri, or anyone else affected,

Accepted unattended-upgrades into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/unattended-upgrades/0.93.1ubuntu2.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-zesty to verification-done-zesty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-zesty. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package unattended-upgrades - 0.93.1ubuntu8

---------------
unattended-upgrades (0.93.1ubuntu8) artful; urgency=medium

  [ Steve Langasek ]
  * Cherry-pick 2e5deed, f26edb4 from upstream to add support for a
    --download-only option, enabling us to download updates at a random time
    of day by default but apply them predictably in the 6am-7am window.
    LP: #1686470.

 -- Dimitri John Ledkov <email address hidden> Mon, 31 Jul 2017 23:28:38 +0100

Changed in unattended-upgrades (Ubuntu Artful):
status: Fix Committed → Fix Released
Revision history for this message
Dimitri John Ledkov (xnox) wrote :

started xenial container, and downgraded systemd to release pocket version.

check timer units, only apt-daily is present.

upgraded to apt from -updatees 1.2.24.

enabled -proposed and installed unattended-upgrades from it, 0.90ubuntu0.7

two .timer units are present: apt-daily.timer random, and apt-daily-upgrade.timer between 6am and 7am.

clearing /var/lib/apt/periodic

$ systemctl start --no-block apt-daily.service; watch -d ls /var/cache/apt/archives/

Observed that systemd debs got downloaded in /var/cache/apt/archives. Observed that unattened-upgrades.log has entreis that it did run, but does not list that it has upgraded anything.

$ tail -f /var/log/unattended-upgrades/unattended-upgrades.log &
$ systemctl start --no-block apt-daily-upgrade.service

Observed that install of the packages started straight away, specially the timestamps in the -dpkg.log of the unattended upgrades.

I believe above validates that apt-daily.timer updates cache and download debs; whilst apt-daily-upgrade.timer installs the debs.

Xenial verification is complete.

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

The update & upgrade jobs did not start on boot, but this is not a regression compared with current xenial.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package unattended-upgrades - 0.90ubuntu0.7

---------------
unattended-upgrades (0.90ubuntu0.7) xenial; urgency=medium

  * Cherry-pick 2e5deed, f26edb4 from upstream to add support for a
    --download-only option, enabling us to download updates at a random time
    of day by default but apply them predictably in the 6am-7am window.
    LP: #1686470.

 -- Steve Langasek <email address hidden> Mon, 31 Jul 2017 12:34:37 -0700

Changed in unattended-upgrades (Ubuntu Xenial):
status: Fix Committed → Fix Released
tags: added: verification-done verification-done-xenial
removed: verification-needed verification-needed-xenial
Changed in unattended-upgrades (Debian):
status: Fix Committed → Fix Released
Revision history for this message
Mikkel Kirkgaard Nielsen (mikini) wrote :

An annoyance about this change is that it reuses the apt-daily.timer for only download and lets the upgrade be triggered by a new apt-daily-upgrade.timer.

Systems using apt <1.2.24 which were sensitive to the upgrade point of time would typically have overridden the apt-daily.timer to happen at a fixed time. After upgrade to apt 1.2.24 those systems would still have apt-daily.timer run at the time chosen by the administrator but triggering /usr/lib/apt/apt.systemd.daily update to download packages would make it bail out (as the unattended-upgrades 0.90-ubuntu0.7 which introduced the --download-only option is not a SRU). But upgrades, and effectively also download until a manual upgrade of unattended-upgrades, would happen at the default apt-daily-upgrade.timer time (06:00+random(60m)).

Security updates that warps carefully scheduled and important system events like this leaves the impression that administrators are not in control of their machines.

Revision history for this message
Mikkel Kirkgaard Nielsen (mikini) wrote :

Sorry, apt 1.2.24 was not a SRU so the point about it being unattended is not valid, others are.

Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote : Reminder of SRU verification policy change

Thank you for taking the time to verify this stable release fix. We have noticed that you have used the verification-done tag for marking the bug as verified and would like to point out that due to a recent change in SRU bug verification policy fixes now have to be marked with per-release tags (i.e. verification-done-$RELEASE). Please remove the verification-done tag and add one for the release you have tested the package in. Thank you!

https://wiki.ubuntu.com/StableReleaseUpdates#Verification

Steve Langasek (vorlon)
tags: removed: verification-done
tags: added: verification-done-zesty
removed: verification-needed-zesty
Revision history for this message
Dimitri John Ledkov (xnox) wrote :

Test of 0.93.1ubuntu2.3 & 1.4.6~17.04.1 together are fine. please release the two together.

@mikini - it is prudent from OS vendor point of view to enable and install security updates by default, as most users neglect to set it up themselves. unattanded-upgrades is one way to do this, but it is an optional component of the system and many users use other means to keep their systems secure. The constraints on randomising downloads is due to mirror load spikes. The constraint on consistent and predicable application of updates is due to unexpected behavior experienced by our user base when random application of updates was in place. Overall it is a sensible default. We do monitor crash and error rates, and that data suggests to us that people routinely postpone upgrades or delay them. As for key crashes we reach 90% of updates applied within 2 weeks. Thus defaults to apply updates within a day are sensible. Please note that adjusting .timer frequency alone will not be sufficient as apt has its own internal timestamps too which throttle unattanded upgrades. If you want full control of upgrades do execute / automate them in a manner that is appropriate for your deployment. There is nothing in place that would prevent you from doing that. Many people use other mechanisms for upgrades, e.g. landscape.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apt - 1.4.6~17.04.1

---------------
apt (1.4.6~17.04.1) zesty; urgency=medium

  * apt.systemd.daily: Do not pass -d to unattended-upgrade for the download
  job. This actually enables debugging. Instead check if unattended-upgrade
  supports an option --download-only (which is yet to be implemented) and use
  that (Closes: #863859)

apt (1.4.5) unstable; urgency=medium

  * Fix parsing of or groups in build-deps with ignored packages (LP: #1694697)

apt (1.4.4) unstable; urgency=medium

  [ Alan Jenkins ]
  * apt.systemd.daily: fix error from locking code (Closes: #862567)

apt (1.4.3) unstable; urgency=medium

  [ Julian Andres Klode ]
  * Do not try to (re)start timers outside 'apt' package (Closes: #862001)

  [ Miroslav Kure ]
  * Updated Czech translation of apt (Closes: #861943)

apt (1.4.2) unstable; urgency=medium

  [ Julian Andres Klode ]
  * Run unattended-upgrade -d in download part
  * apt.systemd.daily: Add locking
  * Split apt-daily timer into two (LP: #1686470)

  [ Matt Kraai ]
  * bash-completion: Fix spelling of autoclean (Closes: #861846)

apt (1.4.1) unstable; urgency=medium

  [ Julian Andres Klode ]
  * systemd: Rework timing and add After=network-online (LP: #1615482)
  * debian/rules: Actually invoke dh_clean in override_dh_clean

  [ Unit 193 ]
  * apt-ftparchive: Support '.ddeb' dbgsym packages

 -- Julian Andres Klode <email address hidden> Thu, 01 Jun 2017 10:50:26 +0200

Changed in apt (Ubuntu Zesty):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package unattended-upgrades - 0.93.1ubuntu2.3

---------------
unattended-upgrades (0.93.1ubuntu2.3) zesty; urgency=medium

  * Cherry-pick 2e5deed, f26edb4 from upstream to add support for a
    --download-only option, enabling us to download updates at a random time
    of day by default but apply them predictably in the 6am-7am window.
    LP: #1686470.

 -- Steve Langasek <email address hidden> Mon, 31 Jul 2017 12:40:12 -0700

Changed in unattended-upgrades (Ubuntu Zesty):
status: Fix Committed → Fix Released
tags: added: id-597a831a26eaec02914e6202
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Dimitri, or anyone else affected,

Accepted unattended-upgrades into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/unattended-upgrades/1.1ubuntu1.18.04.7~16.04.0 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in unattended-upgrades (Ubuntu Xenial):
status: Fix Released → Fix Committed
tags: added: verification-needed verification-needed-xenial
removed: verification-done-xenial
Revision history for this message
Haw Loeung (hloeung) wrote :

Wasn't this already verified by Dimitri on 2017-08-01?

Revision history for this message
Steve Langasek (vorlon) wrote :

Yes. There is a follow-up unattended-upgrades SRU happening to xenial to get its behavior more in-line with bionic and above, and it seems its changelog has some overbroad bug references by virtue of being a wholesale backport.

Revision history for this message
Balint Reczey (rbalint) wrote :

Indeed, this bug was referenced by accident.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package unattended-upgrades - 0.90ubuntu0.10

---------------
unattended-upgrades (0.90ubuntu0.10) xenial-security; urgency=medium

  * No change rebuild in the -security pocket (See LP #1686470)

 -- Marc Deslauriers <email address hidden> Fri, 18 Jan 2019 13:34:27 -0500

Changed in unattended-upgrades (Ubuntu Xenial):
status: Fix Committed → Fix Released
Mathew Hodson (mhodson)
tags: added: verification-done-xenial
removed: verification-needed verification-needed-xenial
Revision history for this message
Jarno Suni (jarnos) wrote :

Then should the version in xenial-proposed be deleted?

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.