| 2020-09-29 17:33:12 |
Chad Smith |
bug |
|
|
added bug |
| 2020-09-29 17:33:18 |
Chad Smith |
ubuntu-release-upgrader (Ubuntu): assignee |
|
Chad Smith (chad.smith) |
|
| 2020-09-29 17:37:11 |
Chad Smith |
ubuntu-release-upgrader (Ubuntu): status |
New |
In Progress |
|
| 2020-09-29 19:44:42 |
Launchpad Janitor |
merge proposal linked |
|
https://code.launchpad.net/~chad.smith/ubuntu-release-upgrader/+git/ubuntu-release-upgrader/+merge/391183 |
|
| 2020-10-01 21:24:34 |
Launchpad Janitor |
merge proposal linked |
|
https://code.launchpad.net/~chad.smith/ubuntu-release-upgrader/+git/ubuntu-release-upgrader/+merge/391725 |
|
| 2020-10-05 21:08:17 |
Launchpad Janitor |
merge proposal linked |
|
https://code.launchpad.net/~chad.smith/ubuntu/+source/ubuntu-release-upgrader/+git/ubuntu-release-upgrader/+merge/391848 |
|
| 2020-10-06 13:30:48 |
Launchpad Janitor |
ubuntu-release-upgrader (Ubuntu): status |
In Progress |
Fix Released |
|
| 2020-10-07 00:25:04 |
Launchpad Janitor |
merge proposal linked |
|
https://code.launchpad.net/~chad.smith/ubuntu/+source/ubuntu-release-upgrader/+git/ubuntu-release-upgrader/+merge/391907 |
|
| 2020-10-07 16:56:22 |
Launchpad Janitor |
merge proposal linked |
|
https://code.launchpad.net/~chad.smith/ubuntu/+source/ubuntu-release-upgrader/+git/ubuntu-release-upgrader/+merge/391937 |
|
| 2020-10-09 16:07:28 |
Chad Smith |
description |
In order for custom PostInstallScript to add/remove apt packages and call apt update, DistUpgradecontroller needs to release the apt cache directory lock before that stage.
If the lock is still in place, DistUpgradeController.runPostUpgradeScripts hits errors such as:
E: Could not get lock /var/lib/apt/lists/lock - open (11: Resource temporarily unavailable)
E: Unable to lock directory /var/lib/apt/lists/ |
=== Begin SRU Template ===
[Impact]
Any Ubuntu Advantage apt-based service is enbled via a PPA. If those PPAs are not listed as valid mirrors in mirrors.cfg the PPAs get disabled across upgrade.
UA-client has a script which will enable those PPAs across upgrade path, but needs the apt cache lock released during runPostInstallScripts.
Validate Ubuntu Advantage apt access is retained across do-release-upgrade path so customers to not lose access to security updates after performing an upgrade.
[Test Case]
#!/bin/bash
"""
SRU Verification ubuntu-release-upgrader + ubuntu=advantage-tools
Test procedure:
- launch container Trusty, Xenial or Bionic
- Attach container to UA subscription (which activates a number of commerical PPAs
- download and run -proposed ubuntu-release-upgrader tool for upgrade release
- Assert successful upgrade
- Confirm valid mirrors not disabled
- Confirm third party non-commercial PPA URLs still disabled
- Confirm third party UA commercial URLs still disabled
(due to expected feature gap)
- Confirm UA status reports esm-infra still disabled (known feature gap)
"""
set -ex
UA_TOKEN=$1
if [ -z "$1" ]; then
echo "Usage: $0 <contractTOKEN>"
exit 1
fi
cat > test-uru.yaml <<EOF
#cloud-config
package_update: true
package_upgrade: true
apt_sources:
- source: "deb http://ppa.launchpad.net/ua-client/proposed/ubuntu trusty main"
apt:
sources:
ua.proposed:
source: deb http://ppa.launchpad.net/ua-client/proposed/ubuntu \$RELEASE main
EOF
# ua.list:
# source: deb http://ppa.launchpad.net/canonical-server/ua-client-daily/ubuntu \$RELEASE main
# keyid: 94E187AD53A59D1847E4880F8A295C4FB8B190B7
cat > checkaptpolicy.sh <<EOF
#!/bin/bash
set -x
RELEASE=\`lsb_release -sc\`
echo -n "Current release: $RELEASE"
echo "Assert no disabled valid mirrors in /eta/apt/sources.list"
! grep disable /etc/apt/sources.list || echo "FAILURE: found disabled valid mirror urls"
echo "Checking commercial Ubuntu Advantage PPAs apt policy and config"
apt-cache policy | grep esm.ubuntu.com
for file in \`ls /etc/apt/sources.list.d/ubuntu-*.list\`; do
echo "--- file: \${file}"
cat \${file}
done
EOF
chmod 755 checkaptpolicy.sh
declare -A NEXTDIST=( [bionic]=focal [xenial]=bionic [trusty]=xenial )
for release in trusty; do
vm=test-sru-$release
echo "--- Launch cloud-init with ${release}-proposed enabled"
#lxc launch ubuntu-daily:${release} ${vm} -c user.user-data="$(cat test-uru.yaml)"
#lxc launch daily:$release -n $vm --cloud-init test-uru.yaml
upgraderelease=${NEXTDIST[$release]}
echo "--- Wait for cloud-init to finish"
if [ "${release}" = "trusty" ]; then
while [ "N 2" != "$(lxc exec ${vm} -- runlevel)" ]; do
echo "waiting on runlevel 2"
sleep 5
done
else
lxc exec ${vm} -- cloud-init status --wait --long
fi
echo "--- Attach Ubuntu-Advantage and enable services"
lxc exec ${vm} -- sudo ua attach ${UA_TOKEN}
lxc exec ${vm} -- sudo ua status | tee ua-status.orig
lxc file push checkaptpolicy.sh ${vm}/
lxc exec ${vm} -- /checkaptpolicy.sh
lxc exec ${vm} -- wget http://archive.ubuntu.com/ubuntu/dists/$upgraderelease-proposed/main/dist-upgrader-all/current/$upgraderelease.tar.gz
lxc exec ${vm} -- tar xzvf $upgraderelease.tar.gz
#lxc exec ${vm} -- wget https://git.launchpad.net/~chad.smith/ubuntu/+source/ubuntu-release-upgrader/plain/DistUpgrade/DistUpgradeController.py?h=uru-xenial-ubuntu-advantage -O DistUpgradeController.py
#lxc exec ${vm} -- wget https://git.launchpad.net/~chad.smith/ubuntu/+source/ubuntu-release-upgrader/plain/data/mirrors.cfg?h=uru-xenial-ubuntu-advantage -O mirrors.cfg
echo "--- Validate UA APT sources after upgrade"
lxc exec ${vm} -- /checkaptpolicy.sh
echo "--- Ensure UA status reports ESM enabled"
lxc exec ${vm} -- sudo ua status | tee ua-status.upgrade
echo "--- Expect no diffs in original and upgrade status"
diff -urN ua-status.orig ua-status.upgrade
done
[Regression Potential]
None as UA support is not yet officially introduced in Xenial or later. Customers currently have to manually re-enable apt config across upgrades.
=== End SRU Template ===
==== Original Description ===
In order for custom PostInstallScript to add/remove apt packages and call apt update, DistUpgradecontroller needs to release the apt cache directory lock before that stage.
If the lock is still in place, DistUpgradeController.runPostUpgradeScripts hits errors such as:
E: Could not get lock /var/lib/apt/lists/lock - open (11: Resource temporarily unavailable)
E: Unable to lock directory /var/lib/apt/lists/ |
|
| 2020-10-09 16:38:46 |
Timo Aaltonen |
ubuntu-release-upgrader (Ubuntu Focal): status |
New |
Fix Committed |
|
| 2020-10-09 16:38:47 |
Timo Aaltonen |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2020-10-09 16:38:49 |
Timo Aaltonen |
bug |
|
|
added subscriber SRU Verification |
| 2020-10-09 16:38:52 |
Timo Aaltonen |
tags |
|
verification-needed verification-needed-focal |
|
| 2020-10-09 20:51:36 |
Timo Aaltonen |
ubuntu-release-upgrader (Ubuntu Bionic): status |
New |
Fix Committed |
|
| 2020-10-09 20:51:41 |
Timo Aaltonen |
tags |
verification-needed verification-needed-focal |
verification-needed verification-needed-bionic verification-needed-focal |
|
| 2020-10-09 20:54:36 |
Timo Aaltonen |
ubuntu-release-upgrader (Ubuntu Xenial): status |
New |
Fix Committed |
|
| 2020-10-09 20:54:41 |
Timo Aaltonen |
tags |
verification-needed verification-needed-bionic verification-needed-focal |
verification-needed verification-needed-bionic verification-needed-focal verification-needed-xenial |
|
| 2020-10-10 02:39:10 |
Chad Smith |
description |
=== Begin SRU Template ===
[Impact]
Any Ubuntu Advantage apt-based service is enbled via a PPA. If those PPAs are not listed as valid mirrors in mirrors.cfg the PPAs get disabled across upgrade.
UA-client has a script which will enable those PPAs across upgrade path, but needs the apt cache lock released during runPostInstallScripts.
Validate Ubuntu Advantage apt access is retained across do-release-upgrade path so customers to not lose access to security updates after performing an upgrade.
[Test Case]
#!/bin/bash
"""
SRU Verification ubuntu-release-upgrader + ubuntu=advantage-tools
Test procedure:
- launch container Trusty, Xenial or Bionic
- Attach container to UA subscription (which activates a number of commerical PPAs
- download and run -proposed ubuntu-release-upgrader tool for upgrade release
- Assert successful upgrade
- Confirm valid mirrors not disabled
- Confirm third party non-commercial PPA URLs still disabled
- Confirm third party UA commercial URLs still disabled
(due to expected feature gap)
- Confirm UA status reports esm-infra still disabled (known feature gap)
"""
set -ex
UA_TOKEN=$1
if [ -z "$1" ]; then
echo "Usage: $0 <contractTOKEN>"
exit 1
fi
cat > test-uru.yaml <<EOF
#cloud-config
package_update: true
package_upgrade: true
apt_sources:
- source: "deb http://ppa.launchpad.net/ua-client/proposed/ubuntu trusty main"
apt:
sources:
ua.proposed:
source: deb http://ppa.launchpad.net/ua-client/proposed/ubuntu \$RELEASE main
EOF
# ua.list:
# source: deb http://ppa.launchpad.net/canonical-server/ua-client-daily/ubuntu \$RELEASE main
# keyid: 94E187AD53A59D1847E4880F8A295C4FB8B190B7
cat > checkaptpolicy.sh <<EOF
#!/bin/bash
set -x
RELEASE=\`lsb_release -sc\`
echo -n "Current release: $RELEASE"
echo "Assert no disabled valid mirrors in /eta/apt/sources.list"
! grep disable /etc/apt/sources.list || echo "FAILURE: found disabled valid mirror urls"
echo "Checking commercial Ubuntu Advantage PPAs apt policy and config"
apt-cache policy | grep esm.ubuntu.com
for file in \`ls /etc/apt/sources.list.d/ubuntu-*.list\`; do
echo "--- file: \${file}"
cat \${file}
done
EOF
chmod 755 checkaptpolicy.sh
declare -A NEXTDIST=( [bionic]=focal [xenial]=bionic [trusty]=xenial )
for release in trusty; do
vm=test-sru-$release
echo "--- Launch cloud-init with ${release}-proposed enabled"
#lxc launch ubuntu-daily:${release} ${vm} -c user.user-data="$(cat test-uru.yaml)"
#lxc launch daily:$release -n $vm --cloud-init test-uru.yaml
upgraderelease=${NEXTDIST[$release]}
echo "--- Wait for cloud-init to finish"
if [ "${release}" = "trusty" ]; then
while [ "N 2" != "$(lxc exec ${vm} -- runlevel)" ]; do
echo "waiting on runlevel 2"
sleep 5
done
else
lxc exec ${vm} -- cloud-init status --wait --long
fi
echo "--- Attach Ubuntu-Advantage and enable services"
lxc exec ${vm} -- sudo ua attach ${UA_TOKEN}
lxc exec ${vm} -- sudo ua status | tee ua-status.orig
lxc file push checkaptpolicy.sh ${vm}/
lxc exec ${vm} -- /checkaptpolicy.sh
lxc exec ${vm} -- wget http://archive.ubuntu.com/ubuntu/dists/$upgraderelease-proposed/main/dist-upgrader-all/current/$upgraderelease.tar.gz
lxc exec ${vm} -- tar xzvf $upgraderelease.tar.gz
#lxc exec ${vm} -- wget https://git.launchpad.net/~chad.smith/ubuntu/+source/ubuntu-release-upgrader/plain/DistUpgrade/DistUpgradeController.py?h=uru-xenial-ubuntu-advantage -O DistUpgradeController.py
#lxc exec ${vm} -- wget https://git.launchpad.net/~chad.smith/ubuntu/+source/ubuntu-release-upgrader/plain/data/mirrors.cfg?h=uru-xenial-ubuntu-advantage -O mirrors.cfg
echo "--- Validate UA APT sources after upgrade"
lxc exec ${vm} -- /checkaptpolicy.sh
echo "--- Ensure UA status reports ESM enabled"
lxc exec ${vm} -- sudo ua status | tee ua-status.upgrade
echo "--- Expect no diffs in original and upgrade status"
diff -urN ua-status.orig ua-status.upgrade
done
[Regression Potential]
None as UA support is not yet officially introduced in Xenial or later. Customers currently have to manually re-enable apt config across upgrades.
=== End SRU Template ===
==== Original Description ===
In order for custom PostInstallScript to add/remove apt packages and call apt update, DistUpgradecontroller needs to release the apt cache directory lock before that stage.
If the lock is still in place, DistUpgradeController.runPostUpgradeScripts hits errors such as:
E: Could not get lock /var/lib/apt/lists/lock - open (11: Resource temporarily unavailable)
E: Unable to lock directory /var/lib/apt/lists/ |
=== Begin SRU Template ===
[Impact]
Any Ubuntu Advantage apt-based service is enbled via a PPA. If those PPAs are not listed as valid mirrors in mirrors.cfg the PPAs get disabled across upgrade.
UA-client has a script which will enable those PPAs across upgrade path, but needs the apt cache lock released during runPostInstallScripts.
Validate Ubuntu Advantage apt access is retained across do-release-upgrade path so customers to not lose access to security updates after performing an upgrade.
[Test Case]
#!/bin/bash
#
# SRU Verification ubuntu-release-upgrader + ubuntu=advantage-tools
# Test procedure:
# - launch container Trusty, Xenial or Bionic
# - Attach container to UA subscription (which activates a number of commerical PPAs
# - download and run -proposed ubuntu-release-upgrader tool for upgrade release
# - Assert successful upgrade
# - Confirm valid mirrors not disabled
# - Confirm third party non-commercial PPA URLs still disabled
# - Confirm third party UA commercial URLs still disabled
# (due to expected feature gap)
# - Confirm UA status reports esm-infra still disabled (known feature gap)
set -ex
UA_TOKEN=$1
if [ -z "$1" ]; then
echo "Usage: $0 <contractTOKEN>"
exit 1
fi
# sources:
# ua.proposed:
# source: deb http://ppa.launchpad.net/ua-client/proposed/ubuntu \$RELEASE main
# keyid: 6E34E7116C0BC933
cat > test-uru.yaml <<EOF
#cloud-config
package_update: true
package_upgrade: true
apt:
apt_sources:
- source: "deb http://ppa.launchpad.net/ua-client/proposed/ubuntu trusty main"
keyid: 6E34E7116C0BC933
EOF
# ua.list:
# source: deb http://ppa.launchpad.net/canonical-server/ua-client-daily/ubuntu \$RELEASE main
# keyid: 94E187AD53A59D1847E4880F8A295C4FB8B190B7
cat > checkaptpolicy.sh <<EOF
#!/bin/bash
set -x
RELEASE=\`lsb_release -sc\`
echo -n "Current release: $RELEASE"
echo "Assert no disabled valid mirrors in /eta/apt/sources.list"
! grep disable /etc/apt/sources.list || echo "FAILURE: found disabled valid mirror urls"
echo "Checking commercial Ubuntu Advantage PPAs apt policy and config"
apt-cache policy | grep esm.ubuntu.com
for file in \`ls /etc/apt/sources.list.d/ubuntu-*.list\`; do
echo "--- file: \${file}"
cat \${file}
done
EOF
chmod 755 checkaptpolicy.sh
declare -A NEXTDIST=( [bionic]=focal [xenial]=bionic [trusty]=xenial )
wait_for_boot() {
local vm=$1 release=$2
echo "--- Wait for cloud-init to finish"
if [ "${release}" = "trusty" ]; then
while [ "N 2" != "$(lxc exec ${vm} -- runlevel)" ]; do
echo "waiting on runlevel 2"
sleep 5
done
status=$(lxc exec ${vm} -- test -f /var/run/cloud-init/result.json && echo "done" || echo "running" )
while [ "done" != "${status}" ]; do
status=$(lxc exec ${vm} -- test -f /var/run/cloud-init/result.json && echo "done" || echo "running" )
echo -n '.'
sleep 5
done
else
lxc exec ${vm} -- cloud-init status --wait --long
fi
}
for release in trusty; do
vm=test-sru-$release
echo "--- Launch cloud-init with ${release}-proposed enabled"
lxc launch ubuntu-daily:${release} ${vm} -c user.user-data="$(cat test-uru.yaml)"
upgraderelease=${NEXTDIST[$release]}
wait_for_boot ${vm} ${release}
echo "--- Attach Ubuntu-Advantage and enable services"
lxc exec ${vm} -- ua attach ${UA_TOKEN}
lxc exec ${vm} -- ua status --wait | tee ua-status.orig
lxc file push checkaptpolicy.sh ${vm}/
lxc exec ${vm} -- /checkaptpolicy.sh > policy.orig
lxc exec ${vm} -- wget http://archive.ubuntu.com/ubuntu/dists/$upgraderelease-proposed/main/dist-upgrader-all/current/$upgraderelease.tar.gz
lxc exec ${vm} -- tar xzvf $upgraderelease.tar.gz
echo "--- Add proposed PPA to valid mirrors to exercise ua-tools do-release-upgrade"
lxc file pull ${vm}/root/mirrors.cfg .
sed -i 's/stable/proposed/' mirrors.cfg
lxc file push mirrors.cfg ${vm}/root/
lxc exec ${vm} -- /root/$upgraderelease --datadir=/root --frontend DistUpgradeViewNonInteractive
lxc exec ${vm} -- reboot || true
wait_for_boot ${vm} ${upgraderelease}
echo "--- Validate UA APT sources after upgrade"
lxc exec ${vm} -- /checkaptpolicy.sh > policy.upgrade
lxc exec test-sru-bionic grep disable /etc/apt/sources.list && "FAILURE: valid mirrors got disabled" || echo "SUCCESS: no valid mirrors disabled"
echo "--- Ensure UA status reports ESM disabled due to pending RT"
lxc exec ${vm} -- ua status --wait | tee ua-status.upgrade
egrep 'esm-infra.*disabled' ua-status.upgrade || echo "FAILURE: unexpected enabled esm-infra"
echo "--- Expect disable reason to be no Release file in esm PPA"
lxc exec test-sru-bionic grep disable /var/log/dist-upgrade/main.log || echo "FAILURE: didn't find disabled update logs for esm"
echo "--- Expect disabled esm-infra in diffs from original and upgrade status"
diff -urN ua-status.orig ua-status.upgrade
done
[Regression Potential]
None as UA support is not yet officially introduced in Xenial or later.
This patch only allows anyone currently overriding PostUpgradeScripts in DistUpgrade.cfg to perform apt update|install operations, which were previously prohibited due to apt lock.
=== End SRU Template ===
==== Original Description ===
In order for custom PostInstallScript to add/remove apt packages and call apt update, DistUpgradecontroller needs to release the apt cache directory lock before that stage.
If the lock is still in place, DistUpgradeController.runPostUpgradeScripts hits errors such as:
E: Could not get lock /var/lib/apt/lists/lock - open (11: Resource temporarily unavailable)
E: Unable to lock directory /var/lib/apt/lists/ |
|
| 2020-10-10 02:40:30 |
Chad Smith |
attachment added |
|
uru-upgrade-to-focal-sru.log https://bugs.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/+bug/1897778/+attachment/5420256/+files/uru-upgrade-to-focal-sru.log |
|
| 2020-10-10 02:41:42 |
Chad Smith |
attachment added |
|
uru-upgrade-to-bionic-sru.log https://bugs.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/+bug/1897778/+attachment/5420261/+files/uru-upgrade-to-bionic-sru.log |
|
| 2020-10-10 02:42:00 |
Chad Smith |
tags |
verification-needed verification-needed-bionic verification-needed-focal verification-needed-xenial |
verification-done-bionic verification-done-focal verification-needed verification-needed-xenial |
|
| 2020-10-10 20:08:12 |
Chad Smith |
description |
=== Begin SRU Template ===
[Impact]
Any Ubuntu Advantage apt-based service is enbled via a PPA. If those PPAs are not listed as valid mirrors in mirrors.cfg the PPAs get disabled across upgrade.
UA-client has a script which will enable those PPAs across upgrade path, but needs the apt cache lock released during runPostInstallScripts.
Validate Ubuntu Advantage apt access is retained across do-release-upgrade path so customers to not lose access to security updates after performing an upgrade.
[Test Case]
#!/bin/bash
#
# SRU Verification ubuntu-release-upgrader + ubuntu=advantage-tools
# Test procedure:
# - launch container Trusty, Xenial or Bionic
# - Attach container to UA subscription (which activates a number of commerical PPAs
# - download and run -proposed ubuntu-release-upgrader tool for upgrade release
# - Assert successful upgrade
# - Confirm valid mirrors not disabled
# - Confirm third party non-commercial PPA URLs still disabled
# - Confirm third party UA commercial URLs still disabled
# (due to expected feature gap)
# - Confirm UA status reports esm-infra still disabled (known feature gap)
set -ex
UA_TOKEN=$1
if [ -z "$1" ]; then
echo "Usage: $0 <contractTOKEN>"
exit 1
fi
# sources:
# ua.proposed:
# source: deb http://ppa.launchpad.net/ua-client/proposed/ubuntu \$RELEASE main
# keyid: 6E34E7116C0BC933
cat > test-uru.yaml <<EOF
#cloud-config
package_update: true
package_upgrade: true
apt:
apt_sources:
- source: "deb http://ppa.launchpad.net/ua-client/proposed/ubuntu trusty main"
keyid: 6E34E7116C0BC933
EOF
# ua.list:
# source: deb http://ppa.launchpad.net/canonical-server/ua-client-daily/ubuntu \$RELEASE main
# keyid: 94E187AD53A59D1847E4880F8A295C4FB8B190B7
cat > checkaptpolicy.sh <<EOF
#!/bin/bash
set -x
RELEASE=\`lsb_release -sc\`
echo -n "Current release: $RELEASE"
echo "Assert no disabled valid mirrors in /eta/apt/sources.list"
! grep disable /etc/apt/sources.list || echo "FAILURE: found disabled valid mirror urls"
echo "Checking commercial Ubuntu Advantage PPAs apt policy and config"
apt-cache policy | grep esm.ubuntu.com
for file in \`ls /etc/apt/sources.list.d/ubuntu-*.list\`; do
echo "--- file: \${file}"
cat \${file}
done
EOF
chmod 755 checkaptpolicy.sh
declare -A NEXTDIST=( [bionic]=focal [xenial]=bionic [trusty]=xenial )
wait_for_boot() {
local vm=$1 release=$2
echo "--- Wait for cloud-init to finish"
if [ "${release}" = "trusty" ]; then
while [ "N 2" != "$(lxc exec ${vm} -- runlevel)" ]; do
echo "waiting on runlevel 2"
sleep 5
done
status=$(lxc exec ${vm} -- test -f /var/run/cloud-init/result.json && echo "done" || echo "running" )
while [ "done" != "${status}" ]; do
status=$(lxc exec ${vm} -- test -f /var/run/cloud-init/result.json && echo "done" || echo "running" )
echo -n '.'
sleep 5
done
else
lxc exec ${vm} -- cloud-init status --wait --long
fi
}
for release in trusty; do
vm=test-sru-$release
echo "--- Launch cloud-init with ${release}-proposed enabled"
lxc launch ubuntu-daily:${release} ${vm} -c user.user-data="$(cat test-uru.yaml)"
upgraderelease=${NEXTDIST[$release]}
wait_for_boot ${vm} ${release}
echo "--- Attach Ubuntu-Advantage and enable services"
lxc exec ${vm} -- ua attach ${UA_TOKEN}
lxc exec ${vm} -- ua status --wait | tee ua-status.orig
lxc file push checkaptpolicy.sh ${vm}/
lxc exec ${vm} -- /checkaptpolicy.sh > policy.orig
lxc exec ${vm} -- wget http://archive.ubuntu.com/ubuntu/dists/$upgraderelease-proposed/main/dist-upgrader-all/current/$upgraderelease.tar.gz
lxc exec ${vm} -- tar xzvf $upgraderelease.tar.gz
echo "--- Add proposed PPA to valid mirrors to exercise ua-tools do-release-upgrade"
lxc file pull ${vm}/root/mirrors.cfg .
sed -i 's/stable/proposed/' mirrors.cfg
lxc file push mirrors.cfg ${vm}/root/
lxc exec ${vm} -- /root/$upgraderelease --datadir=/root --frontend DistUpgradeViewNonInteractive
lxc exec ${vm} -- reboot || true
wait_for_boot ${vm} ${upgraderelease}
echo "--- Validate UA APT sources after upgrade"
lxc exec ${vm} -- /checkaptpolicy.sh > policy.upgrade
lxc exec test-sru-bionic grep disable /etc/apt/sources.list && "FAILURE: valid mirrors got disabled" || echo "SUCCESS: no valid mirrors disabled"
echo "--- Ensure UA status reports ESM disabled due to pending RT"
lxc exec ${vm} -- ua status --wait | tee ua-status.upgrade
egrep 'esm-infra.*disabled' ua-status.upgrade || echo "FAILURE: unexpected enabled esm-infra"
echo "--- Expect disable reason to be no Release file in esm PPA"
lxc exec test-sru-bionic grep disable /var/log/dist-upgrade/main.log || echo "FAILURE: didn't find disabled update logs for esm"
echo "--- Expect disabled esm-infra in diffs from original and upgrade status"
diff -urN ua-status.orig ua-status.upgrade
done
[Regression Potential]
None as UA support is not yet officially introduced in Xenial or later.
This patch only allows anyone currently overriding PostUpgradeScripts in DistUpgrade.cfg to perform apt update|install operations, which were previously prohibited due to apt lock.
=== End SRU Template ===
==== Original Description ===
In order for custom PostInstallScript to add/remove apt packages and call apt update, DistUpgradecontroller needs to release the apt cache directory lock before that stage.
If the lock is still in place, DistUpgradeController.runPostUpgradeScripts hits errors such as:
E: Could not get lock /var/lib/apt/lists/lock - open (11: Resource temporarily unavailable)
E: Unable to lock directory /var/lib/apt/lists/ |
=== Begin SRU Template ===
[Impact]
Any Ubuntu Advantage apt-based service is enbled via a PPA. If those PPAs are not listed as valid mirrors in mirrors.cfg the PPAs get disabled across upgrade.
UA-client has a script which will enable those PPAs across upgrade path, but needs the apt cache lock released during runPostInstallScripts.
Validate Ubuntu Advantage apt access is retained across do-release-upgrade path so customers to not lose access to security updates after performing an upgrade.
[Test Case]
#!/bin/bash
#
# SRU Verification ubuntu-release-upgrader + ubuntu=advantage-tools
# Test procedure:
# - launch container Trusty, Xenial or Bionic
# - Attach container to UA subscription (which activates a number of commerical PPAs
# - download and run -proposed ubuntu-release-upgrader tool for upgrade release
# - Assert successful upgrade
# - Confirm valid mirrors not disabled
# - Confirm third party non-commercial PPA URLs still disabled
# - Confirm third party UA commercial URLs still disabled
# (due to expected feature gap)
# - Confirm UA status reports esm-infra still disabled (known feature gap)
set -e
UA_TOKEN=$1
if [ -z "$1" ]; then
echo "Usage: $0 <contractTOKEN>"
exit 1
fi
# sources:
# ua.proposed:
# source: deb http://ppa.launchpad.net/ua-client/proposed/ubuntu \$RELEASE main
# keyid: 6E34E7116C0BC933
# sources:
# ua.proposed:
# source: deb http://ppa.launchpad.net/canonical-server/ua-client-daily/ubuntu \$RELEASE main
# keyid: 94E187AD53A59D1847E4880F8A295C4FB8B190B7
cat > test-uru.yaml <<EOF
#cloud-config
package_update: true
package_upgrade: true
apt:
sources:
ua.daily:
source: deb http://ppa.launchpad.net/canonical-server/ua-client-daily/ubuntu \$RELEASE main
keyid: 94E187AD53A59D1847E4880F8A295C4FB8B190B7
EOF
# ua.list:
# source: deb http://ppa.launchpad.net/canonical-server/ua-client-daily/ubuntu \$RELEASE main
# keyid: 94E187AD53A59D1847E4880F8A295C4FB8B190B7
cat > checkaptpolicy.sh <<EOF
#!/bin/bash
set -x
RELEASE=\`lsb_release -sc\`
echo -n "Current release: $RELEASE"
echo "Assert no disabled valid mirrors in /eta/apt/sources.list"
! grep disable /etc/apt/sources.list || echo "FAILURE: found disabled valid mirror urls"
echo "Checking commercial Ubuntu Advantage PPAs apt policy and config"
apt-cache policy | grep esm.ubuntu.com
for file in \`ls /etc/apt/sources.list.d/ubuntu-*.list\`; do
echo "--- file: \${file}"
cat \${file}
done
EOF
chmod 755 checkaptpolicy.sh
declare -A NEXTDIST=( [bionic]=focal [xenial]=bionic [trusty]=xenial )
wait_for_boot() {
local vm=$1 release=$2
echo "--- Wait for cloud-init to finish"
sleep 5
if [ "${release}" = "trusty" ]; then
while [ "N 2" != "$(lxc exec ${vm} -- runlevel)" ]; do
echo "waiting on runlevel 2"
sleep 5
done
status=$(lxc exec ${vm} -- test -f /var/run/cloud-init/result.json && echo "done" || echo "running" )
while [ "done" != "${status}" ]; do
status=$(lxc exec ${vm} -- test -f /var/run/cloud-init/result.json && echo "done" || echo "running" )
echo -n '.'
sleep 5
done
else
lxc exec ${vm} -- cloud-init status --wait --long
fi
}
for release in xenial; do
vm=test-sru-$release
echo "--- Launch cloud-init with ua-client-${release}-daily enabled"
lxc launch ubuntu-daily:${release} ${vm} -c user.user-data="$(cat test-uru.yaml)"
upgraderelease=${NEXTDIST[$release]}
if [ "${release}" != "trusty" ]; then
dist_upgrade_disable_reason="no Release"
fi
wait_for_boot ${vm} ${release}
echo "--- Attach Ubuntu-Advantage, enabling services"
lxc exec ${vm} -- ua attach ${UA_TOKEN}
lxc exec ${vm} -- ua status --wait | tee ua-status.orig
lxc file push checkaptpolicy.sh ${vm}/
lxc exec ${vm} -- /checkaptpolicy.sh > policy.orig
lxc exec ${vm} -- wget http://archive.ubuntu.com/ubuntu/dists/$upgraderelease-proposed/main/dist-upgrader-all/current/$upgraderelease.tar.gz
lxc exec ${vm} -- tar xzvf $upgraderelease.tar.gz
echo "--- Add proposed PPA to valid mirrors to exercise ua-tools do-release-upgrade"
lxc file pull ${vm}/root/mirrors.cfg .
echo "http://ppa.launchpad.net/canonical-server/ua-client-daily/ubuntu/" >> mirrors.cfg
lxc file push mirrors.cfg ${vm}/root/
lxc exec ${vm} -- /root/$upgraderelease --datadir=/root --frontend DistUpgradeViewNonInteractive | egrep 'Warning|Error|ubuntu-advantage'
lxc exec ${vm} -- reboot || true
sleep 10
lxc exec ${vm} grep disable /etc/apt/sources.list && "FAILURE: valid mirrors got disabled" || echo "SUCCESS: no valid mirrors disabled"
lxc exec ${vm} -- ua status --wait | tee ua-status.upgrade
if [ ! -z "${dist_upgrade_disable_reason}" ]; then
echo "--- Expect do-release-upgrade disable reason to be '$dist_upgrade_disable_reason' for ESM* PPAs"
lxc exec ${vm} grep "${dist_upgrade_disable_reason}" /var/log/dist-upgrade/main.log || echo "FAILURE: didn't find disabled update logs for esm"
fi
echo "--- Expect no diffs from original and upgraded ua status"
status_diff=$(diff -urN ua-status.orig ua-status.upgrade)
[ -z "$status_diff" ] && echo "SUCCESS: no diff" || echo -e "FAILURE: unexpected diff\n${status_diff}"
echo "--- Ensure UA re-enabled esm across upgrade"
lxc exec ${vm} -- egrep 'lts|change' /var/log/ubuntu-advantage.log
echo "--- Ensure UA status reports ESM enabled after upgrade"
egrep 'esm-infra.*enabled' ua-status.upgrade || echo "FAILURE: expected enabled esm-infra after upgrade"
done
[Regression Potential]
None as UA support is not yet officially introduced in Xenial or later.
This patch only allows anyone currently overriding PostUpgradeScripts in DistUpgrade.cfg to perform apt update|install operations, which were previously prohibited due to apt lock.
=== End SRU Template ===
==== Original Description ===
In order for custom PostInstallScript to add/remove apt packages and call apt update, DistUpgradecontroller needs to release the apt cache directory lock before that stage.
If the lock is still in place, DistUpgradeController.runPostUpgradeScripts hits errors such as:
E: Could not get lock /var/lib/apt/lists/lock - open (11: Resource temporarily unavailable)
E: Unable to lock directory /var/lib/apt/lists/ |
|
| 2020-10-10 21:20:38 |
Chad Smith |
attachment added |
|
uru-upgrade-to-focal-sru.log https://bugs.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/+bug/1897778/+attachment/5420649/+files/uru-upgrade-to-focal-sru.log |
|
| 2020-10-10 21:21:10 |
Chad Smith |
attachment added |
|
uru-upgrade-to-bionic-sru.log https://bugs.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/+bug/1897778/+attachment/5420650/+files/uru-upgrade-to-bionic-sru.log |
|
| 2020-10-10 21:21:35 |
Chad Smith |
attachment added |
|
uru-upgrade-to-xenial-sru.log https://bugs.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/+bug/1897778/+attachment/5420651/+files/uru-upgrade-to-xenial-sru.log |
|
| 2020-10-10 21:21:48 |
Chad Smith |
tags |
verification-done-bionic verification-done-focal verification-needed verification-needed-xenial |
verification-done verification-done-bionic verification-done-focal verification-done-xenial |
|
| 2020-10-16 03:16:32 |
Chris Halse Rogers |
removed subscriber Ubuntu Stable Release Updates Team |
|
|
|
| 2020-10-16 03:17:38 |
Launchpad Janitor |
ubuntu-release-upgrader (Ubuntu Focal): status |
Fix Committed |
Fix Released |
|
| 2020-10-16 03:18:05 |
Launchpad Janitor |
ubuntu-release-upgrader (Ubuntu Bionic): status |
Fix Committed |
Fix Released |
|
| 2020-10-16 03:18:43 |
Launchpad Janitor |
ubuntu-release-upgrader (Ubuntu Xenial): status |
Fix Committed |
Fix Released |
|
