| 2019-05-08 16:04:23 |
Dmitrii Shcherbakov |
bug |
|
|
added bug |
| 2019-05-08 16:04:47 |
Dmitrii Shcherbakov |
bug |
|
|
added subscriber Corey Bryant |
| 2019-05-08 16:05:07 |
Dmitrii Shcherbakov |
bug |
|
|
added subscriber Chris Sanders |
| 2019-05-08 16:05:21 |
Dmitrii Shcherbakov |
bug |
|
|
added subscriber Canonical Field Critical |
| 2019-05-08 16:05:30 |
Dmitrii Shcherbakov |
description |
[rocky][19.04] Upgrading a DVR deployment from Queens to Rocky purged neutron-l3-agent package
Upgrading OpenStack from Queens to Rocky results in removal of neutron-l3-agent package which leads to a networking outage.
The issue with the fact that first python3 packages are installed and then python2 packages are purged. The neutron-l3-agent package depends on both.
At the same time neutron-dhcp-agent survives the upgrade while both packages seem to have identical "Depends" directives for python[3]-neutron:
https://git.launchpad.net/~ubuntu-server-dev/ubuntu/+source/neutron/tree/debian/control?h=stable/rocky#n178
Package: neutron-l3-agent
Architecture: all
Depends:
conntrack,
iputils-arping,
keepalived,
neutron-metadata-agent (= ${source:Version}),
python-neutron (= ${source:Version}) | python3-neutron (= ${source:Version}),
python-neutron-fwaas (>= 1:13.0.0~) | python3-neutron-fwaas (>= 1:13.0.0~),
https://git.launchpad.net/~ubuntu-server-dev/ubuntu/+source/neutron/tree/debian/control?h=stable/rocky#n156
Package: neutron-dhcp-agent
Architecture: all
Depends:
dnsmasq-base (>= 2.76),
dnsmasq-utils (>= 2.76),
neutron-metadata-agent (= ${source:Version}),
python-neutron (= ${source:Version}) | python3-neutron (= ${source:Version}),
When I try to install neutron-l3-agent it tries to pull python2 dependencies again (even though the python3 ones are already there):
https://paste.ubuntu.com/p/J7Vq8KKnBf/
More info below:
------
2019-05-07 16:42:41 DEBUG neutron-plugin-api-relation-changed Selecting previously unselected package neutron-l3-agent.
2019-05-07 16:42:41 DEBUG neutron-plugin-api-relation-changed Preparing to unpack .../6-neutron-l3-agent_2%3a12.0.5-0ubuntu1_all.deb ...
2019-05-07 16:42:41 DEBUG neutron-plugin-api-relation-changed Unpacking neutron-l3-agent (2:12.0.5-0ubuntu1) ...
2019-05-07 16:42:47 DEBUG neutron-plugin-api-relation-changed Setting up neutron-l3-agent (2:12.0.5-0ubuntu1) ...
2019-05-07 16:42:47 DEBUG neutron-plugin-api-relation-changed Created symlink /etc/systemd/system/multi-user.target.wants/neutron-l3-agent.service → /lib/systemd/syst
em/neutron-l3-agent.service.
/var/log/apt/term.log:
update-alternatives: using /usr/bin/python3-nova-metadata-wsgi to provide /usr/bin/nova-metadata-wsgi (nova-metadata-wsgi) in auto mode
(Reading database ... ^M(Reading database ... 5%^M(Reading database ... 10%^M(Reading database ... 15%^M(Reading database ... 20%^M(Reading database ... 25%^M(Reading database ... 30%^M(Reading database ... 35%^M(Reading database ... 40%^M(Reading database ... 45%^M(Reading database ... 50%^M(Reading database ... 55%^M(Reading database ... 60%^M(Reading database ... 65%^M(Reading database ... 70%^M(Reading database ... 75%^M(Reading database ... 80%^M(Reading database ... 85%^M(Reading database ... 90%^M(Reading database ... 95%^M(Reading database ... 100%^M(Reading database ... 113229 files and directories currently installed.)
Purging configuration files for python-neutron (2:13.0.2-0ubuntu1~cloud0) ...
Purging configuration files for neutron-l3-agent (2:13.0.2-0ubuntu1~cloud0) ...
Purging configuration files for python-neutron-fwaas (1:13.0.1-0ubuntu1~cloud0) ...
Purging configuration files for python-nova (2:18.1.0-0ubuntu1~cloud0) ...
Processing triggers for ureadahead (0.100.0-21) ...
Processing triggers for systemd (237-3ubuntu10.21) ...
Log ended: 2019-05-08 13:46:55
/var/log/apt/history.log:
# install python3 packages
Start-Date: 2019-05-08 13:45:18
Commandline: apt-get --assume-yes --option=Dpkg::Options::=--force-confold install nova-compute genisoimage librbd1 xfsprogs nfs-common ceph-common nova-compute-kvm n
ova-api-metadata python3-nova python3-memcache python3-rados python3-rbd python3-neutron
# purge python2 packages
Start-Date: 2019-05-08 13:46:40
Commandline: apt-get --assume-yes purge python-neutron python-nova
Purge: neutron-l3-agent:amd64 (2:13.0.2-0ubuntu1~cloud0), python-nova:amd64 (2:18.1.0-0ubuntu1~cloud0), python-neutron:amd64 (2:13.0.2-0ubuntu1~cloud0), python-neutron-fwaas:amd64 (1:13.0.1-0ubuntu1~cloud0)
End-Date: 2019-05-08 13:46:55
apt policy python3-neutron
https://paste.ubuntu.com/p/3vKsFBf9Hn/
apt-rdepends:
neutron-l3-agent
Depends: conntrack
Depends: iputils-arping
Depends: keepalived
Depends: neutron-metadata-agent (= 2:13.0.2-0ubuntu1~cloud0)
Depends: python-neutron (= 2:13.0.2-0ubuntu1~cloud0)
Depends: python-neutron-fwaas (>= 1:13.0.0~)
Depends: python3-neutron (= 2:13.0.2-0ubuntu1~cloud0)
Depends: python3-neutron-fwaas (>= 1:13.0.0~)
# ...
neutron-dhcp-agent
Depends: dnsmasq-base (>= 2.76)
Depends: dnsmasq-utils (>= 2.76)
Depends: neutron-metadata-agent (= 2:13.0.2-0ubuntu1~cloud0)
Depends: python-neutron (= 2:13.0.2-0ubuntu1~cloud0)
Depends: python3-neutron (= 2:13.0.2-0ubuntu1~cloud0)
# ...
Charm code:
https://opendev.org/openstack/charm-neutron-openvswitch/src/branch/stable/19.04/hooks/neutron_ovs_utils.py#L323-L325
if cmp_release >= 'rocky':
return PURGE_PACKAGES
return []
https://opendev.org/openstack/charm-neutron-openvswitch/src/branch/master/hooks/neutron_ovs_utils.py#L119-L122
PURGE_PACKAGES = [
'python-neutron',
'python-neutron-fwaas',
]
https://opendev.org/openstack/charm-neutron-openvswitch/src/branch/master/hooks/neutron_ovs_hooks.py#L115-L124
install_packages()
install_tmpfilesd()
# NOTE(jamespage): purge any packages as a result of py3 switch
# at rocky.
packages_to_purge = determine_purge_packages()
request_nova_compute_restart = False
if packages_to_purge:
purge_packages(packages_to_purge)
request_nova_compute_restart = True |
Upgrading OpenStack from Queens to Rocky results in removal of neutron-l3-agent package which leads to a networking outage.
The issue with the fact that first python3 packages are installed and then python2 packages are purged. The neutron-l3-agent package depends on both.
At the same time neutron-dhcp-agent survives the upgrade while both packages seem to have identical "Depends" directives for python[3]-neutron:
https://git.launchpad.net/~ubuntu-server-dev/ubuntu/+source/neutron/tree/debian/control?h=stable/rocky#n178
Package: neutron-l3-agent
Architecture: all
Depends:
conntrack,
iputils-arping,
keepalived,
neutron-metadata-agent (= ${source:Version}),
python-neutron (= ${source:Version}) | python3-neutron (= ${source:Version}),
python-neutron-fwaas (>= 1:13.0.0~) | python3-neutron-fwaas (>= 1:13.0.0~),
https://git.launchpad.net/~ubuntu-server-dev/ubuntu/+source/neutron/tree/debian/control?h=stable/rocky#n156
Package: neutron-dhcp-agent
Architecture: all
Depends:
dnsmasq-base (>= 2.76),
dnsmasq-utils (>= 2.76),
neutron-metadata-agent (= ${source:Version}),
python-neutron (= ${source:Version}) | python3-neutron (= ${source:Version}),
When I try to install neutron-l3-agent it tries to pull python2 dependencies again (even though the python3 ones are already there):
https://paste.ubuntu.com/p/J7Vq8KKnBf/
More info below:
------
2019-05-07 16:42:41 DEBUG neutron-plugin-api-relation-changed Selecting previously unselected package neutron-l3-agent.
2019-05-07 16:42:41 DEBUG neutron-plugin-api-relation-changed Preparing to unpack .../6-neutron-l3-agent_2%3a12.0.5-0ubuntu1_all.deb ...
2019-05-07 16:42:41 DEBUG neutron-plugin-api-relation-changed Unpacking neutron-l3-agent (2:12.0.5-0ubuntu1) ...
2019-05-07 16:42:47 DEBUG neutron-plugin-api-relation-changed Setting up neutron-l3-agent (2:12.0.5-0ubuntu1) ...
2019-05-07 16:42:47 DEBUG neutron-plugin-api-relation-changed Created symlink /etc/systemd/system/multi-user.target.wants/neutron-l3-agent.service → /lib/systemd/syst
em/neutron-l3-agent.service.
/var/log/apt/term.log:
update-alternatives: using /usr/bin/python3-nova-metadata-wsgi to provide /usr/bin/nova-metadata-wsgi (nova-metadata-wsgi) in auto mode
(Reading database ... ^M(Reading database ... 5%^M(Reading database ... 10%^M(Reading database ... 15%^M(Reading database ... 20%^M(Reading database ... 25%^M(Reading database ... 30%^M(Reading database ... 35%^M(Reading database ... 40%^M(Reading database ... 45%^M(Reading database ... 50%^M(Reading database ... 55%^M(Reading database ... 60%^M(Reading database ... 65%^M(Reading database ... 70%^M(Reading database ... 75%^M(Reading database ... 80%^M(Reading database ... 85%^M(Reading database ... 90%^M(Reading database ... 95%^M(Reading database ... 100%^M(Reading database ... 113229 files and directories currently installed.)
Purging configuration files for python-neutron (2:13.0.2-0ubuntu1~cloud0) ...
Purging configuration files for neutron-l3-agent (2:13.0.2-0ubuntu1~cloud0) ...
Purging configuration files for python-neutron-fwaas (1:13.0.1-0ubuntu1~cloud0) ...
Purging configuration files for python-nova (2:18.1.0-0ubuntu1~cloud0) ...
Processing triggers for ureadahead (0.100.0-21) ...
Processing triggers for systemd (237-3ubuntu10.21) ...
Log ended: 2019-05-08 13:46:55
/var/log/apt/history.log:
# install python3 packages
Start-Date: 2019-05-08 13:45:18
Commandline: apt-get --assume-yes --option=Dpkg::Options::=--force-confold install nova-compute genisoimage librbd1 xfsprogs nfs-common ceph-common nova-compute-kvm n
ova-api-metadata python3-nova python3-memcache python3-rados python3-rbd python3-neutron
# purge python2 packages
Start-Date: 2019-05-08 13:46:40
Commandline: apt-get --assume-yes purge python-neutron python-nova
Purge: neutron-l3-agent:amd64 (2:13.0.2-0ubuntu1~cloud0), python-nova:amd64 (2:18.1.0-0ubuntu1~cloud0), python-neutron:amd64 (2:13.0.2-0ubuntu1~cloud0), python-neutron-fwaas:amd64 (1:13.0.1-0ubuntu1~cloud0)
End-Date: 2019-05-08 13:46:55
apt policy python3-neutron
https://paste.ubuntu.com/p/3vKsFBf9Hn/
apt-rdepends:
neutron-l3-agent
Depends: conntrack
Depends: iputils-arping
Depends: keepalived
Depends: neutron-metadata-agent (= 2:13.0.2-0ubuntu1~cloud0)
Depends: python-neutron (= 2:13.0.2-0ubuntu1~cloud0)
Depends: python-neutron-fwaas (>= 1:13.0.0~)
Depends: python3-neutron (= 2:13.0.2-0ubuntu1~cloud0)
Depends: python3-neutron-fwaas (>= 1:13.0.0~)
# ...
neutron-dhcp-agent
Depends: dnsmasq-base (>= 2.76)
Depends: dnsmasq-utils (>= 2.76)
Depends: neutron-metadata-agent (= 2:13.0.2-0ubuntu1~cloud0)
Depends: python-neutron (= 2:13.0.2-0ubuntu1~cloud0)
Depends: python3-neutron (= 2:13.0.2-0ubuntu1~cloud0)
# ...
Charm code:
https://opendev.org/openstack/charm-neutron-openvswitch/src/branch/stable/19.04/hooks/neutron_ovs_utils.py#L323-L325
if cmp_release >= 'rocky':
return PURGE_PACKAGES
return []
https://opendev.org/openstack/charm-neutron-openvswitch/src/branch/master/hooks/neutron_ovs_utils.py#L119-L122
PURGE_PACKAGES = [
'python-neutron',
'python-neutron-fwaas',
]
https://opendev.org/openstack/charm-neutron-openvswitch/src/branch/master/hooks/neutron_ovs_hooks.py#L115-L124
install_packages()
install_tmpfilesd()
# NOTE(jamespage): purge any packages as a result of py3 switch
# at rocky.
packages_to_purge = determine_purge_packages()
request_nova_compute_restart = False
if packages_to_purge:
purge_packages(packages_to_purge)
request_nova_compute_restart = True |
|
| 2019-05-08 16:51:21 |
Launchpad Janitor |
neutron (Ubuntu): status |
New |
Confirmed |
|
| 2019-05-08 18:41:23 |
Corey Bryant |
attachment added |
|
lp-1828259-no-charm.txt https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1828259/+attachment/5262411/+files/lp-1828259-no-charm.txt |
|
| 2019-05-08 19:02:32 |
Corey Bryant |
attachment added |
|
lp-1828259-with-charm.txt https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1828259/+attachment/5262432/+files/lp-1828259-with-charm.txt |
|
| 2019-05-08 19:55:13 |
Corey Bryant |
attachment added |
|
lp-1828259-py3-installed-but-installs-py2.txt https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1828259/+attachment/5262439/+files/lp-1828259-py3-installed-but-installs-py2.txt |
|
| 2019-05-08 19:59:43 |
Corey Bryant |
neutron (Ubuntu): status |
Confirmed |
Incomplete |
|
| 2019-05-08 21:26:58 |
Corey Bryant |
attachment added |
|
bundle-snippet.txt https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1828259/+attachment/5262456/+files/bundle-snippet.txt |
|
| 2019-05-08 23:23:23 |
Dmitrii Shcherbakov |
bug task added |
|
charm-neutron-openvswitch |
|
| 2019-05-08 23:23:35 |
Dmitrii Shcherbakov |
bug task added |
|
charm-nova-compute |
|
| 2019-05-09 12:04:44 |
Corey Bryant |
charm-nova-compute: status |
New |
Triaged |
|
| 2019-05-09 12:04:46 |
Corey Bryant |
neutron (Ubuntu): status |
Incomplete |
Invalid |
|
| 2019-05-09 12:04:48 |
Corey Bryant |
charm-neutron-openvswitch: importance |
Undecided |
High |
|
| 2019-05-09 12:04:50 |
Corey Bryant |
charm-nova-compute: importance |
Undecided |
High |
|
| 2019-05-09 12:04:52 |
Corey Bryant |
charm-neutron-openvswitch: status |
New |
Triaged |
|
| 2019-05-09 12:53:46 |
Corey Bryant |
charm-neutron-openvswitch: status |
Triaged |
Opinion |
|
| 2019-05-09 12:53:48 |
Corey Bryant |
charm-neutron-openvswitch: importance |
High |
Undecided |
|
| 2019-05-09 18:29:56 |
Corey Bryant |
charm-nova-compute: status |
Triaged |
In Progress |
|
| 2019-05-09 18:29:59 |
Corey Bryant |
charm-nova-compute: assignee |
|
Corey Bryant (corey.bryant) |
|
| 2019-05-10 15:36:07 |
OpenStack Infra |
charm-nova-compute: status |
In Progress |
Fix Committed |
|
| 2019-05-10 21:06:58 |
Dmitrii Shcherbakov |
charm-nova-compute: milestone |
|
19.04 |
|
| 2019-05-10 21:07:21 |
Dmitrii Shcherbakov |
charm-nova-compute: status |
Fix Committed |
Fix Released |
|
