switching from external-network-id and external-port to data-port and bridge-mappings does not remove incorrect nics from bridges
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Charms Deployment Guide |
Fix Released
|
Medium
|
Alex Kavanagh | ||
OpenStack Neutron Gateway Charm |
Invalid
|
High
|
Unassigned | ||
OpenStack Neutron Open vSwitch Charm |
Invalid
|
Undecided
|
Unassigned |
Bug Description
charm cs:neutron-
I upgraded a site from Mitaka to Newton. There are 4 neutron-gateway applications, one for each external network, hosting a set of routers that connect to those nets.
The charm config includes a setting for external-network-id and external-port. Each of the applications has a different nic for the external network, i.e. eth1, eth2, eth3 and eth4, so if I have application gateway3 it is using eth3 as the external-port.
Each of the external networks is configured in openstack as such:
:~$ neutron net-show ext_net
+------
| Field | Value |
+------
| admin_state_up | True |
| availability_
| availability_zones | nova |
| created_at | 2016-08-
| description | |
| id | foo |
| ipv4_address_scope | |
| ipv6_address_scope | |
| is_default | False |
| l2_adjacency | True |
| mtu | 1458 |
| name | ext_net |
| project_id | foo |
| provider:
| provider:
| provider:
| revision_number | 0 |
| router:external | True |
| shared | False |
| status | ACTIVE |
| subnets | foo |
| tags | |
| tenant_id | foo |
| updated_at | 2016-09-
+------
On running the openstack-upgrade action on the neutron-gateway-X units, network connectivity to the external networks was lost.
After some time, we discovered that these deprecated options should have been changed out, so we reset external-
data-port=
We also used mysql edits to reconfigure the networks to have network_type of flat, and the physical_network set to whichever of physnet1 or whatever it needed.
We expected this to reconfigure the networks correctly, but in fact what happened is that the 'old' interface, e.g. eth3, was left in br-ex as well as the eth1 added. We probably should have just created an entirely new bridge rather than re-using br-ex. The two interfaces in the bridge caused some kind of storm and the entire physical network was saturated.
While this is clearly a design fault from the deployment point of view, it would be good to firstly have some massive warning flags about these older configs breaking on upgrade, also if an interface is in a bridge that shouldn't be it would be good to remove it rather than leave it there.
Changed in charm-neutron-gateway: | |
importance: | Undecided → High |
milestone: | none → 19.04 |
assignee: | nobody → Frode Nordahl (fnordahl) |
Changed in charm-neutron-gateway: | |
assignee: | Frode Nordahl (fnordahl) → Alex Kavanagh (ajkavanagh) |
Changed in charm-deployment-guide: | |
status: | New → In Progress |
assignee: | nobody → Alex Kavanagh (ajkavanagh) |
importance: | Undecided → Medium |
Changed in charm-neutron-gateway: | |
milestone: | 19.04 → 19.07 |
Changed in charm-neutron-gateway: | |
milestone: | 19.07 → 19.10 |
Changed in charm-neutron-gateway: | |
milestone: | 19.10 → 20.01 |
Changed in charm-deployment-guide: | |
status: | Fix Committed → Fix Released |
Changed in charm-neutron-gateway: | |
milestone: | 20.01 → 20.05 |
Changed in charm-neutron-gateway: | |
assignee: | Alex Kavanagh (ajkavanagh) → Aurelien Lourot (aurelien-lourot) |
Changed in charm-neutron-gateway: | |
milestone: | 20.05 → 20.08 |
Changed in charm-neutron-gateway: | |
milestone: | 20.08 → none |
Changed in charm-neutron-gateway: | |
milestone: | none → 20.08 |
milestone: | 20.08 → none |
milestone: | none → 20.10 |
Changed in charm-neutron-gateway: | |
milestone: | 20.10 → 21.01 |
Changed in charm-neutron-gateway: | |
milestone: | 21.01 → none |
On Wed, Dec 19, 2018 at 8:40 PM Xav Paice <email address hidden> wrote:
After some time, we discovered that these deprecated options should have network- id, and external-port to
> been changed out, so we reset external-
> default and configured all 4 gateway applications
The options should still work even though they are deprecated. If they are
in config.yaml and not removed they should still work. If they don't that's
a bug in the charm.
It would be nice to have a formal way to run a health check on config
options prior to charm upgrade. For example if ext-port had been removed in
a charm release you'd be able to run it and find out.