Activity log for bug #2060460

Date Who What changed Old value New value Message
2024-04-08 11:15:40 Francesco De Simone bug added bug
2024-04-08 20:49:00 Francesco De Simone description Hi, I am currently facing a bug during a microcloud deployment. If I bootstrap three nodes and establish an uplink connection, then try to configure network forward ip, one of the machines becomes unreachable. This issue does not occur with the latest/edge version of microovn snap list: ``` Name Version Rev Tracking Publisher Notes core20 20240111 2182 latest/stable canonical✓ base core22 20240111 1122 latest/stable canonical✓ base lxd 5.20-f3dd836 27049 latest/stable canonical✓ in-cohort microceph 0+git.4a608fc 793 quincy/stable canonical✓ in-cohort microcloud 1.1-04a1c49 734 latest/stable canonical✓ in-cohort microovn 22.03.3+snap0e23a0e4f5 395 22.03/stable canonical✓ in-cohort snapd 2.61.2 21184 latest/stable canonical✓ snapd ``` lxc network show default: ``` config: bridge.mtu: "1442" ipv4.address: 10.99.49.1/24 ipv4.nat: "true" ipv6.address: fd42:42b0:f235:53cb::1/64 ipv6.nat: "true" network: UPLINK volatile.network.ipv4.address: 10.205.17.135 description: "" name: default type: ovn used_by: - /1.0/instances/u1 - /1.0/instances/u2 - /1.0/instances/u3 - /1.0/profiles/default managed: true status: Created locations: - microcloudp01 - microcloudp03 - microcloudp02 ``` lxc network show UPLINK: ``` config: ipv4.gateway: 10.205.17.129/25 ipv4.ovn.ranges: 10.205.17.135-10.205.17.199 ipv4.routes: 10.205.17.200/29 volatile.last_state.created: "false" description: "" name: UPLINK type: physical used_by: - /1.0/networks/default managed: true status: Created locations: - microcloudp01 - microcloudp03 - microcloudp02 ``` how to reproduce: add a forward ip to a container: ``` lxc network forward create default 10.205.17.201 target_address=10.99.49.4 ``` Move the container around the cluster and test its reachability until one node is not reachable: ``` ping 10.205.17.201 PING 10.205.17.201 (10.205.17.201) 56(84) bytes of data. 64 bytes from 10.205.17.201: icmp_seq=1 ttl=63 time=4.12 ms 64 bytes from 10.205.17.201: icmp_se: 3 time=1.60 ms --- 10.205.17.201 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1001ms $ lxc stop u1 $ lxc move u1 --target microcloudp02 $ lxc start u1 $ ping 10.205.17.201 PING 10.205.17.201 (10.205.17.201) 56(84) bytes of data. --- 10.205.17.201 ping statistics --- 4 packets transmitted, 0 received, 100% packet loss, time 3076ms ``` Hi, I am currently facing a bug during a microcloud deployment. If I bootstrap three nodes and establish an uplink connection, then try to configure network forward ip, one of the machines becomes unreachable. This issue does not occur with the latest/edge version of microovn snap list: ``` Name Version Rev Tracking Publisher Notes core20 20240111 2182 latest/stable canonical✓ base core22 20240111 1122 latest/stable canonical✓ base lxd 5.20-f3dd836 27049 latest/stable canonical✓ in-cohort microceph 0+git.4a608fc 793 quincy/stable canonical✓ in-cohort microcloud 1.1-04a1c49 734 latest/stable canonical✓ in-cohort microovn 22.03.3+snap0e23a0e4f5 395 22.03/stable canonical✓ in-cohort snapd 2.61.2 21184 latest/stable canonical✓ snapd ``` lxc network show default: ``` config:   bridge.mtu: "1442"   ipv4.address: 10.99.49.1/24   ipv4.nat: "true"   ipv6.address: fd42:42b0:f235:53cb::1/64   ipv6.nat: "true"   network: UPLINK   volatile.network.ipv4.address: 10.205.17.135 description: "" name: default type: ovn used_by: - /1.0/instances/u1 - /1.0/instances/u2 - /1.0/instances/u3 - /1.0/profiles/default managed: true status: Created locations: - microcloudp01 - microcloudp03 - microcloudp02 ``` lxc network show UPLINK: ``` config:   ipv4.gateway: 10.205.17.129/25   ipv4.ovn.ranges: 10.205.17.135-10.205.17.199   ipv4.routes: 10.205.17.200/29   volatile.last_state.created: "false" description: "" name: UPLINK type: physical used_by: - /1.0/networks/default managed: true status: Created locations: - microcloudp01 - microcloudp03 - microcloudp02 ``` how to reproduce: add a forward ip to a container: ``` lxc network forward create default 10.205.17.201 target_address=10.99.49.4 ``` Move the container around the cluster and test its reachability until one node is not reachable: ``` ping 10.205.17.201 PING 10.205.17.201 (10.205.17.201) 56(84) bytes of data. 64 bytes from 10.205.17.201: icmp_seq=1 ttl=63 time=4.12 ms 64 bytes from 10.205.17.201: icmp_se: 3 time=1.60 ms --- 10.205.17.201 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1001ms $ lxc stop u1 $ lxc move u1 --target microcloudp02 $ lxc start u1 $ ping 10.205.17.201 PING 10.205.17.201 (10.205.17.201) 56(84) bytes of data. --- 10.205.17.201 ping statistics --- 4 packets transmitted, 0 received, 100% packet loss, time 3076ms ``` UPDATE: adding some additional logs: https://pastebin.canonical.com/p/fr6mNG6gDz/
2024-04-11 08:14:27 Francesco De Simone bug added subscriber Canonical Field High
2024-04-11 12:17:15 Frode Nordahl microovn: status New Confirmed
2024-04-16 08:59:43 Frode Nordahl summary Network forward feature is not functioning on a random node within the cluster with microovn 22.03/stable [22.03] ICMP to network forward listen address not forwarded for instance colocated with active gateway chassis
2024-04-16 09:11:41 Frode Nordahl summary [22.03] ICMP to network forward listen address not forwarded for instance colocated with active gateway chassis [22.03] ICMP to network forward listen address not forwarded for instance colocated with chassis bound to chassisredirect port