From Rocky to Stein, router in one network node creation issue

Bug #1869855 reported by Kevin Zhao
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kolla-ansible
Invalid
Undecided
Unassigned

Bug Description

Upgrade from Rocky to Stein,

After that we find 1 nodes behavior is weird.
The router here are:
root@uk-dc-cavium-01:~# ip netns exec qrouter-44cc73d8-fb3c-408b-be22-682931cb1fc1 ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever

While in other nodes:
root@uk-dc-cavium-02:~# ip netns exec qrouter-44cc73d8-fb3c-408b-be22-682931cb1fc1 ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2841: ha-be3caa07-49: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8950 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether fa:16:3e:69:4c:14 brd ff:ff:ff:ff:ff:ff
    inet 169.254.192.21/18 brd 169.254.255.255 scope global ha-be3caa07-49
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe69:4c14/64 scope link
       valid_lft forever preferred_lft forever
2875: qr-d4dab906-11: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8950 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether fa:16:3e:71:e2:fa brd ff:ff:ff:ff:ff:ff
2941: qg-5e90ae7f-1a: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether fa:16:3e:50:dd:ee brd ff:ff:ff:ff:ff:ff

Or:
root@uk-dc-cavium-03:~# ip netns exec qrouter-44cc73d8-fb3c-408b-be22-682931cb1fc1 ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
350: ha-eb12605a-94: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8950 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether fa:16:3e:f2:39:3f brd ff:ff:ff:ff:ff:ff
    inet 169.254.192.8/18 brd 169.254.255.255 scope global ha-eb12605a-94
       valid_lft forever preferred_lft forever
    inet 169.254.0.1/24 scope global ha-eb12605a-94
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fef2:393f/64 scope link
       valid_lft forever preferred_lft forever
375: qr-d4dab906-11: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8950 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether fa:16:3e:71:e2:fa brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.1/24 scope global qr-d4dab906-11
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe71:e2fa/64 scope link nodad
       valid_lft forever preferred_lft forever
397: qg-5e90ae7f-1a: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether fa:16:3e:50:dd:ee brd ff:ff:ff:ff:ff:ff
    inet 213.146.141.112/32 scope global qg-5e90ae7f-1a
       valid_lft forever preferred_lft forever
    inet 213.146.141.33/25 scope global qg-5e90ae7f-1a
       valid_lft forever preferred_lft forever
    inet 213.146.155.37/32 scope global qg-5e90ae7f-1a
       valid_lft forever preferred_lft forever
    inet 213.146.155.76/32 scope global qg-5e90ae7f-1a
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe50:ddee/64 scope link nodad
       valid_lft forever preferred_lft forever

=================================================================

In the problem node, when IP addr, we can get a lot of interface:

root@uk-dc-cavium-01:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: control: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000
    link/ether 1c:1b:0d:98:97:38 brd ff:ff:ff:ff:ff:ff
    inet 10.20.20.3/16 brd 10.20.255.255 scope global control
       valid_lft forever preferred_lft forever
    inet6 fe80::1e1b:dff:fe98:9738/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 1c:1b:0d:98:97:39 brd ff:ff:ff:ff:ff:ff
4: public: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000
    link/ether 1c:1b:0d:98:97:3a brd ff:ff:ff:ff:ff:ff
    inet 213.146.141.4/25 brd 213.146.141.127 scope global public
       valid_lft forever preferred_lft forever
    inet6 fe80::1e1b:dff:fe98:973a/64 scope link
       valid_lft forever preferred_lft forever
5: eth3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 1c:1b:0d:98:97:3b brd ff:ff:ff:ff:ff:ff
6: eth4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 1c:1b:0d:98:97:3c brd ff:ff:ff:ff:ff:ff
7: internal: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000
    link/ether ec:0d:9a:d9:87:f6 brd ff:ff:ff:ff:ff:ff
    inet 10.22.20.1/16 brd 10.22.255.255 scope global internal
       valid_lft forever preferred_lft forever
    inet6 fe80::ee0d:9aff:fed9:87f6/64 scope link
       valid_lft forever preferred_lft forever
8: external: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq master ovs-system state UP group default qlen 1000
    link/ether ec:0d:9a:d9:87:f7 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::ee0d:9aff:fed9:87f7/64 scope link
       valid_lft forever preferred_lft forever
9: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:39:d1:c3:b3 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 scope global docker0
       valid_lft forever preferred_lft forever
10: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 6a:f3:52:7e:8a:9c brd ff:ff:ff:ff:ff:ff
290: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 46:3c:3b:9f:4e:41 brd ff:ff:ff:ff:ff:ff
40: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 2a:dd:1f:9d:26:46 brd ff:ff:ff:ff:ff:ff
307: br-ex: <BROADCAST,MULTICAST> mtu 9000 qdisc noop state DOWN group default qlen 1000
    link/ether ec:0d:9a:d9:87:f7 brd ff:ff:ff:ff:ff:ff
52: tap40915253-f5: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 8a:8a:d0:81:f1:73 brd ff:ff:ff:ff:ff:ff
59: tapf2ec4a58-e0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 8e:da:1a:9b:fc:8c brd ff:ff:ff:ff:ff:ff
846: vxlan_sys_4789: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65535 qdisc noqueue master ovs-system state UNKNOWN group default qlen 1000
    link/ether 92:c0:23:d8:5e:1c brd ff:ff:ff:ff:ff:ff
    inet6 fe80::90c0:23ff:fed8:5e1c/64 scope link
       valid_lft forever preferred_lft forever
600: qg-2ac6f1b1-06: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 8e:dd:8e:ff:b1:d4 brd ff:ff:ff:ff:ff:ff
605: tap92dabaa3-b7: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 26:00:5b:fc:93:f8 brd ff:ff:ff:ff:ff:ff
606: ha-30a37c5a-45: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 72:ba:c1:8c:4d:4b brd ff:ff:ff:ff:ff:ff
607: ha-48c277e3-dd: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether ce:2b:5d:0f:59:9e brd ff:ff:ff:ff:ff:ff
609: ha-6548d93e-15: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether de:fc:7b:24:2e:90 brd ff:ff:ff:ff:ff:ff
611: qg-d8c6dc57-f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 86:7b:8c:83:df:b9 brd ff:ff:ff:ff:ff:ff
620: qr-4f055af8-29: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether b2:1f:5d:78:50:37 brd ff:ff:ff:ff:ff:ff
621: qg-1d13a0c3-58: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 06:c8:7c:00:16:3d brd ff:ff:ff:ff:ff:ff
624: tap976105c8-88: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 2a:6c:c0:4d:47:02 brd ff:ff:ff:ff:ff:ff
625: qg-536d5df4-c2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 3a:54:7f:4f:9d:f5 brd ff:ff:ff:ff:ff:ff
629: ha-12734cef-cc: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 46:39:6c:f6:6a:b8 brd ff:ff:ff:ff:ff:ff
631: ha-0356e11d-64: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 56:17:6d:44:a3:32 brd ff:ff:ff:ff:ff:ff
636: tapa37a33ae-53: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 5a:b2:77:a6:ad:17 brd ff:ff:ff:ff:ff:ff
639: qg-28def6aa-ad: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 6a:db:3d:a8:b5:ae brd ff:ff:ff:ff:ff:ff
641: ha-62e9a3e1-e2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 7a:e0:ac:9b:7c:b4 brd ff:ff:ff:ff:ff:ff
642: qg-5e90ae7f-1a: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 02:02:66:87:a6:ab brd ff:ff:ff:ff:ff:ff
643: qr-15b97210-b5: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether be:1b:13:e8:d2:45 brd ff:ff:ff:ff:ff:ff
645: ha-9ad84712-b3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 8e:2d:4a:9c:ae:44 brd ff:ff:ff:ff:ff:ff
647: ha-ca347ff7-c7: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 6a:e4:5c:45:8b:d8 brd ff:ff:ff:ff:ff:ff
648: qr-894f7b63-82: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether f6:ab:4e:32:cc:67 brd ff:ff:ff:ff:ff:ff
652: qg-d36393c8-a1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 62:be:50:35:2e:3a brd ff:ff:ff:ff:ff:ff
654: qg-a4f5970a-9d: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether c6:4f:75:01:39:a3 brd ff:ff:ff:ff:ff:ff
656: tapa673812c-a0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether b6:5d:95:92:db:15 brd ff:ff:ff:ff:ff:ff
657: qg-381a8968-50: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 32:cc:0c:78:ba:17 brd ff:ff:ff:ff:ff:ff
658: qg-9e6dea7d-ae: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 8e:6a:53:dc:fa:b0 brd ff:ff:ff:ff:ff:ff
659: qg-8c92a80d-59: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 4e:88:a2:2c:1c:c8 brd ff:ff:ff:ff:ff:ff
660: qr-f4c29d10-73: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether b2:43:4c:72:b6:02 brd ff:ff:ff:ff:ff:ff
662: qg-f5035cdb-40: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 3a:02:aa:70:08:d9 brd ff:ff:ff:ff:ff:ff
666: qr-36c19935-bd: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 42:48:57:ca:c2:68 brd ff:ff:ff:ff:ff:ff
667: qr-aa9493a4-19: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state D

===================================================

While in common node:
root@uk-dc-cavium-03:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: control: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000
    link/ether 1c:1b:0d:94:ab:0a brd ff:ff:ff:ff:ff:ff
    inet 10.20.20.6/16 brd 10.20.255.255 scope global control
       valid_lft forever preferred_lft forever
    inet6 fe80::1e1b:dff:fe94:ab0a/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 1c:1b:0d:94:ab:0b brd ff:ff:ff:ff:ff:ff
4: public: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000
    link/ether 1c:1b:0d:94:ab:0c brd ff:ff:ff:ff:ff:ff
    inet 213.146.141.5/25 brd 213.146.141.127 scope global public
       valid_lft forever preferred_lft forever
    inet6 fe80::1e1b:dff:fe94:ab0c/64 scope link
       valid_lft forever preferred_lft forever
5: eth3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 1c:1b:0d:94:ab:0d brd ff:ff:ff:ff:ff:ff
6: eth4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 1c:1b:0d:94:ab:0e brd ff:ff:ff:ff:ff:ff
7: internal: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000
    link/ether ec:0d:9a:d9:84:22 brd ff:ff:ff:ff:ff:ff
    inet 10.22.20.3/16 brd 10.22.255.255 scope global internal
       valid_lft forever preferred_lft forever
    inet6 fe80::ee0d:9aff:fed9:8422/64 scope link
       valid_lft forever preferred_lft forever
8: external: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq master ovs-system state UP group default qlen 1000
    link/ether ec:0d:9a:d9:84:23 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::ee0d:9aff:fed9:8423/64 scope link
       valid_lft forever preferred_lft forever
9: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:10:de:9a:a8 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 scope global docker0
       valid_lft forever preferred_lft forever
10: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 1a:94:c8:96:03:90 brd ff:ff:ff:ff:ff:ff
13: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 16:d7:77:34:81:49 brd ff:ff:ff:ff:ff:ff
532: vxlan_sys_4789: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65535 qdisc noqueue master ovs-system state UNKNOWN group default qlen 1000
    link/ether 06:e7:52:63:79:ce brd ff:ff:ff:ff:ff:ff
    inet6 fe80::4e7:52ff:fe63:79ce/64 scope link
       valid_lft forever preferred_lft forever
31: br-ex: <BROADCAST,MULTICAST> mtu 9000 qdisc noop state DOWN group default qlen 1000
    link/ether ec:0d:9a:d9:84:23 brd ff:ff:ff:ff:ff:ff
84: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 82:3d:a9:24:bf:4e brd ff:ff:ff:ff:ff:ff
89: tap4650386a-47: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 1a:61:02:a6:4e:75 brd ff:ff:ff:ff:ff:ff

Revision history for this message
Kevin Zhao (kevin-zhao) wrote :
Download full text (7.6 KiB)

Common router log:

653216:2020-03-31 10:10:42.439 35 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-ef38069e-ba1c-41e7-92ff-2b7346cb2167', 'sysctl', '-w', 'net.ipv6.conf.qg-a5a56605-b9.forwarding=0'] create_process /var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/utils.py:87
653325:2020-03-31 10:10:43.660 35 DEBUG neutron.agent.l3.ha [-] Closing metadata proxy for router ef38069e-ba1c-41e7-92ff-2b7346cb2167 _update_metadata_proxy /var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/l3/ha.py:213
653444:2020-03-31 10:10:44.697 35 DEBUG neutron.agent.l3.router_info [-] Terminating radvd daemon in router device: ef38069e-ba1c-41e7-92ff-2b7346cb2167 disable_radvd /var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/l3/router_info.py:550
653445:2020-03-31 10:10:44.699 35 DEBUG neutron.agent.linux.utils [-] Unable to access /var/lib/neutron/external/pids/ef38069e-ba1c-41e7-92ff-2b7346cb2167.pid.radvd get_value_from_file /var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/utils.py:261
653446:2020-03-31 10:10:44.701 35 DEBUG neutron.agent.linux.utils [-] Unable to access /var/lib/neutron/external/pids/ef38069e-ba1c-41e7-92ff-2b7346cb2167.pid.radvd get_value_from_file /var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/utils.py:261
653447:2020-03-31 10:10:44.702 35 DEBUG neutron.agent.linux.external_process [-] No process started for ef38069e-ba1c-41e7-92ff-2b7346cb2167 disable /var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/external_process.py:122
653448:2020-03-31 10:10:44.703 35 DEBUG neutron.agent.linux.ra [-] radvd disabled for router ef38069e-ba1c-41e7-92ff-2b7346cb2167 disable /var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/ra.py:198
653460:2020-03-31 10:10:45.334 35 DEBUG neutron.agent.l3.ha [-] Updating server with HA routers states {'c30a2186-3606-4feb-9a57-1f44088b858c': 'active', 'a62e219b-b0d8-4204-ab75-a640defab8da': 'active', 'ef38069e-ba1c-41e7-92ff-2b7346cb2167': 'standby'} notify_server /var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/l3/ha.py:229
653585:2020-03-31 10:10:48.572 35 DEBUG neutron.agent.l3.agent [req-c3b04c82-5718-4413-81bc-aa6d001df008 - - - - -] Got routers updated notification :[u'ef38069e-ba1c-41e7-92ff-2b7346cb2167'] routers_updated /var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/l3/agent.py:497
653586:2020-03-31 10:10:48.577 35 INFO neutron.agent.l3.agent [-] Starting router update for ef38069e-ba1c-41e7-92ff-2b7346cb2167, action 3, priority 1, update_id d3fa8e74-a463-4856-b323-afd30769ce67. Wait time elapsed: 0.002
653836:2020-03-31 10:10:59.971 35 DEBUG neutron.agent.l3.router_info [-] Process updates, router ef38069e-ba1c-41e7-92ff-2b7346cb2167 process /var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/l3/router_info.py:1183
653849:2020-03-31 10:11:00.745 35 DEBUG oslo_concurrency.lockutils [-] Acquired lock "iptables-qrouter-ef38069e-ba1c-41e7-92ff-2b7346cb2167" lock /var/lib/kolla/venv/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:265
653850:2020-03-31 10:11...

Read more...

Revision history for this message
Kevin Zhao (kevin-zhao) wrote :
Download full text (16.2 KiB)

Issue router log in l3:
656707:2020-03-31 10:13:50.428 35 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-44cc73d8-fb3c-408b-be22-682931cb1fc1', 'iptables-save'] create_process /var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/utils.py:87
656714:2020-03-31 10:13:51.254 35 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-44cc73d8-fb3c-408b-be22-682931cb1fc1', 'ip6tables-save'] create_process /var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/utils.py:87
656722:2020-03-31 10:13:51.992 35 DEBUG oslo_concurrency.lockutils [-] Releasing lock "iptables-qrouter-44cc73d8-fb3c-408b-be22-682931cb1fc1" lock /var/lib/kolla/venv/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:281
656723:2020-03-31 10:13:51.994 35 DEBUG neutron.agent.l3.ha_router [-] Processing HA router with HA port: {u'allowed_address_pairs': [], u'extra_dhcp_opts': [], u'updated_at': u'2020-03-31T09:13:41Z', u'device_owner': u'network:router_ha_interface', u'revision_number': 24, u'port_security_enabled': False, u'binding:profile': {}, u'binding:vnic_type': u'normal', u'fixed_ips': [{u'subnet_id': u'0bc00bb1-9c15-47e0-83c5-706134235cd4', u'prefixlen': 18, u'ip_address': u'169.254.192.4'}], u'id': u'd05bc2a7-a29a-46b7-8c4f-238a1fc0b0bc', u'security_groups': [], u'binding:vif_details': {u'port_filter': True, u'datapath_type': u'system', u'ovs_hybrid_plug': True}, u'address_scopes': {u'4': None, u'6': None}, u'binding:vif_type': u'ovs', u'mac_address': u'fa:16:3e:a2:b5:66', u'project_id': u'', u'status': u'ACTIVE', u'subnets': [{u'dns_nameservers': [], u'ipv6_ra_mode': None, u'gateway_ip': None, u'cidr': u'169.254.192.0/18', u'id': u'0bc00bb1-9c15-47e0-83c5-706134235cd4', u'subnetpool_id': None}], u'binding:host_id': u'uk-dc-cavium-01', u'description': u'', u'tags': [], u'device_id': u'44cc73d8-fb3c-408b-be22-682931cb1fc1', u'name': u'HA port tenant a9b848288ea842b6a3ee1966fe96aa4b', u'admin_state_up': True, u'network_id': u'cc4b14d5-2864-4f7b-823b-33898f939b01', u'tenant_id': u'', u'created_at': u'2018-09-05T21:55:38Z', u'mtu': 8950, u'extra_subnets': []} process /var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py:488
656725:2020-03-31 10:13:51.997 35 DEBUG neutron.agent.linux.keepalived [-] Router 44cc73d8-fb3c-408b-be22-682931cb1fc1 keepalived config: global_defs {
656731: script "/var/lib/neutron/ha_confs/44cc73d8-fb3c-408b-be22-682931cb1fc1/ha_check_script_1.sh"
656768:2020-03-31 10:13:52.001 35 DEBUG neutron.agent.linux.utils [-] Unable to access /var/lib/neutron/ha_confs/44cc73d8-fb3c-408b-be22-682931cb1fc1.pid.keepalived get_value_from_file /var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/utils.py:261
656769:2020-03-31 10:13:52.004 35 DEBUG neutron.agent.linux.utils [-] Unable to access /var/lib/neutron/ha_confs/44cc73d8-fb3c-408b-be22-682931cb1fc1.pid.keepalived-vrrp get_value_from_file /var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/utils.py:261
656770:2020-03-31 10:13:52.007 35 DE...

Revision history for this message
Michal Nasiadka (mnasiadka) wrote :

Stein EOL

Changed in kolla-ansible:
status: New → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.