when I use dvr mode for neutron,it's not functional

[zhangss]

cat /etc/kolla/globals.yaml
enable_neutron_dvr: "yes"

kolla-ansible -i multinode deploy

when not bind floating ip ,vm can ping public network, but once bind floating ip ,vm can not ping public network,and it's look like dvr mode not functional.

[Radosław Piliszek (yoctozepto)]

Hi Zhang, anything in logs? Especially neutron-server, neutron-l3-agent and neutron-openvswitch-agent (or linuxbridge if you are using it instead).

[zhangss (intentc)]

Hello, my situation is that when using non distributed routing to set up an external network, the floating IP created can be Pingtong, but when using distributed routing to set up an external network (all through a physical network card, Flattype), when creating floating IP to bind to VM instance, it is impossible to Ping the external network gateway and the Internet from the inside of the instance. Looking at the logs from the network and computing nodes, I haven't found any exceptions at present, and I'm confused now.

[zhangss (intentc)]

This is distributed router's details.

| admin_state_up | UP |
| availability_zone_hints | |
| availability_zones | nova |
| created_at | 2020-05-17T10:30:48Z |
| description | ...

[Radosław Piliszek (yoctozepto)]

triaging it in https://review.opendev.org/728954

[Radosław Piliszek (yoctozepto)]

I could not reproduce.

[zhangss (intentc)]

Is it because of kolla yaml's enable_ neutron_ DVR parameters conflict with other parameters, which makes DVR mode unavailable and cannot be connected to the external network after binding DVR router's floating IP.

[Radosław Piliszek (yoctozepto)]

Is it a question? :-) Are there any other parameters that you are setting?

[zhangss (intentc)]

I'm not sure which parameter, but in the DVR mode of kolla ansible deployment, when the external network of flat is set as the gateway in the distributed routing, the floating IP created cannot be used on the virtual machine.
After the floating IP is bound, the virtual machine cannot Ping the floating IP through the external network and the virtual machine.

[Radosław Piliszek (yoctozepto)]

Well, it certainly works (both ping and ssh) to the vip address. The setup is just as you described. There is a flat network acting as external for a distributed router. The source is some other machine in the flat network, the destination is the instance with vip address. Are you trying something else there?

[zhangss (intentc)]

The problem is when I set enable_ neutron_ DVR: "yes", after the deployment, the floating IP generated by DVR cannot be pinged, and when the floating IP is bound, it cannot be pinged from the internal virtual machine to the external network. I'm not sure if I need to set other parameters to yes or no in yaml file.

[Radosław Piliszek (yoctozepto)]

Hmm, what release is that? (e.g. Train) What distro? I'm notifying Neutron because it works for me and I have no hints. It should really say something in logs.

[zhangss (intentc)]

Release is stein .
This is all logs about dvr floating IP,I cannot find any error.

2020-05-19 13:34:31.534 77 DEBUG neutron.agent.l3.dvr_fip_ns [-] Create FIP link interfaces for router 6b60bbb0-9aae-4934-8e8a-b87950e40c47 create_rtr_2_fip_link /usr/lib/python2.7/site-packages/neutron/agent/l3/dvr_fip_ns.py:410
2020-05-19 13:34:31.671 77 DEBUG oslo_service.periodic_task [req-4054e986-9b34-4ffe-8e3f-d7f31b9ec33c - - - - -] Running periodic task L3NATAgentWithStateReport.periodic_sync_routers_task run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:217
2020-05-19 13:34:32.847 77 DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-lock-fip-e54f20d6-4815-45de-90b9-4cc567f8bfe6-fg-d8163890-a8" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:265
2020-05-19 13:34:32.900 77 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'fip-e54f20d6-4815-45de-90b9-4cc567f8bfe6', 'ip', '-4', 'route', 'replace', 'default', 'via', '10.16.32.1', 'dev', 'fg-d8163890-a8', 'table', '2852022899'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:87
2020-05-19 13:34:32.939 77 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'fip-e54f20d6-4815-45de-90b9-4cc567f8bfe6', 'arping', '-U', '-I', 'fg-d8163890-a8', '-c', '1', '-w', '1.5', '10.16.32.146'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:87
2020-05-19 13:34:33.271 77 DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-lock-fip-e54f20d6-4815-45de-90b9-4cc567f8bfe6-fg-d8163890-a8" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:281
2020-05-19 13:34:33.310 77 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'fip-e54f20d6-4815-45de-90b9-4cc567f8bfe6', 'arping', '-A', '-I', 'fg-d8163890-a8', '-c', '1', '-w', '1.5', '10.16.32.146'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:87
2020-05-19 13:34:33.400 77 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-6b60bbb0-9aae-4934-8e8a-b87950e40c47', 'ip', '-4', 'route', 'replace', 'default', 'via', '169.254.106.115', 'dev', 'rfp-6b60bbb0-9', 'table', '16'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:87
2020-05-19 13:34:33.765 77 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-6b60bbb0-9aae-4934-8e8a-b87950e40c47', 'sysctl', '-w', 'net.ipv4.conf.all.send_redirects=0'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:87
2020-05-19 13:34:34.143 77 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-6b60bbb0-9aae-4934-8e8a-b87950e40c47', 'ip', '-4', 'route', 'replace', 'default', 'via', '192.168.30.33', 'dev', 'qr-8c241e1e-7f', ...

[Maciej Jozefczyk (maciejjozefczyk)]

Do I understand it correctly that you first deploy environment without specifying DVR and then specify it?
What about bridge-mappings on compute hosts? Those are properly configured?
Do you have created those external bridges on compute hosts and plugged the flat network there?

[Maciej Jozefczyk (maciejjozefczyk)]

Please paste Neutron configuration from controller and compute where the FIP VM is not reachable, along with bridge setup.

[Slawek Kaplonski (slaweq)]

To debug that from Neutron point of view, please include logs from the neutron agents as Maciej mentioned above, but also please add output of commands like:

openstack network agent list

and then

openstack network agent show <agent_id>

for each L3 and ovs agent. There should be info about L3 agent's mode and if distributed routing is enabled in the neutron-ovs-agent's config.

[Launchpad Janitor (janitor)]

[Expired for neutron because there has been no activity for 60 days.]

[Launchpad Janitor (janitor)]

[Expired for kolla-ansible because there has been no activity for 60 days.]