network DHCP agent ports status is DOWN in multi network creation simultaneously

Bug #1968859 reported by shahab taee
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
New
Medium
Unassigned

Bug Description

when we creating multi network (about 5 networks and even more) and their subnet using command line or terraform at the same time, DHCP ports status of some of these networks marked as Down,we see all logs in debug mode and can't find any error or warning related to this issue, this issue can be solved when we restart "neutron-dhcp-agent.service" and Port synchronization is performed.
we are using openstack victoria nad neutron version "17.2.2".
please let me know how we can resolve this issue.

Revision history for this message
shahab taee (shahabtaee) wrote :
Revision history for this message
LIU Yulong (dragon889) wrote :

How many schelduled networks/ports on your running agents? How many neutron RPC workers do you have? This looks like the resource scale issue for L2 agent or DHCP agent. If your cloud has no customized code, you can upgrade to Xena to use distributed DHCP. And then you can stop dhcp agents.

Revision history for this message
shahab taee (shahabtaee) wrote :

we currently have 140 networks In our cluster, we are going to create 5 networks with 3 DHCP ports, also we have 16 rpc_workers, we haven't set any limitation for our agents.

Changed in neutron:
importance: Undecided → Medium
Revision history for this message
Rodolfo Alonso (rodolfo-alonso-hernandez) wrote :

Hello Shahab:

The DHCP port must be created by the DHCP agent and then bound by OVS (if you are using OVS backend). You should see something like this in the Neutron server:
  Provisioning complete for port 4fa9d6bd-ef07-4e9b-badb-fabcd9572088 triggered by entity DHCP.
  Provisioning complete for port 4fa9d6bd-ef07-4e9b-badb-fabcd9572088 triggered by entity L2.

This is the message you'll see in the DHCP agent:
  DHCP configuration for ports {'4fa9d6bd-ef07-4e9b-badb-fabcd9572088'} is completed

And this in the OVS agent:
  Setting status for 4fa9d6bd-ef07-4e9b-badb-fabcd9572088 to UP

Please, try to reproduce the error with the DHCP agent, OVS agent and Neutron server in debug mode, post the logs and the network, subnets and DHCP ports IDs. That will help us to debug this issue.

Regards.

Revision history for this message
shahab taee (shahabtaee) wrote :

Dear Rodolfo

we faced this issue after upgrade from "ussuri" to "victoria".
about logs you mentioned above, we compare a faulty network with healthy network logs and found faulty network receives one less update.
finally, we have resolved this issue by disabling ipv6 in our network hosts as we don't use IPv6 in our setup, but we can't find any relation between main issue and the solution.
do you have any opinion about this resolution

best regards

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.