In HA, there is a potential race condition between the openvswitch agent and other agents that "own", depend on or manipulate ports. As the neutron server resumes on a failover it will not immediately be aware of openvswitch agents that have also been activated on failover and act as though there are no active openvswitch agents (this is an example, it most likely affects other L2 agents). If an agent such as the L3 agent starts and begins resync before the neutron server is aware of the active openvswitch agent, ports for the routers on that agent will be marked as "binding_failed". Currently this is a "terminal" state for the port as neutron does not attempt to rebind failed bindings on the same host.
Unfortunately, the neutron agents do not provide even a best-effort deterministic indication to the outside service manager (systemd, pacemaker, etc...) that it has fully initialized and the neutron server should be aware that it is active. Agents should follow the same pattern as wsgi based services and notify systemd after it can be reasonably assumed that the neutron server should be aware that it is alive. That way service startup order logic or constraints can properly start an agent that is dependent on other agents *after* neutron should be aware that the required agents are active.
I've triaged this bug myself, you can reproduce it by:
1) starting a 2 or 3 network nodes, and setting up ha routers
2) creating a few ha routers (10 would suffice)
3) stopping ovs-agent & l3-agent & dhcp agent on all the nodes for T>agent_down_time
4) starting them all at once.
like 50% of the time:
1) l3-agent will try to rebind some of the router ports before any ovs-agent has reported himself (via heartbeat) as UP.
2) The result is the port being moved into binding failed status.
3) Then ovs-agent boots up, and marks the ports as dead internal VLAN (4095).
4) This recovers if you restart the l3-agent again, because that tries again to rebind the port, and some agent is up now.
[5) I'm not sure now if you needed to restart OVS agent again or not]