metering-agent failed to get traffic counters when no router-namespace where meter-label-rules were added

Bug #1572548 reported by Sergey Belous
22
This bug affects 4 people
Affects Status Importance Assigned to Milestone
neutron
Won't Fix
Low
Sana Khan

Bug Description

Removing router from l3 agent cause errors from neutron-metering-agent. The neutron-meter-agent continues try to get traffic counters from iptables on router-namespace, which is not exist after removing router from l3-agent.

Steps to reproduce:
1. Create internal net, subnet, router. Set external gateway for router, add interface to router for created net.
2. Create neutron-meter-label
3. Use 'neutron l3-agent-router-remove' command to remove a router from a L3 agent
Trace in neutron-metering-agent logs:
2016-04-20 12:28:26.699 ERROR neutron.agent.linux.utils [-] Exit code: 1; Stdin: ; Stdout: ; Stderr: Cannot open network namespace "qrouter-c8acd926-88e5-4292-883e-e928fb3f6d32": No such file or directory

2016-04-20 12:28:26.700 ERROR neutron.services.metering.drivers.iptables.iptables_driver [-] Failed to get traffic counters, router: {u'status': u'ACTIVE', u'name': u'router05', u'gw_port_id': u'f36b30d3-5290-4896-837c-108b8cd4f3dc', u'admin_state_up': True, u'tenant_id': u'1c0eb24bdbb1406bb7d1346f36064ebd', u'_metering_labels': [{u'rules': [{u'remote_ip_prefix': u'0.0.0.0/0', u'direction': u'egress', u'metering_label_id': u'67dae290-38cb-4962-80ab-d6d3404dc6df', u'id': u'799f3361-6f90-4e36-a338-311d6e7c9d5b', u'excluded': False}], u'id': u'67dae290-38cb-4962-80ab-d6d3404dc6df'}], u'id': u'c8acd926-88e5-4292-883e-e928fb3f6d32'}
2016-04-20 12:28:26.700 TRACE neutron.services.metering.drivers.iptables.iptables_driver Traceback (most recent call last):
2016-04-20 12:28:26.700 TRACE neutron.services.metering.drivers.iptables.iptables_driver File "/opt/stack/neutron/neutron/services/metering/drivers/iptables/iptables_driver.py", line 355, in get_traffic_counters
2016-04-20 12:28:26.700 TRACE neutron.services.metering.drivers.iptables.iptables_driver chain, wrap=False, zero=True)
2016-04-20 12:28:26.700 TRACE neutron.services.metering.drivers.iptables.iptables_driver File "/opt/stack/neutron/neutron/agent/linux/iptables_manager.py", line 712, in get_traffic_counters
2016-04-20 12:28:26.700 TRACE neutron.services.metering.drivers.iptables.iptables_driver current_table = self.execute(args, run_as_root=True)
2016-04-20 12:28:26.700 TRACE neutron.services.metering.drivers.iptables.iptables_driver File "/opt/stack/neutron/neutron/agent/linux/utils.py", line 137, in execute
2016-04-20 12:28:26.700 TRACE neutron.services.metering.drivers.iptables.iptables_driver raise RuntimeError(msg)
2016-04-20 12:28:26.700 TRACE neutron.services.metering.drivers.iptables.iptables_driver RuntimeError: Exit code: 1; Stdin: ; Stdout: ; Stderr: Cannot open network namespace "qrouter-c8acd926-88e5-4292-883e-e928fb3f6d32": No such file or directory
2016-04-20 12:28:26.700 TRACE neutron.services.metering.drivers.iptables.iptables_driver
2016-04-20 12:28:26.700 TRACE neutron.services.metering.drivers.iptables.iptables_driver

Sergey Belous (sbelous)
Changed in neutron:
assignee: nobody → Sergey Belous (sbelous)
Doug Wiegley (dougwig)
Changed in neutron:
status: New → Confirmed
importance: Undecided → Low
Revision history for this message
Hunt Xu (huntxu) wrote :

Hi Sergey,
I have also encountered this problem and worked out a patch. If you are not currently working on this, could you please assign this bug to me?

Revision history for this message
Sergey Belous (sbelous) wrote :

HuntXu, yep, sure. I can't assign this bug to you, but I can remove me from assignee and then you can assign this bug to yourself.

Changed in neutron:
assignee: Sergey Belous (sbelous) → nobody
Hunt Xu (huntxu)
Changed in neutron:
assignee: nobody → HuntXu (huntxu)
status: Confirmed → In Progress
Revision history for this message
Sergey Belous (sbelous) wrote :

Can you please also check, is it related problem and can it [1] fix the problem with metering agent?

[1] https://review.openstack.org/#/c/309050/

Revision history for this message
Hunt Xu (huntxu) wrote :

Unfortunately iptables_manager does not use ip_lib to run the commands, so [1] could not fix this problem.

[1] https://review.openstack.org/#/c/309050/

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (master)

Change abandoned by Hunt Xu (<email address hidden>) on branch: master
Review: https://review.openstack.org/316622

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/316688

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (master)

Change abandoned by Armando Migliaccio (<email address hidden>) on branch: master
Review: https://review.openstack.org/316688
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

Revision history for this message
Armando Migliaccio (armando-migliaccio) wrote :

Needs a new owner.

Changed in neutron:
status: In Progress → Incomplete
assignee: Hunt Xu (huntxu) → nobody
tags: added: low-hanging-fruit metering
Changed in neutron:
assignee: nobody → Sana Khan (sana.khan)
Revision history for this message
Rodolfo Alonso (rodolfo-alonso-hernandez) wrote :

Bug closed due to lack of activity, please feel free to reopen if needed.

Changed in neutron:
status: Incomplete → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.