metering-agent failed to get traffic counters when no router-namespace where meter-label-rules were added

Bug #1572548 reported by Sergey Belous on 2016-04-20
20
This bug affects 4 people
Affects Status Importance Assigned to Milestone
neutron
Low
Sana Khan

Bug Description

Removing router from l3 agent cause errors from neutron-metering-agent. The neutron-meter-agent continues try to get traffic counters from iptables on router-namespace, which is not exist after removing router from l3-agent.

Steps to reproduce:
1. Create internal net, subnet, router. Set external gateway for router, add interface to router for created net.
2. Create neutron-meter-label
3. Use 'neutron l3-agent-router-remove' command to remove a router from a L3 agent
Trace in neutron-metering-agent logs:
2016-04-20 12:28:26.699 ERROR neutron.agent.linux.utils [-] Exit code: 1; Stdin: ; Stdout: ; Stderr: Cannot open network namespace "qrouter-c8acd926-88e5-4292-883e-e928fb3f6d32": No such file or directory

2016-04-20 12:28:26.700 ERROR neutron.services.metering.drivers.iptables.iptables_driver [-] Failed to get traffic counters, router: {u'status': u'ACTIVE', u'name': u'router05', u'gw_port_id': u'f36b30d3-5290-4896-837c-108b8cd4f3dc', u'admin_state_up': True, u'tenant_id': u'1c0eb24bdbb1406bb7d1346f36064ebd', u'_metering_labels': [{u'rules': [{u'remote_ip_prefix': u'0.0.0.0/0', u'direction': u'egress', u'metering_label_id': u'67dae290-38cb-4962-80ab-d6d3404dc6df', u'id': u'799f3361-6f90-4e36-a338-311d6e7c9d5b', u'excluded': False}], u'id': u'67dae290-38cb-4962-80ab-d6d3404dc6df'}], u'id': u'c8acd926-88e5-4292-883e-e928fb3f6d32'}
2016-04-20 12:28:26.700 TRACE neutron.services.metering.drivers.iptables.iptables_driver Traceback (most recent call last):
2016-04-20 12:28:26.700 TRACE neutron.services.metering.drivers.iptables.iptables_driver File "/opt/stack/neutron/neutron/services/metering/drivers/iptables/iptables_driver.py", line 355, in get_traffic_counters
2016-04-20 12:28:26.700 TRACE neutron.services.metering.drivers.iptables.iptables_driver chain, wrap=False, zero=True)
2016-04-20 12:28:26.700 TRACE neutron.services.metering.drivers.iptables.iptables_driver File "/opt/stack/neutron/neutron/agent/linux/iptables_manager.py", line 712, in get_traffic_counters
2016-04-20 12:28:26.700 TRACE neutron.services.metering.drivers.iptables.iptables_driver current_table = self.execute(args, run_as_root=True)
2016-04-20 12:28:26.700 TRACE neutron.services.metering.drivers.iptables.iptables_driver File "/opt/stack/neutron/neutron/agent/linux/utils.py", line 137, in execute
2016-04-20 12:28:26.700 TRACE neutron.services.metering.drivers.iptables.iptables_driver raise RuntimeError(msg)
2016-04-20 12:28:26.700 TRACE neutron.services.metering.drivers.iptables.iptables_driver RuntimeError: Exit code: 1; Stdin: ; Stdout: ; Stderr: Cannot open network namespace "qrouter-c8acd926-88e5-4292-883e-e928fb3f6d32": No such file or directory
2016-04-20 12:28:26.700 TRACE neutron.services.metering.drivers.iptables.iptables_driver
2016-04-20 12:28:26.700 TRACE neutron.services.metering.drivers.iptables.iptables_driver

Sergey Belous (sbelous) on 2016-04-20
Changed in neutron:
assignee: nobody → Sergey Belous (sbelous)
Doug Wiegley (dougwig) on 2016-04-20
Changed in neutron:
status: New → Confirmed
importance: Undecided → Low
Hunt Xu (huntxu) wrote :

Hi Sergey,
I have also encountered this problem and worked out a patch. If you are not currently working on this, could you please assign this bug to me?

Sergey Belous (sbelous) wrote :

HuntXu, yep, sure. I can't assign this bug to you, but I can remove me from assignee and then you can assign this bug to yourself.

Changed in neutron:
assignee: Sergey Belous (sbelous) → nobody
Hunt Xu (huntxu) on 2016-05-10
Changed in neutron:
assignee: nobody → HuntXu (huntxu)
status: Confirmed → In Progress
Sergey Belous (sbelous) wrote :

Can you please also check, is it related problem and can it [1] fix the problem with metering agent?

[1] https://review.openstack.org/#/c/309050/

Hunt Xu (huntxu) wrote :

Unfortunately iptables_manager does not use ip_lib to run the commands, so [1] could not fix this problem.

[1] https://review.openstack.org/#/c/309050/

Change abandoned by Hunt Xu (<email address hidden>) on branch: master
Review: https://review.openstack.org/316622

Change abandoned by Armando Migliaccio (<email address hidden>) on branch: master
Review: https://review.openstack.org/316688
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

Needs a new owner.

Changed in neutron:
status: In Progress → Incomplete
assignee: Hunt Xu (huntxu) → nobody
tags: added: low-hanging-fruit metering
Sana Khan (sana.khan) on 2016-07-08
Changed in neutron:
assignee: nobody → Sana Khan (sana.khan)
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers