[RFE] L3 IPs monitor/metering via current QoS functionality (tc filters)

Bug #1817881 reported by LIU Yulong on 2019-02-27
This bug affects 1 person
Affects Status Importance Assigned to Milestone
LIU Yulong

Bug Description

For now, L3 IPs are all have bandwidth QoS functionality. Floating IPs and gateway IPs have the same TC rules. And for one specific IP, it can not be set in two hosts for current neutron architecture. That is saying, where the IP is working, we can get the TC statistic data for it. Yes, the TC filter rules have that data for us:

Command line example:
# ip netns exec snat-867e1473-4495-4513-8759-dee4cb1b9cef tc -s -d -p filter show dev qg-91293cf7-64
filter parent 1: protocol ip pref 1 u32
filter parent 1: protocol ip pref 1 u32 fh 800: ht divisor 1
filter parent 1: protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 not_in_hw (rule hit 180 success 180)
  match IP src (success 180 )
 police 0x2 rate 1024Kbit burst 128Kb mtu 64Kb action drop overhead 0b linklayer ethernet
 ref 1 bind 1 installed 86737 sec used 439 sec

 Sent 17640 bytes 180 pkts (dropped 0, overlimits 0)

So, we can use this data to enable the L3 IPs metering directly by l3 agent itself. Because we have that TC filters for all the statistic data we need. neutron metering agent seems now not so much widely used, and it is a little heavy for cloud users.

About how to deal with the data:
1. retrieve the data from the TC rules periodically
2. store the data to local store file
3. report the data to ceilometer/metering service via RPC notification or UDP
4. some other service like zabbix read the local store data

Miguel Lavalle (minsel) on 2019-03-01
tags: added: rfe
Miguel Lavalle (minsel) on 2019-03-01
Changed in neutron:
importance: Undecided → Wishlist
Miguel Lavalle (minsel) wrote :

What you are saying is:

1) Use tc counters
2) Implement in the L3 agent instead of the metering agent


Would we preserve the same service plugin and ReST API?

LIU Yulong (dragon889) wrote :

@Miguel, sorry for the late reply.
Yes, it is tc counters.
We do not need new service plugin or REST api, current floating IP qos implementation is enough.
As I had written in the bug description, the metering agent is a bit heavy for the cloud deployment. Users need to add extra actions to enable the metering functionality. For this proposal here, I would like to add a L3 agent extension or refactor to achieve the L3 IP metering. Some new config settings will be needed for this, such as, how long to retrieve the data, how to store the data, how to report the data, etc.

Miguel Lavalle (minsel) wrote :

OK, let's see what the drivers team says

tags: added: rfe-triaged
removed: rfe
YAMAMOTO Takashi (yamamoto) wrote :


so, your proposal is:

- make it l3-agent config. if it's enabled, all qos-enabled fip handled by the agent is metered.

- no rest api to control the metering functionality


LIU Yulong (dragon889) wrote :

Hi YAMAMOTO Takashi,
Yes, the config will be needed to enable the l3 agent 'self service' metering.
For the REST API, I can image some new idea, for instance:
1. disable one IP's metering.
2. clean one IP's metering data.
3. change one IP's metering frequency (interval).
4. change one IP's report interval.
But yes, this will become more complicated. It is clear to find without these API changes, the IP's metering will be triggered by the fixed settings in l3_agent.ini. But we can let it be the first step of "self service" metering.

And, what's more, maybe we can expand such metering function to L2 agent too. But this will be another story.
Let's handle the L3 IPs metering first.

Miguel Lavalle (minsel) on 2019-03-21
Changed in neutron:
status: New → Confirmed

Fix proposed to branch: master
Review: https://review.opendev.org/658511

Changed in neutron:
assignee: nobody → LIU Yulong (dragon889)
status: Confirmed → In Progress
LIU Yulong (dragon889) wrote :

@Miguel Lavalle
We discussed this once in drivers meeting.
So basicly we have a basic consensus is adding a new l3 agent extension. It is fine to me.
We need to retrigger another discuss in drivers meeting?

Slawek Kaplonski (slaweq) wrote :

Hi Liu,

I was thinking about this proposal once again. And I'm still not convinced if we should do it in L3 agent.
As we all know, we have metering agent and service plugin already - this service plugin provides API - https://docs.openstack.org/api-ref/network/v2/?#metering-labels-and-rules-metering-labels-metering-label-rules - now You are proposing to add new API and new implementation for same thing basically. I'm affraid that from user perspective it may not be the best solution to provide 2 different APIs to the same thing basically.
Maybe You could simply propose another driver (based on tc counters) for metering agent: https://github.com/openstack/neutron/tree/master/neutron/services/metering/drivers ? That way change would be smaller IMO and there would be no new API for the same thing.

Slawek Kaplonski (slaweq) wrote :

Hi Liu,

Did You saw my last comment here? What do You think about it? Do You still plan to work on this feature?

Slawek Kaplonski (slaweq) wrote :

Hi Liu,

Are You still interested in impementing this RFE? If so, can You reply to my last comments?

Slawek Kaplonski (slaweq) wrote :

On the last drivers meeting we decided to approve this RFE.
However we would like to treat it as "next generation metering" in Neutron with potentially deprecation of old metering agent and we want to discuss details about that during the Virtual PTG.

tags: added: rfe-approved
removed: rfe-triaged
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers