[RFE] L3 IPs monitor/metering via current QoS functionality (tc filters)

[LIU Yulong]

For now, L3 IPs are all have bandwidth QoS functionality. Floating IPs and gateway IPs have the same TC rules. And for one specific IP, it can not be set in two hosts for current neutron architecture. That is saying, where the IP is working, we can get the TC statistic data for it. Yes, the TC filter rules have that data for us:
https://review.openstack.org/#/c/453458/10/neutron/agent/linux/l3_tc_lib.py@143

Command line example:
# ip netns exec snat-867e1473-4495-4513-8759-dee4cb1b9cef tc -s -d -p filter show dev qg-91293cf7-64
filter parent 1: protocol ip pref 1 u32
filter parent 1: protocol ip pref 1 u32 fh 800: ht divisor 1
filter parent 1: protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 not_in_hw (rule hit 180 success 180)
  match IP src 172.16.100.10/32 (success 180 )
 police 0x2 rate 1024Kbit burst 128Kb mtu 64Kb action drop overhead 0b linklayer ethernet
 ref 1 bind 1 installed 86737 sec used 439 sec

 Sent 17640 bytes 180 pkts (dropped 0, overlimits 0)

So, we can use this data to enable the L3 IPs metering directly by l3 agent itself. Because we have that TC filters for all the statistic data we need. neutron metering agent seems now not so much widely used, and it is a little heavy for cloud users.

About how to deal with the data:
1. retrieve the data from the TC rules periodically
2. store the data to local store file
3. report the data to ceilometer/metering service via RPC notification or UDP
4. some other service like zabbix read the local store data

[Miguel Lavalle (minsel)]

What you are saying is:

1) Use tc counters
2) Implement in the L3 agent instead of the metering agent

Right?

Would we preserve the same service plugin and ReST API?

[LIU Yulong (dragon889)]

@Miguel, sorry for the late reply.
Yes, it is tc counters.
We do not need new service plugin or REST api, current floating IP qos implementation is enough.
As I had written in the bug description, the metering agent is a bit heavy for the cloud deployment. Users need to add extra actions to enable the metering functionality. For this proposal here, I would like to add a L3 agent extension or refactor to achieve the L3 IP metering. Some new config settings will be needed for this, such as, how long to retrieve the data, how to store the data, how to report the data, etc.

[Miguel Lavalle (minsel)]

OK, let's see what the drivers team says

[YAMAMOTO Takashi (yamamoto)]

@LIU

so, your proposal is:

- make it l3-agent config. if it's enabled, all qos-enabled fip handled by the agent is metered.

- no rest api to control the metering functionality

right?

[LIU Yulong (dragon889)]

Hi YAMAMOTO Takashi,
Yes, the config will be needed to enable the l3 agent 'self service' metering.
For the REST API, I can image some new idea, for instance:
1. disable one IP's metering.
2. clean one IP's metering data.
3. change one IP's metering frequency (interval).
4. change one IP's report interval.
But yes, this will become more complicated. It is clear to find without these API changes, the IP's metering will be triggered by the fixed settings in l3_agent.ini. But we can let it be the first step of "self service" metering.

And, what's more, maybe we can expand such metering function to L2 agent too. But this will be another story.
Let's handle the L3 IPs metering first.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/658511

[LIU Yulong (dragon889)]

@Miguel Lavalle
We discussed this once in drivers meeting.
http://eavesdrop.openstack.org/meetings/neutron_drivers/2019/neutron_drivers.2019-03-29-14.00.log.html
So basicly we have a basic consensus is adding a new l3 agent extension. It is fine to me.
We need to retrigger another discuss in drivers meeting?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/675654

[Slawek Kaplonski (slaweq)]

Hi Liu,

I was thinking about this proposal once again. And I'm still not convinced if we should do it in L3 agent.
As we all know, we have metering agent and service plugin already - this service plugin provides API - https://docs.openstack.org/api-ref/network/v2/?#metering-labels-and-rules-metering-labels-metering-label-rules - now You are proposing to add new API and new implementation for same thing basically. I'm affraid that from user perspective it may not be the best solution to provide 2 different APIs to the same thing basically.
Maybe You could simply propose another driver (based on tc counters) for metering agent: https://github.com/openstack/neutron/tree/master/neutron/services/metering/drivers ? That way change would be smaller IMO and there would be no new API for the same thing.

[Slawek Kaplonski (slaweq)]

Hi Liu,

Did You saw my last comment here? What do You think about it? Do You still plan to work on this feature?

[Slawek Kaplonski (slaweq)]

Hi Liu,

Are You still interested in impementing this RFE? If so, can You reply to my last comments?

[Slawek Kaplonski (slaweq)]

On the last drivers meeting we decided to approve this RFE.
However we would like to treat it as "next generation metering" in Neutron with potentially deprecation of old metering agent and we want to discuss details about that during the Virtual PTG.

[Slawek Kaplonski (slaweq)]

This bug has had a related patch abandoned and has been automatically un-assigned due to inactivity. Please re-assign yourself if you are continuing work or adjust the state as appropriate if it is no longer valid.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Slawek Kaplonski <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/675654
Reason: This review is > 4 weeks without comment, and failed Zuul jobs the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Slawek Kaplonski <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/766875
Reason: This review is > 4 weeks without comment, and failed Zuul jobs the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[Slawek Kaplonski (slaweq)]

This bug has had a related patch abandoned and has been automatically un-assigned due to inactivity. Please re-assign yourself if you are continuing work or adjust the state as appropriate if it is no longer valid.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Slawek Kaplonski <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/675654
Reason: This review is > 4 weeks without comment, and failed Zuul jobs the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "liuyulong <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron-specs/+/658511
Reason: Restore if someday we want this.