[RFE] Add common classifier resource

Bug #1476527 reported by YujiAzama on 2015-07-21
48
This bug affects 4 people
Affects Status Importance Assigned to Milestone
neutron
Wishlist
Igor D.C.

Bug Description

[Existing problem]
Currently, in neutron each service/s which requires a classifier implements the same of their own. This introduce lot of redundancy and maintenance issues.

[Proposal]
In order to address the above problem, the specification [1]_ introduces a common classifier resource for Neutron.

[Benefits]
- The new resource can be leveraged to better realize other Neutron services needing traffic classification. Traffic classification is commonly needed by many Neutron services (e.g. Service Function Chaining [2]_, QoS [3]_, Tap as a Service [4]_, FWaaS [5]_, Group-Based Policy [6]_, and Forwarding Rule [7]_), but each service uses its own classifier resource similar to others. A common traffic classifier resource should exist in Neutron so that it could be leveraged across network services avoiding redundancy and maintenance issues.
- We can also think of deploying a classifier independently for any service/s, which does classification at the ingress/egress. In turn, the services will solely be responsible for their required work.
Example: For Service Function Chaining, flow classifier could be implemented by Neutron while the service chaining can be done by the third party ML2 plug-in.
- Comparison between this proposal with the exiting one's is also captured [9]_.

[What is the enhancement?]
- Add traffic classifiers to Neutron API as extension.
- Classifier can be independently deployed at the ingress/egress without depending upon any of the service/s.
- In future, we can also think of extending the same interface for filtering routes which might be required for other ongoing work like BGP dynamic routing [8]_ happening in Neutron in the Liberty release.

[Related information]
[1] Specification
    https://review.openstack.org/#/c/190463/
[2] API for Service Chaining,
   https://review.openstack.org/#/c/192933/
[3] QoS,
   https://review.openstack.org/#/c/88599/
[4] Tap as a Service,
   https://review.openstack.org/#/c/96149/
[5] Firewall as a Service,
   http://specs.openstack.org/openstack/neutron-specs/specs/api/firewall_as_a_service__fwaas_.html
[6] Group-based Policy,
   https://review.openstack.org/#/c/168444/
[7] Forwarding Rule,
   https://review.openstack.org/#/c/186663/
[8] Neutron route policy support for dynamic routing protocols
   https://blueprints.launchpad.net/neutron/+spec/neutron-route-policy-support-for-dynamic-routing-protocol
[9] Details about common classifier and it's comparison with SG and other's
   https://etherpad.openstack.org/p/flow-classifier

YujiAzama (azama-yuji) on 2015-07-21
description: updated
Changed in neutron:
assignee: nobody → YujiAzama (azama-yuji)
description: updated
description: updated
description: updated
description: updated
description: updated
description: updated
description: updated
Kyle Mestery (mestery) wrote :

Seems like Mitaka to me.

Sean M. Collins (scollins) wrote :

Mitaka - especially since this is supposed to "bring balance to the force" as it were, and consolidate multiple implementations that we have in tree (Security Groups, Firewall as a Service, Service Function Chaining, and Quality-Of-Service)

Akihiro Motoki (amotoki) wrote :

Defining a common classifier sounds good. It will bring users to consistent API definitions across resources.
Although a corresponding proposal was proposed for review, I would suggest to continue the discussion as a neutron-specs review.

I think it is better the discussion covers:
- common classifier resource definitions (accepted values, how fields are interpreted and so on)
- how common classifier resource can be consumed by each implementation
- how to deal with unnecessary fields (required fields varies depending on features. users needs a way to know which fields are available or not)

LP BP here: https://blueprints.launchpad.net/neutron/+spec/classifier
Spec here: https://review.openstack.org/#/c/190463/

I think we can approve this and let the review happen on the spec now.

Changed in neutron:
status: New → Triaged

Agreed.

tags: added: rfe-approved
removed: rfe

However I wonder, do we want to have a classifier as its own project? So that it can be better leveraged across the various components?

On second thought I think this needs more discussion

tags: added: rfe
removed: rfe-approved
vikram.choudhary (vikschw) wrote :

@Yuji: Since, it needs more discussion so I am taking this to me right now. Once the design is finalized, we can implement it together. Hope this would be fine with you.

Changed in neutron:
assignee: YujiAzama (azama-yuji) → vikram.choudhary (vikschw)
Doug Wiegley (dougwig) wrote :

Summit discussion/decision was to put this into its own neutron library repo, named neutron-classifier

vikram.choudhary (vikschw) wrote :

Hi All,

As part of Mitaka, we would like to start this effort as a separate lib called "neutron-classifier".

In the beginning this repo will only support a unified DB and validation model which would be used by FWaaS, SFC, Security-groups, etc. Later we can define separate NBI's for this repo.

Summit discussion:
https://etherpad.openstack.org/p/common_classifier_tokyo_summit_session

Sean M. Collins (scollins) wrote :

I posted some info on the mailing list about this effort

http://lists.openstack.org/pipermail/openstack-dev/2015-November/078289.html

Change abandoned by Armando Migliaccio (<email address hidden>) on branch: master
Review: https://review.openstack.org/190463
Reason: This review was last updated before Mitaka started - September, 23, 2015 - and it still targets Liberty, which has released. Feel free to resubmit to Mikata, but please follow the latest feature submission guidelines as defined in: http://docs.openstack.org/developer/neutron/policies/blueprints.html

My suggestion is to follow the usual convention of using 'networking-' as prefix.

That said, I believe this should be its own initiative whose deliverables will be a set of libraries and services with well defined interfaces that can/will be shared across the Neutron stadium projects that required traffic classification.

Changed in neutron:
assignee: vikram.choudhary (vikschw) → nobody

If ultimately we proceed with a initiative, the faith of this bug report would either be 'Won't fix' or it would need to target the newly created project 'networking-classifier'

Miguel Angel Ajo (mangelajo) wrote :

@armax, that can make sense, if we're able to, from a library, define a data model which would be reusable for all projects, and keep all the common code in it's own repository. My worry here is, how do we handle the data model here?

According to meeting [1], it's been agreed that this should be handled as a standalone initiative that can be reused across the entire stadium. Technical details are in process of being investigated, but eventually this would end up being its own neutron stadium repo/LP project, and *not* the Neutron/Neutron-specs repos.

Mark bug rfe-approved, but eventually this bug will have to target a 'classifier' project.

[1] http://eavesdrop.openstack.org/meetings/neutron_drivers/2015/neutron_drivers.2015-11-10-15.05.log.html

tags: added: rfe-approved
removed: rfe
Changed in neutron:
importance: Undecided → Wishlist

Change abandoned by Doug Wiegley (<email address hidden>) on branch: master
Review: https://review.openstack.org/232825
Reason: This review is > 4 weeks without comment and currently blocked by a core reviewer with a -2. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and contacting the reviewer with the -2 on this review to ensure you address their concerns.

Update:

We have created a repo and have a small core team that has been iterating on a library that can be re-used by other networking projects. It is under development at this point, and we are actively working on it. Right now it is very rough, I am trying to use it for the FwaaS v2 initiative to help shake out all the rough edges - so, it's not quite ready to be picked up and used by other projects yet. But we're working on it.

git.openstack.org/cgit/openstack/neutron-classifier

Changed in neutron:
status: Triaged → In Progress
Henry Gessau (gessau) on 2016-03-24
summary: - RFE - Add common classifier resource
+ [RFE] Add common classifier resource
Changed in neutron:
assignee: nobody → cathy Hong Zhang (cathy-h-zhang)
Igor D.C. (igordcard) wrote :

Is this RFE to be reused or a new one created (based on yesterday's meeting)?

According to the consensus of the Common Flow Classifier discussion meeting (http://eavesdrop.openstack.org/meetings/network_common_flow_classifier/2016/network_common_flow_classifier.2016-05-17-17.02.html), I will submit a new RFE for flow classifiers in neutron-core and we will develop FC as a RFE over neutron-core.

Changed in neutron:
assignee: cathy Hong Zhang (cathy-h-zhang) → nobody
Sean M. Collins (scollins) wrote :

Has it? Really? Wouldn't it be better to continue using this bug since we already had a bunch of work done in this context? Rather than making all new RFE bugs?

Not sure if the original owners are still sparing time working on the bug and associated spec anymore. The spec associated with this bug is in "abandoned" status now.

Based on the new consensus of people interested in this initiative (http://eavesdrop.openstack.org/meetings/network_common_flow_classifier/2016/network_common_flow_classifier.2016-05-17-17.02.html), this feature is to be developed as a RFE over neutron-core. But this bug is positioned as a separate project on a separate repo. The recent meeting consensus is to create a new bug. With a bigger team joining the design discussion on this feature now, it seems a new spec and new codes catching the new design consensus are the way to go.

Igor D.C. (igordcard) wrote :

Since #1583299 is now marked as a duplicate of this bug, please ignore post #23.

I'm moving the proposed model here:

The Common Classifier model should be both explicit and extensible.
This figure shows the model proposed: [1].

Separating the traffic classification in individual types will allow future types to be added and agreed in the future, while keeping the remaining ones intact. Existing types can of course be updated and properly versioned. Looking at the neutron-classifier project, I already see hints of an architecture like this one, as can be seen in [2], so maybe there's already some interest in this approach.

So, how to define this model and how can it be used?
- The Common Classifier allows Classification Rules to be defined and used by an OpenStack service.
- 1 Classification Rule is of a single type, e.g. either Ethernet, IP, or HTTP, and contains a definition that depends on the type.
- The definition contains the fields that the type supports.
- Not all supported fields need to be defined - only the ones required by the consuming service.
- The consuming service should contain any additional logic, such as:
    - AND/ORing many Classification Rules (CRs), if needed;
    - Setting directionality constraints on the CRs, if needed;
    - Scoping the CRs to local resources (e.g. Neutron ports), if needed;
    - Other logic or local constraints.

[1] http://i.imgur.com/4reoOnB.png
[2] https://github.com/openstack/neutron-classifier/blob/10b2eb3127f4809e52e3cf1627c34228bca80101/neutron_classifier/common/constants.py#L17

SFC/classifier team seeks to design and implement the feature more tightly with neutron core, without a separate repo. It's advised that drivers team run that one once more to see if the latest resolutions can be changed.

tags: added: rfe
removed: rfe-approved
Changed in neutron:
status: In Progress → Triaged

After reviewing the spec I think there's substantial lack of information to determine where this initiative is best placed. I can see that Neutron may as well be the place where the classifier end-to-end system lives but we need to understand crisply how this is interacted with from other systems.

I think the spec is getting clearer, but I have yet to review the latest patchset. Based on last week's conversation, I think that this effort should continue to evolve in the neutron-classifier repo, and in the future consider it for inclusion in the Stadium according to the selection criteria as agreed on [1]. If people want to iterate/achieve consensus on [2], I don't mind, but it'll obviously not officially merge' until the neutron-classifier project is included in the Neutron governance. So my suggestion is to do all the legwork necessary to get the neutron-classifier repo in the shape it deserves and get at least one consumer of this project before it can be considered under responsibility of the Neutron team.

[1] http://specs.openstack.org/openstack/neutron-specs/specs/newton/neutron-stadium.html
[2] https://review.openstack.org/#/c/333993/3/specs/newton/common-classifier.rst

Based on last week's discussion, I think there's agreement on making progress on this topic. The spec needs to achieve consensus [1], the code needs to shape up [2], but neither are expected to happen in time for Newton, so postponing for now, but that shouldn't folks from continuing working on it.

[1] https://review.openstack.org/#/c/333993/
[2] https://review.openstack.org/#/c/333993/

tags: added: rfe-postponed
removed: rfe
Igor D.C. (igordcard) wrote :

Assigning myself, finishing the spec at the moment: https://review.openstack.org/#/c/333993
For most of code that will be presented as a PoC, kudos to David Shaughnessy and earlier openstack/neutron-classifier contributors Sean Collins and Ramanjaneya Reddy Palleti.

Changed in neutron:
assignee: nobody → Igor Duarte Cardoso (igordcard)
Igor D.C. (igordcard) on 2017-03-03
tags: added: rfe
removed: rfe-postponed

Discussed during the drivers meeting today. There is agreement we should move forward on this one. We will only need to have agreement on spec write up (at least high level, so that later we don't need to change API), and then set the separate repo so that the subteam can make progress largely without involvement from existing core team.

tags: added: rfe-approved
removed: rfe
Igor D.C. (igordcard) wrote :

Thanks Ihar, I will signal once there is sufficient agreement on the spec, the API and the effort required to evolve it.
Will it be a completely new separate repo or will you reuse neutron-classifier? The current PoC is already based on neutron-classifier.

I posted the following blueprint to track the effort: https://blueprints.launchpad.net/neutron/+spec/common-classification-framework

Reviewed: https://review.openstack.org/333993
Committed: https://git.openstack.org/cgit/openstack/neutron-specs/commit/?id=ac282399d4bf910f900628522d277d998fed0ce9
Submitter: Jenkins
Branch: master

commit ac282399d4bf910f900628522d277d998fed0ce9
Author: Igor Duarte Cardoso <email address hidden>
Date: Fri Jun 24 13:42:19 2016 +0100

    Neutron Common Classification Framework

    This patch documents the Common Classification Framework.

    Neutron features/projects that may need classification are:
    - Security group rules
    - openstack/neutron-fwaas
    - openstack/networking-sfc
    - openstack/networking-bgpvpn
    - openstack/tap-as-a-service
    - Neutron QoS

    Furthermore, there are other projects with classication APIs, such as
    openstack/group-based-policy and it's possible that others will want
    to support classifications in their own APIs, further reinventing the wheel
    and fragmenting the language used in the OpenStack ecosystem when it comes
    to defining traffic classifications.

    Change-Id: I6df22d9e766d8aeabc2f99262cadeec332d809b0
    Related-Bug: #1476527

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers