Reviewed:  https://review.openstack.org/204950
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=d3708de0cb1dacc610bc9585a55dd260a862104d
Submitter: Jenkins
Branch:    feature/qos

commit ea02f25fbef43580c619eb778aab3dc874427eef
Author: Moshe Levi <email address hidden>
Date:   Wed Jul 15 11:13:50 2015 +0300

    Update OVS Agent to work with Agent Extension Mgr
    
    Change-Id: I1887d402a4377babb648182727cf51b9f2627e1c

commit c22cce92f8cc93392e07124d00771c3c5a962cba
Author: Jakub Libosvar <email address hidden>
Date:   Thu Jul 16 14:23:26 2015 +0200

    Instantiate qos agent driver
    
    Change-Id: Icd8a725f231e2749bb81da0bcad0f7ef95beb676

commit ddd9ee9a4c1dc36b405cb4cb854bb1b46ed32e98
Author: Ihar Hrachyshka <email address hidden>
Date:   Thu Jul 16 15:15:11 2015 +0200

    objects.rule: enable database tests for QosRule
    
    While at it, simplified inheritance mess in test_policy.
    
    Change-Id: I6cf2394a27f8bb29a18c99effe2dc2251c138d59

commit 0395f142031bb200373b439c7798629eee1321eb
Author: Mike Kolesnik <email address hidden>
Date:   Wed Jul 15 10:44:15 2015 +0300

    Handle qos_policy on network/port create/update
    
    Added handling for qos_policy_id field in the network and port
    entities via ML2 extension driver.
    The QoS profile will be associated to the network/port when requested as
    part of the entity creation or update.
    
    Allow ML2 extension manager to not register for any api extension
    (new use case).
    
    ===
    
    Extend the resources using the QoS extension class
    
    Since the QoS extension for plugins is handles by this class, it makes
    sense for it to handle also property extension of resources.
    
    For ML2 this means that that extend_{network,port}_dict functions will
    handle the extension of resources by calling QosExtensionHandler.
    This logic can easily be reused by other plugins.
    
    Note: we should make sure that resource extension does not require db
    access, otherwise we see DBDeadLock errors and random tempest failures.
    To achieve this, we define a new SQLAlchemy joined relationship on
    policy bindings to make networks and ports receive those bindings on
    their fetch from database. After that, the only work to do left for
    resource extension handler is to copy the fetched policy into resource
    dictionary.
    
    ===
    
    Also enable new qos ml2 extension until we configure it in gate via
    project-config and devstack-gate to make sure it's enabled and tested.
    
    Co-Authored-By: Ihar Hrachyshka <email address hidden>
    Partially-implements: blueprint quantum-qos-api
    Change-Id: I1b7d4611215a471d5c24eb3d7208dcddb7e015f4

commit 0a33e355bcfb0a44ad3617f659ab452a04abdbdb
Author: Ihar Hrachyshka <email address hidden>
Date:   Fri Jul 10 18:00:34 2015 +0200

    objects.qos.policy: support per type rule lists as synthetic fields
    
    This is a significant piece of work.
    
    It enables neutron objects to define fields that are lazily loaded on
    field access. To achieve that,
    
    - field should be mentioned in cls.synthetic_fields
    - obj_load_attr should be extended to lazily fetch and cache the field
    
    Based on this work, we define per type rule fields that are lists of
    appropriate neutron objects. (At the moment, we have only single type
    supported, but I tried hard to make it easily extendable, with little or
    no coding needed when a new rule type object definition is added to
    rule.py: for example, we inspect object definitions based on
    VALID_RULE_TYPES, and define appropriate fields for the policy object).
    
    To implement lazy loading for those fields, I redefined get_by_id for
    rules that now meld fields from both base and subtype db models into the
    corresponding neutron object.
    
    Added a simple test that checks bandwidth_rules attribute behaves for
    policies.
    
    Some objects unit test framework rework was needed to accomodate
    synthetic fields that are not propagated to db layer.
    
    Change-Id: Ia16393453b1ed48651fbd778bbe0ac6427560117

commit 6140da5f25da126cdbfae3e9ee397315e7834fc5
Author: Miguel Angel Ajo <email address hidden>
Date:   Tue Jul 14 14:31:55 2015 +0200

    QoS Service devref
    
    This is initial version that should be ready for merge. After we get it
    in the tree, we'll be able to expand on demand or even as part of
    patches that change the implementation in significant way.
    
    Co-Authored-By: Ihar Hrachyshka <email address hidden>
    Change-Id: I3d89bcbb9c64b87d7425155c33b2f753176e94fd

commit 4af0de954aabb51560d4c28f8ea246a53d214b20
Author: Mike Kolesnik <email address hidden>
Date:   Tue Jun 30 12:07:48 2015 +0300

    Implement QoS plugin
    
    Initial implementation of the QoS service plugin that just implements
    CRUD for policy and rule.
    
    There are no tests yet.
    
    path_prefix is now provided as an attribute to the plugin base, since
    that's required by the COMMON_PREFIXES removal from master branch.
    
    Partially-implements: blueprint quantum-qos-api
    Change-Id: Icf821dec17f435d8e47e1047fb05225e7dd071f0

commit bcb4d237a79be9af18f1bcc792e1827c18b058d2
Author: Kevin Benton <email address hidden>
Date:   Thu Jul 16 02:07:48 2015 -0700

    Add oslo db retry decorator to non-CRUD actions
    
    The previously added decorators to the create and update handlers
    in the API layer only applied to actions that followed the standard
    create/update path. However, for API operations like add_router_interface,
    a different path is followed that wasn't covered by a retry decorator.
    
    This patch adds the decorator to handle deadlocks in those operations as
    well.
    
    Closes-Bug: #1475218
    Change-Id: Ib354074e6a3f68cedb95fd774f905d94ca16a830
    (cherry picked from commit 435ffa7c67cf8668063588e2af760c1ff595dfbb)

commit a1c05891a52fba19dec1122e25144446ee8717ae
Author: Jakub Libosvar <email address hidden>
Date:   Tue Jul 14 12:42:57 2015 +0000

    Add qos section to ovs agent config
    
    [qos] section is introduced with qos driver for ovs agent. Similar
    manner should follow for all l2 agents using qos with different default
    drivers.
    
    Change-Id: I3c6a3711d3cd9924d55cf6d0ed84be18c993c275

commit e19eb49c1c066c8fa4a3c19183bca1daef553a5c
Author: Miguel Angel Ajo <email address hidden>
Date:   Wed Jul 15 15:42:52 2015 +0200

    Mute neutron.callbacks notification logs.
    
    We believe they could be the source of some yielding,
    and consequient DBDeadlocks we're experiencing when using
    AFTER_READ to extend resources.
    
    This will need to be revised.
    
    Change-Id: I040b3a3c9e137267dfe237dd90fb525026b0d16e

commit 1ec79bad468f65d0adfb16f652170f059c92a3bc
Author: Jakub Libosvar <email address hidden>
Date:   Wed Jul 15 15:47:40 2015 +0200

    Small fixes in test_qos_agent UT
    
    Change-Id: Ib09824552edd287ec1df3bdd6700c9ce8a02df29

commit 3cbe446cb6db0ee18c928bb3ec1ff688db5febd4
Author: Gal Sagie <email address hidden>
Date:   Tue Jul 14 17:37:08 2015 +0300

    Add unit tests and fixes for OVS Agent QoS Extension Driver
    
    Add basic unit tests and fix some of the issues while doing
    the tests
    
    blueprint ml2-ovs-qos-with-bwlimiting
    
    Change-Id: I3962dd2c0e1273905781faf3f5c51886dea21cd4

commit cdcffc709baa3290934e6fc3e7f87862dbbbe0e1
Author: Gal Sagie <email address hidden>
Date:   Wed Jul 1 15:36:51 2015 +0300

    Add OVS QoS extension agent driver
    
    This is a prototype for the OVS agent driver for QoS
    extension.
    Should be adjusted to final API's once they are added
    
    blueprint ml2-ovs-qos-with-bwlimiting
    
    Depends-On: I9e388173dfe0eb43c961018bd687bc86f34c7a6a
    Change-Id: Ie952b63e3760d1924a34676e97319ec4301effca

commit 9544ffb615f69f25268f0f2fd3d92ca4acc7d8cb
Author: Moshe Levi <email address hidden>
Date:   Wed Jun 24 18:44:08 2015 +0300

    Qos Agent Extension
    
    This patch introduces the following:
    QosAgentExtension - implementation of AgentCoreResourceExtension
    QosAgentDriver - interface class
    This will allow any agent to implement their own low level driver for
    Qos Agent Extension.
    
    Co-Authored-By: Miguel Angel Ajo <email address hidden>
    
    Change-Id: I9e388173dfe0eb43c961018bd687bc86f34c7a6a

commit dd6cd44b2155605345a6925d3373bb6afe0bcf62
Author: Miguel Angel Ajo <email address hidden>
Date:   Thu Jul 9 16:39:05 2015 +0200

    Implement QoS policy detach from port and network
    
    Includes db.qos.api calls to delete port<->qos_profile and
    network<->qos_profile bindings.
    
    Change-Id: I8ab3e885bdf010fe95529157f3db4f1089326c86

commit 6f3b70b5061af332c7accd348ec31b453c272e98
Author: Irena Berezovsky <email address hidden>
Date:   Tue Jul 7 11:54:39 2015 +0000

    Add API stub for QoS support rule_type resource
    
    Rule_type is added as an API Resource.
    Using this API User can get list of  supported
    QoS rule types.
    
    Change-Id: Ica80753ce3052b68ba8db2d2760ed4310ec0b976

commit 954e9de8b3b697b39e087f2d03b49f0856c44c32
Author: Ihar Hrachyshka <email address hidden>
Date:   Thu Jul 9 14:12:56 2015 +0200

    objects.base: fixed object.delete()
    
    It was using wrong call to delete an object from a session.
    
    Also, expanded test_create() test to check all basic operations:
    get_by_id, create, update, and delete.
    
    To facilitate update() check, introduced a new field for objects called
    fields_to_update that stores names of fields that are not expected to be
    updated in an object. This allows us to keep the above mentioned test
    untangled from specific object type.
    
    Also made get_by_id() behave correctly (returning None) if the object
    does not exist.
    
    Change-Id: I1aecb2e7c4d8cb8f239072d1cb9df3db29dcedde

commit 31e09028e3b3c85954a9aebf32c412bd897afdbb
Author: Ihar Hrachyshka <email address hidden>
Date:   Thu Jul 9 12:37:01 2015 +0200

    objects.qos.policy: fixed get_*_policy and attach_* methods
    
    Added sql unit tests for those methods.
    
    Change-Id: I6e95aa6cb61d5cc36600394b2198587793da8a0e

commit 3edec57c2250daafdcdac88581efa1acc5acf237
Author: Ihar Hrachyshka <email address hidden>
Date:   Wed Jul 8 18:06:12 2015 +0200

    objects.base: reset changes after getting objects from database
    
    Now all objects are comparable.
    
    We need to reset changes, otherwise an object that is constructed and
    .create()d is different from the one that is .get_by_id()d from database
    (for primitive serialization contains list of changed fields for
    versioned objects).
    
    Added initial sql test case for objects (just create-fetch for policy
    for now, but can be easily extended to other types).
    
    Change-Id: I012b5fe4e95f166f66da91274734d7184c224dfd

commit 78703ddefbbf93dda8ecb173f14086880bb9f82f
Author: Ihar Hrachyshka <email address hidden>
Date:   Wed Jul 8 17:47:12 2015 +0200

    BaseObjectTestCase: rename test_class into _test_class
    
    It seems that testtools are trying to load that class as if it's a test
    case (for it considers everything inside a test class named as test_* as
    a test case).
    
    Change-Id: Ic4342cc0637d659191f084467ccdb9c90e89a023

commit 00589382cf1ea7034d5dee5aec8bf3814f7e92a5
Author: John Schwarz <email address hidden>
Date:   Thu Jul 2 12:40:11 2015 +0300

    Cleanup rule models and objects
    
    - drop tenant_id for base and bandwidth_limit rules;
    - added 'to_dict' function to convert objects into dicts.
    
    Change-Id: I28167e356e70235304b166c997df52ca1b28f836

commit a71cb067d0100a1ec3f6df037c6c4740a6b2771a
Author: Ihar Hrachyshka <email address hidden>
Date:   Fri Jul 3 01:57:03 2015 +0300

    objects.qos: fixed create and update for QosBandwidthLimitRule
    
    To simplify Qos*Rule object type implementation, renamed 'qos_rule_id'
    field for qos_bandwidth_limit_rule table into 'id'.
    
    Also added unit test coverage for the object type.
    
    Change-Id: Id6bc992af9f1ab46c022d3c88aa66a0f3bb7f227

commit cbc7826f2c0b77f1283ed62a3adb332c639cbdd5
Author: Ihar Hrachyshka <email address hidden>
Date:   Fri Jul 3 00:52:20 2015 +0300

    objects.qos: added unit tests for QosPolicy neutron object
    
    Change-Id: Icecb3fc08c81bd9fb9f8bad54ed800a4eb55d399

commit 878e85527fa6833a85d7d9ad15e63e26aeb00ccd
Author: Ihar Hrachyshka <email address hidden>
Date:   Thu Jul 2 23:30:36 2015 +0300

    objects.base: avoid db access if object does not have changes
    
    Also cover base object with unit tests.
    
    Change-Id: I2f58c767ba35eeee2f9ecc363e5b042ea8638faa

commit 4ee9eebd7dd5ed7c87d481eda500b664ae564644
Author: John Schwarz <email address hidden>
Date:   Thu Jul 2 12:32:05 2015 +0300

    QoS extension fixes
    
    This patch introduces small fixes to the QoS extensions:
    1. Adding a common tenant_id field for QoS API calls.
    2. Making sure the function interface is correct.
    
    Co-Authored-By: Irena Berezovsky <email address hidden>
    Change-Id: If9c7a7b9b8a5d2367d8f3225fbf07d8e3ec8865d

commit c26142be338ef14f5fd0a4c7aa5b4ac42bc685d1
Author: Moshe Levi <email address hidden>
Date:   Wed Jun 24 18:10:05 2015 +0300

    AgentExtensionsManager and AgentCoreResourceExtension
    
    This patch introduces the following classes:
    L2Agent - abstract class for common L2Agent implementions.
    AgentExtensionsManager - to load AgentCoreResourceExtension.
    AgentCoreResourceExtension - interface class to define
    the AgentCoreResourceExtension API.
    This allows better segregation between L2 Agent Core
    and L2 Agent Extensions.
    
    The patch is missing unit test but it was tested manually.
    I added a unit tests @TODO comments to come back
    to them later.
    
    Change-Id: I813de7ff1bee188f4294f4b3eb3645ebd903297b

commit 2d38c742e84aebccd367eb7312ddaaeb0d02bc01
Author: Moshe Levi <email address hidden>
Date:   Tue Jun 30 22:23:26 2015 +0300

    Generic Resources RPC
    
    This patch adds Generic Resource RPC from agent to server.
    
    Change-Id: I0ac8a009e781b6edb283d8634b1a2f047db092dc

commit dc802438887265c1faa0a92df7c7c701854cf5fc
Author: Miguel Angel Ajo <email address hidden>
Date:   Thu Jun 11 15:21:28 2015 +0200

    Generic rpc callback mechanism which could be reused
    
    This is a publisher/subscriber messaging mechanism optimized
    for agent consumption and server production without the need
    of creating new rpc messages when new resources are introduced.
    
    Oslo versionedobjects are the perfect match to ensure
    cross version compatibility even if the published/subscribed
    resources format change over time.
    
    This is still a basic stub allowing get_info of the resources,
    and the next change will introduce the RPC methods to call
    get_info: I0ac8a009e781b6edb283d8634b1a2f047db092dc
    
    The plugin is returning stub objects to be consumed from the
    agent to test the basic behaviour until we have DB.
    
    TODO: Update documentation, according to code changes,
          enforce versioned objects only doing deserial/serialization.
    
    Co-Authored-By: Miguel Angel Ajo <email address hidden>
    Co-Authored-By: Eran Gampel <email address hidden>
    
    Change-Id: I524cf5a14e99dc6bee4d4261557d98c75efa0809

commit e3dba1424114575581c153e02227282e036ad0a2
Author: Miguel Angel Ajo <email address hidden>
Date:   Fri Jun 19 16:43:52 2015 +0200

    Introduce the AFTER_READ callback for ports and networks
    
    This callback can be used by extensions and service plugins to extend
    port and network information on read time, without the need of plugin
    mixins.
    
    Partially-implements: blueprint quantum-qos-api
    Change-Id: Ifc92c19a69d28784c030d605c2eb161c2ba4b3f5

commit 3de65f57e30b73f5d7efc0344a102f1e40a6b40e
Author: Mike Kolesnik <email address hidden>
Date:   Tue Jun 30 15:21:30 2015 +0300

    Add extension callbacks support for networks
    
    Add callbacks for extention to the network resource so that interested
    extensions can do custom logic when a network is created or updated.
    
    Currently it will be done the same way port notifications are done -
    i.e. only in ML2 plugin. We can revisit this in a floow up patch if the
    whole notification logic should be moved somewhere else as this merits
    further discussion which is out of scope for a mere patch.
    
    This will be utilized in a follow up commit.
    
    Partially-implements: blueprint quantum-qos-api
    Change-Id: I38528863e1145caf05fe3b2425511d1c5b5c0f93

commit b167ec6d4f5577c54a1a02e6a835b875a6e2150a
Author: Ihar Hrachyshka <email address hidden>
Date:   Thu Jul 2 11:14:17 2015 +0300

    [qos] policy: add methods to interact with policy bindings
    
    Detachment is not supported in this patch.
    
    blueprint quantum-qos-api
    
    Change-Id: I66f87b99241a25d39d08c124bae3779c872bc567

commit e90b28662eb769e26f2f6c202a71910819c3ff41
Author: Ihar Hrachyshka <email address hidden>
Date:   Tue Jun 30 13:32:27 2015 +0300

    First QoS versioned objects, ever
    
    Well, first versioned objects in the tree.
    
    Binding to networks and ports is not implemented.
    
    No tests. Checked manually.
    
    blueprint quantum-qos-api
    
    Co-Authored-By: vikram.choudhary <email address hidden>
    Change-Id: I9b6cacfda4f40230d746222bed5b6c490be63743

commit 3d6666af4921a28ccb95d25d88087e89e23e3ec5
Author: Gal Sagie <email address hidden>
Date:   Wed Jul 1 19:15:55 2015 +0300

    Add bandwidth_limit rule type constant
    
    Change-Id: I7228b3a288848833947271a0966ca415bfaa07c4

commit 3e2ef95dfdfe19a0097f8d68f453cbabbb8f2169
Author: Irena Berezovsky <email address hidden>
Date:   Tue Jun 30 12:04:39 2015 +0000

    Add bandwidth_limit rules as sub-collection of qos policy
    
    This patch add support for qos extension according to the
    spec modification [1].
    
    [1] https://review.openstack.org/#/c/197004/
    
    Change-Id: I9226932191464face6e20625e35ad4b7529db4ca

commit 4310b4c2a68914aa78275a78f882c70eb57c1fde
Author: Ramanjaneya <email address hidden>
Date:   Wed Jun 24 17:24:11 2015 +0530

    QoS: db models and migration rules
    
    This patch includes db models and migration rules for initial QoS objects.
    
    Comparing to the spec, it adds two more service tables to maintain links
    between networks and ports and their respective policies. We maintain
    uniqueness as a unique constraint.
    
    In some parallel world, we could have an additional field for networks
    and ports that could be nullable to point to a policy. That said, it
    breaks qos isolation a bit, and will also be a bit more painful if and
    when we decide to spin out qos service pieces outside the tree.
    
    blueprint quantum-qos-api
    
    Co-Authored-By: Ramanjaneya <email address hidden>
    Co-Authored-By: vikram.choudhary <email address hidden>
    Co-Authored-By: Ihar Hrachyshka <email address hidden>
    Co-Authored-By: Miguel Angel Ajo <email address hidden>
    
    Change-Id: I55a7dac602e2e770c21b6c7957430cb7115e5bdc

commit be1d242fa3efcef9e7095fbbf0e2de022fa86a45
Author: Gal Sagie <email address hidden>
Date:   Sat Jun 27 13:16:11 2015 +0300

    Add Create/Destroy API to OVS QoS BW Limiting
    
    Add infrastructure needed for the implementations
    (CLI and native) and add API to ovs_lib
    Add functional tests for ovs_lib
    
    blueprint ml2-ovs-qos-with-bwlimiting
    
    Change-Id: Ided0740548987ca91f1549f251c7906e6449f91d

commit b17f865f85fddafaab367f55470c2b7ae8135e62
Author: Ihar Hrachyshka <email address hidden>
Date:   Wed Jul 1 10:42:09 2015 +0300

    docs: link quality of service doc stub to devref index
    
    Since I433126a8247e7e1c316f2c96bb21e15582b247ce, doc warnings are
    considered as failures in gate.
    
    Unlinked .rst file generates one, making docs job broken for
    feature/qos. Same for an empty file with no title.
    
    Change-Id: Iba82d9728e99238bcc55b12f0ab9eb936fd62147

commit 2ff19be1db96f97b833052af633abda55f497f1d
Author: Miguel Angel Ajo <email address hidden>
Date:   Fri Jun 19 16:45:13 2015 +0200

    QoS service plugin stub
    
    This patch introduces the QoS service plugin which implements
    a stub of the API extension.
    
    This is patch is a basic step to be able to create an experimental
    job enabling this service so we can do api tests.
    
    Change-Id: Ib583e98c232ca628ba2a4bd48527eb84584c6212

commit 96d1cb1ae2f0188988102a56c2886870af94d88e
Author: Miguel Angel Ajo <email address hidden>
Date:   Fri Jun 19 16:28:26 2015 +0200

    Create the QoS API extension stub
    
    This patch introduces the QoS API extension, in a basic
    form where we could, in combination with the service plugin
    stub, start creating some experimental test jobs that install
    the service plugin.
    
    Please not that URL mapping is not fully according to spec,
    neither it does include any testing. We need to work that out.
    
    blueprint quantum-qos-api
    Change-Id: I86e8048e2d9b84690dbede9a94cfc884985069c5

commit 4d5ae8852a79eb4ba041122e65052abf8c196efb
Author: Miguel Angel Ajo <email address hidden>
Date:   Tue Jun 9 11:55:58 2015 +0200

    Change defaultbranch in .gitreview
    
    This is a branch-only change to make sure anybody
    working on the branch will push to the right one.
    
    Change-Id: I2d35d0659bd3f06c570ba99e8b8a41b620253e75