DM didnt push required config to MX on a setup

[Vedamurthy Joshi]

R2.20 Build 99 Ubuntu 14.04 Juno multinode setup

While running DM based tests for BMS inter-vn scenario, it was seen that DM hadnt pushed required config to MX
Per discussion with Suresh/Nischal/Prakash, it is due to address_family attribute in the session being empty

But, since the bgp-peer object itself has these family attributes, DM should consider it.
I still need to see who set the session attribute in bgp_router_refs(will update the bug), but above issue should be addressed nevertheless.

root@nodec1:~/contrail-test# curl -u admin:contrail123 http://127.0.0.1:8095/bgp-router/589dd1b9-7eb4-48de-8536-72e0dc92b23f | python -m json.tool
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
100 2766 100 2766 0 0 959k 0 --:--:-- --:--:-- --:--:-- 1350k
{
    "bgp-router": {
        "bgp_router_parameters": {
            "address": "7.7.7.77",
            "address_families": {
                "family": [
                    "route-target",
                    "inet-vpn",
                    "e-vpn",
                    "inet6-vpn"
                ]
            },
            "auth_data": null,
            "autonomous_system": 64512,
            "hold_time": 90,
            "identifier": "7.7.7.77",
            "local_autonomous_system": null,
            "port": null,
            "vendor": "juniper"
        },
        "bgp_router_refs": [
            {
                "attr": {
                    "session": [
                        {
                            "attributes": [
                                {
                                    "address_families": {
                                        "family": []
                                    },
                                    "auth_data": null,
                                    "bgp_router": null
                                }
                            ],
                            "uuid": null
                        }
                    ]
                },
                "href": "http://127.0.0.1:8095/bgp-router/3c79c915-125e-40c5-b3c8-c326745e6457",
                "to": [
                    "default-domain",
                    "default-project",
                    "ip-fabric",
                    "__default__",
                    "mx1"
                ],
                "uuid": "3c79c915-125e-40c5-b3c8-c326745e6457"
            },
            {
                "attr": {
                    "session": [
                        {
                            "attributes": [
                                {
                                    "address_families": {
                                        "family": []
                                    },
                                    "auth_data": null,
                                    "bgp_router": null
                                }
                            ],
                            "uuid": null
                        }
                    ]
                },
                "href": "http://127.0.0.1:8095/bgp-router/8d6a81ab-c18a-451f-a1a6-53de491a6af2",
                "to": [
                    "default-domain",
                    "default-project",
                    "ip-fabric",
                    "__default__",
                    "nodec1"
                ],
                "uuid": "8d6a81ab-c18a-451f-a1a6-53de491a6af2"
            },
            {
                "attr": {
                    "session": [
                        {
                            "attributes": [
                                {
                                    "address_families": {
                                        "family": []
                                    },
                                    "auth_data": null,
                                    "bgp_router": null
                                }
                            ],
                            "uuid": null
                        }
                    ]
                },
                "href": "http://127.0.0.1:8095/bgp-router/ba6a3d2a-be95-4f1b-99ec-38cc7fc35b68",
                "to": [
                    "default-domain",
                    "default-project",
                    "ip-fabric",
                    "__default__",
                    "nodec3"
                ],
                "uuid": "ba6a3d2a-be95-4f1b-99ec-38cc7fc35b68"
            },
            {
                "attr": {
                    "session": [
                        {
                            "attributes": [
                                {
                                    "address_families": {
                                        "family": []
                                    },
                                    "auth_data": null,
                                    "bgp_router": null
                                }
                            ],
                            "uuid": null
                        }
                    ]
                },
                "href": "http://127.0.0.1:8095/bgp-router/e5a10f3b-b6dc-47f0-a4b2-e79dca028edc",
                "to": [
                    "default-domain",
                    "default-project",
                    "ip-fabric",
                    "__default__",
                    "nodec2"
                ],
                "uuid": "e5a10f3b-b6dc-47f0-a4b2-e79dca028edc"
            }
        ],
        "display_name": "walsh",
        "fq_name": [
            "default-domain",
            "default-project",
            "ip-fabric",
            "__default__",
            "walsh"
        ],
        "href": "http://127.0.0.1:8095/bgp-router/589dd1b9-7eb4-48de-8536-72e0dc92b23f",
        "id_perms": {
            "created": "2015-08-21T09:51:40.337529",
            "creator": null,
            "description": null,
            "enable": true,
            "last_modified": "2015-08-21T09:51:40.495023",
            "permissions": {
                "group": "admin",
                "group_access": 7,
                "other_access": 7,
                "owner": "TestBasicBMSInterVN",
                "owner_access": 7
            },
            "user_visible": true,
            "uuid": {
                "uuid_lslong": 9598985965885633087,
                "uuid_mslong": 6385490441306130654
            }
        },
        "name": "walsh",
        "parent_href": "http://127.0.0.1:8095/routing-instance/f16c30c4-86c4-4ba7-870f-87c319c8cd37",
        "parent_type": "routing-instance",
        "parent_uuid": "f16c30c4-86c4-4ba7-870f-87c319c8cd37",
        "physical_router_back_refs": [
            {
                "attr": null,
                "href": "http://127.0.0.1:8095/physical-router/8047a7d1-9233-4ff0-b20c-6fabd0d5056c",
                "to": [
                    "default-global-system-config",
                    "walsh"
                ],
                "uuid": "8047a7d1-9233-4ff0-b20c-6fabd0d5056c"
            }
        ],
        "uuid": "589dd1b9-7eb4-48de-8536-72e0dc92b23f"
    }
}
----------------
MX netconf log :

Sep 21 20:47:02 Started tracing session: 88559
Sep 21 20:47:02 [88559] Incoming: <?xml version="1.0" encoding="UTF-8"?><nc:hello xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"><nc:capabilities><nc:capability>urn:ietf:para
ms:netconf:capability:writable-running:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:rollback-on-error:1.0</nc:capability><nc:capability>urn:ietf:params:ne
tconf:capability:validate:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:confirmed-commit:1.0</nc:capability><nc:capability>urn:ietf
Sep 21 20:47:02 Started tracing session: 88560
Sep 21 20:47:02 [88560] Incoming: <?xml version="1.0" encoding="UTF-8"?><nc:hello xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"><nc:capabilities><nc:capability>urn:ietf:para
ms:netconf:capability:writable-running:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:rollback-on-error:1.0</nc:capability><nc:capability>urn:ietf:params:ne
tconf:capability:validate:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:confirmed-commit:1.0</nc:capability><nc:capability>urn:ietf
Sep 21 20:47:02 [88559] Incoming: <?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:c65e54be-6229-11e5-a505-00
2590c30852"><nc:edit-config><nc:target><nc:candidate/></nc:target><nc:test-option>test-then-set</nc:test-option><nc:default-operation>merge</nc:default-operation><config><configu
ration><groups operation="replace"><name>__contrail__</name><protocols><bgp><group operation="replace"><name>__contrail__</name><type>internal</type><mult
Sep 21 20:47:02 [88559] Outgoing: <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/14.2I0/junos" xmlns:nc="urn:ietf:params:xml
:ns:netconf:base:1.0" message-id="urn:uuid:c65e54be-6229-11e5-a505-002590c30852">
Sep 21 20:47:02 [88559] Outgoing: <rpc-error>
Sep 21 20:47:02 [88559] Outgoing: <error-severity>warning</error-severity>
Sep 21 20:47:02 [88559] Outgoing: <error-path>[edit groups __contrail__ protocols bgp group __contrail__ neighbor 10.204.216.253]</error-path>
Sep 21 20:47:02 [88559] Outgoing: <error-message>mgd: statement has no contents; ignored</error-message>
Sep 21 20:47:02 [88559] Outgoing: <error-info>
Sep 21 20:47:02 [88559] Outgoing: <bad-element>family</bad-element>
Sep 21 20:47:02 [88559] Outgoing: </error-info>
Sep 21 20:47:02 [88559] Outgoing: </rpc-error>
Sep 21 20:47:02 [88559] Outgoing: <rpc-error>
Sep 21 20:47:02 [88559] Outgoing: <error-severity>warning</error-severity>
Sep 21 20:47:02 [88559] Outgoing: <error-path>[edit groups __contrail__ protocols bgp group __contrail_external__ neighbor 10.204.216.58]</error-path>
Sep 21 20:47:02 [88559] Outgoing: <error-message>mgd: statement has no contents; ignored</error-message>
Sep 21 20:47:02 [88559] Outgoing: <error-info>
Sep 21 20:47:02 [88559] Outgoing: <bad-element>family</bad-element>
Sep 21 20:47:02 [88559] Outgoing: </error-info>
Sep 21 20:47:02 [88559] Outgoing: </rpc-error>
Sep 21 20:47:02 [88559] Outgoing: <rpc-error>
Sep 21 20:47:02 [88559] Outgoing: <error-severity>warning</error-severity>
Sep 21 20:47:02 [88559] Outgoing: <error-path>[edit groups __contrail__ protocols bgp group __contrail_external__ neighbor 10.204.216.59]</error-path>
Sep 21 20:47:02 [88559] Outgoing: <error-message>mgd: statement has no contents; ignored</error-message>
Sep 21 20:47:02 [88559] Outgoing: <error-info>
Sep 21 20:47:02 [88559] Outgoing: <bad-element>family</bad-element>
Sep 21 20:47:02 [88559] Outgoing: </error-info>
Sep 21 20:47:02 [88559] Outgoing: </rpc-error>
Sep 21 20:47:02 [88559] Outgoing: <rpc-error>
Sep 21 20:47:02 [88559] Outgoing: <error-severity>warning</error-severity>
Sep 21 20:47:02 [88559] Outgoing: <error-path>[edit groups __contrail__ protocols bgp group __contrail_external__ neighbor 10.204.216.60]</error-path>
Sep 21 20:47:02 [88559] Outgoing: <error-message>mgd: statement has no contents; ignored</error-message>
Sep 21 20:47:02 [88559] Outgoing: <error-info>
Sep 21 20:47:02 [88559] Outgoing: <bad-element>family</bad-element>
Sep 21 20:47:02 [88559] Outgoing: </error-info>
Sep 21 20:47:02 [88559] Outgoing: </rpc-error>
Sep 21 20:47:03 [88559] Outgoing: <ok/>
Sep 21 20:47:03 [88559] Outgoing: </rpc-reply>
Sep 21 20:47:03 [88559] Outgoing: ]]>]]>
Sep 21 20:47:03 [88559] Incoming: <?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:c73fb058-6229-11e5-a505-00
2590c30852"><nc:commit/></nc:rpc>]]>]]>
Sep 21 20:47:03 [88559] Outgoing: <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/14.2I0/junos" xmlns:nc="urn:ietf:params:xml
:ns:netconf:base:1.0" message-id="urn:uuid:c73fb058-6229-11e5-a505-002590c30852">
Sep 21 20:47:13 [88559] Outgoing: <ok/>
Sep 21 20:47:14 [88559] Outgoing: </rpc-reply>
Sep 21 20:47:14 [88559] Outgoing: ]]>]]>
Sep 21 20:47:14 [88559] Incoming: <?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:cd845cac-6229-11e5-a505-00
2590c30852"><nc:close-session/></nc:rpc>]]>]]>
Sep 21 20:47:14 [88559] Outgoing: <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/14.2I0/junos" xmlns:nc="urn:ietf:params:xml
:ns:netconf:base:1.0" message-id="urn:uuid:cd845cac-6229-11e5-a505-002590c30852">
Sep 21 20:47:14 [88559] Outgoing: <ok/>
Sep 21 20:47:14 [88559] Outgoing: </rpc-reply>
Sep 21 20:47:14 [88559] Outgoing: ]]>]]>
Sep 21 20:47:14 [88559] Outgoing: <!-- session end at 2015-09-21 20:47:14 UTC -->
Sep 21 20:47:16 Started tracing session: 88900
Sep 21 20:47:16 [88900] Incoming: <?xml version="1.0" encoding="UTF-8"?><nc:hello xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"><nc:capabilities><nc:capability>urn:ietf:para
ms:netconf:capability:writable-running:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:rollback-on-error:1.0</nc:capability><nc:capability>urn:ietf:params:ne
tconf:capability:validate:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:confirmed-commit:1.0</nc:capability><nc:capability>urn:ietf
Sep 21 20:47:16 [88900] Incoming: <?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:ced8fec8-6229-11e5-a505-00
2590c30852"><nc:edit-config><nc:target><nc:candidate/></nc:target><nc:test-option>test-then-set</nc:test-option><nc:default-operation>merge</nc:default-operation><config><configu
ration><groups operation="replace"><name>__contrail__</name><protocols><bgp><group operation="replace"><name>__contrail__</name><type>internal</type><mult
Sep 21 20:47:16 [88900] Outgoing: <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/14.2I0/junos" xmlns:nc="urn:ietf:params:xml
:ns:netconf:base:1.0" message-id="urn:uuid:ced8fec8-6229-11e5-a505-002590c30852">
Sep 21 20:47:16 [88900] Outgoing: <rpc-error>
Sep 21 20:47:16 [88900] Outgoing: <error-severity>warning</error-severity>
Sep 21 20:47:16 [88900] Outgoing: <error-path>[edit groups __contrail__ protocols bgp group __contrail__ neighbor 10.204.216.253]</error-path>
Sep 21 20:47:16 [88900] Outgoing: <error-message>mgd: statement has no contents; ignored</error-message>
Sep 21 20:47:16 [88900] Outgoing: <error-info>
Sep 21 20:47:16 [88900] Outgoing: <bad-element>family</bad-element>
Sep 21 20:47:16 [88900] Outgoing: </error-info>
Sep 21 20:47:16 [88900] Outgoing: </rpc-error>
Sep 21 20:47:16 [88900] Outgoing: <rpc-error>
Sep 21 20:47:16 [88900] Outgoing: <error-severity>warning</error-severity>
Sep 21 20:47:16 [88900] Outgoing: <error-path>[edit groups __contrail__ protocols bgp group __contrail_external__ neighbor 10.204.216.58]</error-path>
Sep 21 20:47:16 [88900] Outgoing: <error-message>mgd: statement has no contents; ignored</error-message>
Sep 21 20:47:16 [88900] Outgoing: <error-info>
Sep 21 20:47:16 [88900] Outgoing: <bad-element>family</bad-element>
Sep 21 20:47:16 [88900] Outgoing: </error-info>
Sep 21 20:47:16 [88900] Outgoing: </rpc-error>
Sep 21 20:47:16 [88900] Outgoing: <rpc-error>
Sep 21 20:47:16 [88900] Outgoing: <error-severity>warning</error-severity>
Sep 21 20:47:16 [88900] Outgoing: <error-path>[edit groups __contrail__ protocols bgp group __contrail_external__ neighbor 10.204.216.59]</error-path>
Sep 21 20:47:16 [88900] Outgoing: <error-message>mgd: statement has no contents; ignored</error-message>
Sep 21 20:47:16 [88900] Outgoing: <error-info>
Sep 21 20:47:16 [88900] Outgoing: <bad-element>family</bad-element>
Sep 21 20:47:16 [88900] Outgoing: </error-info>
Sep 21 20:47:16 [88900] Outgoing: </rpc-error>
Sep 21 20:47:16 [88900] Outgoing: <rpc-error>
Sep 21 20:47:16 [88900] Outgoing: <error-severity>warning</error-severity>
Sep 21 20:47:16 [88900] Outgoing: <error-path>[edit groups __contrail__ protocols bgp group __contrail_external__ neighbor 10.204.216.60]</error-path>
Sep 21 20:47:16 [88900] Outgoing: <error-message>mgd: statement has no contents; ignored</error-message>
Sep 21 20:47:16 [88900] Outgoing: <error-info>
Sep 21 20:47:16 [88900] Outgoing: <bad-element>family</bad-element>
Sep 21 20:47:16 [88900] Outgoing: </error-info>
Sep 21 20:47:16 [88900] Outgoing: </rpc-error>
Sep 21 20:47:17 [88900] Outgoing: <ok/>
Sep 21 20:47:17 [88900] Outgoing: </rpc-reply>
Sep 21 20:47:17 [88900] Outgoing: ]]>]]>
Sep 21 20:47:17 [88900] Incoming: <?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:cfa33a94-6229-11e5-a505-00
2590c30852"><nc:commit/></nc:rpc>]]>]]>
Sep 21 20:47:17 [88900] Outgoing: <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/14.2I0/junos" xmlns:nc="urn:ietf:params:xml
:ns:netconf:base:1.0" message-id="urn:uuid:cfa33a94-6229-11e5-a505-002590c30852">
Sep 21 20:47:28 [88900] Outgoing: <ok/>
Sep 21 20:47:29 [88900] Outgoing: </rpc-reply>
Sep 21 20:47:29 [88900] Outgoing: ]]>]]>
Sep 21 20:47:29 [88900] Incoming: <?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:d67477ca-6229-11e5-a505-00
2590c30852"><nc:close-session/></nc:rpc>]]>]]>
Sep 21 20:47:29 [88900] Outgoing: <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/14.2I0/junos" xmlns:nc="urn:ietf:params:xml
:ns:netconf:base:1.0" message-id="urn:uuid:d67477ca-6229-11e5-a505-002590c30852">
Sep 21 20:47:29 [88900] Outgoing: <ok/>
Sep 21 20:47:29 [88900] Outgoing: </rpc-reply>
Sep 21 20:47:29 [88900] Outgoing: ]]>]]>

[Vedamurthy Joshi (vedujoshi)]

Have ibgp-auto-mesh set and create bgp-routers…one would see the same empty family values in session object

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/14455
Submitter: Suresh Balineni (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/14457
Submitter: Suresh Balineni (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/14457
Committed: http://github.org/Juniper/contrail-controller/commit/5c67f2e51194f42aca5258dbc9c37a47065bf827
Submitter: Zuul
Branch: R2.20

commit 5c67f2e51194f42aca5258dbc9c37a47065bf827
Author: sbalineni <email address hidden>
Date: Thu Oct 15 15:28:46 2015 -0700

DM: DM should not add family tag if family is not present

Sometimes, it is noticed that Bgp Session paramter has config
something like with all empty values:

"session": [
{
"attributes": [
{
"address_families": {
"family": []
},
"auth_data": null,
"bgp_router": null
}
],
"uuid": null
}

DM should check for real presense of family value.

Change-Id: I4d48e1da97999a52e4c381521a69ba5735145360
Closes-Bug: #1499187

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/14622
Submitter: Suresh Balineni (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/14622
Committed: http://github.org/Juniper/contrail-controller/commit/b68d61caf88b68c5646836e8a00027fb1b4f5253
Submitter: Zuul
Branch: master

commit b68d61caf88b68c5646836e8a00027fb1b4f5253
Author: sbalineni <email address hidden>
Date: Fri Oct 23 10:42:04 2015 -0700

DM: Periodic push of config and control the push interval

- on commit failure, re push the config in incremental intervals
start with 15 seconds delay and double it until it reaches max config
value which is 600seconds, and then continue to re push
on succesful commit, reset this value to 15 seconds

default start time and max interval is daemon config parameters

- on succesful commit, wait for some time before pushing the next commit
this is mainly for not sending too many commits in short intervals.

delay is computed based on the previous commit size.
default value: 1 second delay per every 100kb

There is a daemon knob to disable this feature.

- Cleaned up some code

- Sometimes, it is noticed that Bgp Session paramter has config
something like with all empty values:

"session": [
{
"attributes": [
{
"address_families": {
"family": []
},
"auth_data": null,
"bgp_router": null
}
],
"uuid": null
}

DM should check for real presense of family value.

Closes-Bug: #1469986
Closes-Bug: #1499187

Change-Id: I708648e02e6dfd8bbdcef0bdb708882245e64c7e