fab add_vrouter_node fails with Netronome build 124

[Savithru Lokanath]

Netronome Package: ns-agilio-vrouter-release_3.1.0.0-124
Contrail Package: contrail-install-packages_3.1.2.0-X

I tried to add a new node to an existing Contrail cluster & ran into an issue where "fab add_vrouter_node,<node>" fails with the following error:

2017-03-21 14:23:58:130572: [root@10.87.65.19] sudo: python -c 'from platform import linux_distribution; print linux_distribution()'
2017-03-21 14:23:58:130721: [root@10.87.65.19] out: ('Ubuntu', '14.04', 'trusty')
2017-03-21 14:23:58:227933: [root@10.87.65.19] out:
2017-03-21 14:23:58:231324:
2017-03-21 14:23:58:260113: Disconnecting from 10.87.65.19... done.
2017-03-21 14:23:58:377769: Disconnecting from 10.87.65.122... done.
2017-03-21 14:23:58:409519:
2017-03-21 14:23:50:519284: Warning: sudo() received nonzero return code 2 while executing 'sudo sed -i 's/ENABLED=.*/ENABLED=1/g' /etc/default/haproxy'!
2017-03-21 14:23:50:519284:
2017-03-21 14:23:50:519284: Traceback (most recent call last):
2017-03-21 14:23:58:260343: File "/usr/local/lib/python2.7/dist-packages/fabric/main.py", line 743, in main
2017-03-21 14:23:58:260364: *args, **kwargs
2017-03-21 14:23:58:260483: File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 415, in execute
2017-03-21 14:23:58:260514: results['<local-only>'] = task.run(*args, **new_kwargs)
2017-03-21 14:23:58:260597: File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 171, in run
2017-03-21 14:23:58:260610: return self.wrapped(*args, **kwargs)
2017-03-21 14:23:58:260649: File "/opt/contrail/utils/fabfile/tasks/misc.py", line 35, in add_vrouter_node
2017-03-21 14:23:58:260660: execute("setup_vrouter_node", env.host_string)
2017-03-21 14:23:58:261287: File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 415, in execute
2017-03-21 14:23:58:261345: results['<local-only>'] = task.run(*args, **new_kwargs)
2017-03-21 14:23:58:261474: File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 171, in run
2017-03-21 14:23:58:261501: return self.wrapped(*args, **kwargs)
2017-03-21 14:23:58:261579: File "/opt/contrail/utils/fabfile/tasks/provision.py", line 1612, in setup_vrouter_node
2017-03-21 14:23:58:261603: execute("setup_only_vrouter_node", 'yes', 'yes', host_string)
2017-03-21 14:23:58:262613: File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 415, in execute
2017-03-21 14:23:58:262658: results['<local-only>'] = task.run(*args, **new_kwargs)
2017-03-21 14:23:58:262818: File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 171, in run
2017-03-21 14:23:58:262849: return self.wrapped(*args, **kwargs)
2017-03-21 14:23:58:262931: File "/opt/contrail/utils/fabfile/tasks/provision.py", line 1697, in setup_only_vrouter_node
2017-03-21 14:23:58:262981: control_data_dict)
2017-03-21 14:23:58:263343: File "/opt/contrail/utils/fabfile/utils/ns_agilio_vrouter.py", line 102, in __init__
2017-03-21 14:23:58:263373: for member in self._node_bond['member']:
2017-03-21 14:23:58:263662: KeyError: 'member'
2017-03-21 14:23:58:263768:

In fabfile/tasks/provision.py:2931 there is a delete oper on bondinfo['member'] which causes testbed variable to exclude bond members referenced later.

This is the workaround, initialized a new variable bond_mode:

vim fabfile/tasks/provision.py:2931

if not bondinfo[host].has_key('member'):
                    raise AttributeError('Bond members are not defined for'+ \
                                         ' host %s, device %s' %(host, device))
                bond_members = " ".join(bondinfo[host]['member'])
+ bond_mode = deepcopy(bondinfo[host])
+ del bond_mode['member']; del bond_mode['name']
+ cmd += ' --members %s --bond-opts \'%s\''%(bond_members,
                                             json.dumps(bond_mode))

[Savithru Lokanath (slokanath)]

Moving to Juniper OpenStack

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/29812
Submitter: venu kolli (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/29812
Committed: http://github.org/Juniper/contrail-fabric-utils/commit/3f86de4d84d3ed9d59d44074f266901e9cba9497
Submitter: Zuul (<email address hidden>)
Branch: R3.1

commit 3f86de4d84d3ed9d59d44074f266901e9cba9497
Author: venu kolli <email address hidden>
Date: Wed Mar 22 13:32:29 2017 -0700

Closes Bug:#1674853 , bondinfo is being modified and later reference to bondinfo from testbed is failing to get the bond member

Change-Id: I0ec35cd59699aaf81cf86b827c2d256dc5054a3d

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/29895
Submitter: venu kolli (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/29895
Committed: http://github.org/Juniper/contrail-fabric-utils/commit/a6b69c3f42ff569aa2f0ab3fe6923f0ceb75f471
Submitter: Zuul (<email address hidden>)
Branch: master

commit a6b69c3f42ff569aa2f0ab3fe6923f0ceb75f471
Author: venu kolli <email address hidden>
Date: Wed Mar 22 13:32:29 2017 -0700

Closes Bug:#1674853 , bondinfo is being modified and later reference to bondinfo from testbed is failing to get the bond member

Change-Id: I0ec35cd59699aaf81cf86b827c2d256dc5054a3d

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/31361
Submitter: venu kolli (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/31361
Committed: http://github.com/Juniper/contrail-fabric-utils/commit/28ebac88b91d632d8bbde957239b6af095cead08
Submitter: Zuul (<email address hidden>)
Branch: R3.2

commit 28ebac88b91d632d8bbde957239b6af095cead08
Author: venu kolli <email address hidden>
Date: Wed Mar 22 13:32:29 2017 -0700

Closes Bug:#1674853 , bondinfo is being modified and later reference to bondinfo from testbed is failing to get the bond member

Change-Id: I0ec35cd59699aaf81cf86b827c2d256dc5054a3d
(cherry picked from commit 3f86de4d84d3ed9d59d44074f266901e9cba9497)