Convert legacy nova VIF plugging to os-vif

[Jan Gutter]

Currently vrouter plugging in OpenStack is fragmented into a number of parallel codepaths:

* The in-tree legacy Nova plugging code for the 'vrouter' VIF type (TAP device plugging)
* In-tree legacy Nova plugging code for the 'hw_veb' VIF (passthrough plugging)
* os-vif code for the 'contrail_vrouter' VIF type (DPDK vhostuser plugging)

The Nova project does not wish to maintain the burden of parallel implementations, and any future work on the legacy VIF plugging methods is unlikely to be accepted.

This bug tracks the migration to os-vif, which will hopefully be completed in the Rocky cycle.

The corresponding work on Nova and os-vif is tracked by this blueprint:
https://blueprints.launchpad.net/nova/+spec/vrouter-os-vif-conversion

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/43359
Submitter: Jan Gutter (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/43394
Submitter: Jan Gutter (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/43399
Submitter: Jan Gutter (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/43359
Submitter: Jan Gutter (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/43394
Submitter: Jan Gutter (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/43399
Submitter: Jan Gutter (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/43359
Committed: http://github.com/Juniper/contrail-nova-vif-driver/commit/208f0d51b6e0c1c44771e96c062588a40491b09d
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 208f0d51b6e0c1c44771e96c062588a40491b09d
Author: Jan Gutter <email address hidden>
Date: Tue May 29 14:50:45 2018 +0200

Build skeleton for 'vrouter' os-vif plugin

The current os-vif plugin operates on the contrail_vrouter VIF type.
This means that the only plugging mode supported is the vhost-user
method with the DPDK datapath.

This changeset renames the 'contrail_vrouter' plugin back to 'vrouter'
and adds a small shim for a 'contrail_vrouter' plugin. The
'contrail_vrouter' plugin will be a strict subset of the 'vrouter'
plugin, supporting only the vhost-user datapath for backwards
compatibility.

This changeset clones the existing unit tests to the 'contrail_vrouter'
plugin, ensuring that the interfaces can be maintained as work progresses
on the 'vrouter' plugin.

This changeset also cleans up a number of PEP8 issues and
failing unit tests in the current os-vif plugin:

* Whitespace issues
* Import ordering
* Filling in proper UUID's in tests
* Better handling of exceptions when config is not available

Change-Id: Ica76793e7977616236b07efbfaa115fe85c97fab
Partial-Bug: #1773981
Signed-off-by: Jan Gutter <email address hidden>
Acked-by: Frikkie Scholtz <email address hidden>

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/43394
Committed: http://github.com/Juniper/contrail-nova-vif-driver/commit/c1354fdb1459adc8df096cc9dae1c353ed4356aa
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit c1354fdb1459adc8df096cc9dae1c353ed4356aa
Author: Jan Gutter <email address hidden>
Date: Wed May 30 13:08:32 2018 +0200

Refactor call to vrouter-port-control

This changeset breaks the vrouter-port-control call out into
discrete functions. This is a cleanup, preserving existing
test expectations, to allow for common code to be used with multiple
plugging modes.

The unit tests have been updated to reflect that the calls to the
vrouter-port-control script are now discrete parameters rather than
a single string.

Change-Id: Ia554b5d1e3036eff06762dd4e8fe71a9e6715a35
Partial-Bug: #1773981
Signed-off-by: Jan Gutter <email address hidden>

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/43399
Committed: http://github.com/Juniper/contrail-nova-vif-driver/commit/ed01d315e5707b4f670468454729dc2031c5f780
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit ed01d315e5707b4f670468454729dc2031c5f780
Author: Jan Gutter <email address hidden>
Date: Wed May 30 15:15:26 2018 +0200

Add kernel vrouter.ko datapath plugging support

This change adds the missing sections to support kernel TAP device
plugging for the os-vif plugin. Currently this is keyed off the VIF type
passed to the plugin.

Note: this plugin requires a minimum of libvirt 1.3.1.
This is also the minimum version supported in OpenStack Rocky.

* Unit tests have been updated:
contrail_vrouter os-vif plugin tests only vhostuser plugging
vrouter os-vif plugin tests both methods

Change-Id: I74dca19a6c8a5676caa6423619b1f593a8cc1b3e
Partial-Bug: #1773981
Signed-off-by: Jan Gutter <email address hidden>

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45345
Submitter: Andrey Pavlov (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45346
Submitter: Andrey Pavlov (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/45345
Committed: http://github.com/Juniper/contrail-container-builder/commit/b4d5dd38e81633c9d763e8a70eed7dfb86d5db0c
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit b4d5dd38e81633c9d763e8a70eed7dfb86d5db0c
Author: Andrey Pavlov <email address hidden>
Date: Tue Aug 7 15:14:56 2018 +0300

Add copying of new part of code for vrouter into nova-compute

Change-Id: Ifb9c568f9b241d0f1f5ccd03c031f22a2609940c
Partial-Bug: #1773981

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/45346
Committed: http://github.com/Juniper/contrail-container-builder/commit/72251d24f17456e3eb6524e4b0aed650088ea90d
Submitter: Zuul v3 CI (<email address hidden>)
Branch: R5.0

commit 72251d24f17456e3eb6524e4b0aed650088ea90d
Author: Andrey Pavlov <email address hidden>
Date: Tue Aug 7 15:14:56 2018 +0300

Add copying of new part of code for vrouter into nova-compute

Change-Id: Ifb9c568f9b241d0f1f5ccd03c031f22a2609940c
Partial-Bug: #1773981

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45505
Submitter: Andrey Pavlov (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45506
Submitter: Andrey Pavlov (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/45505
Committed: http://github.com/Juniper/contrail-packages/commit/65d550bdfb0f8eee8278dfd3cd0393b0f381ec7d
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 65d550bdfb0f8eee8278dfd3cd0393b0f381ec7d
Author: Andrey Pavlov <email address hidden>
Date: Sat Aug 11 12:44:11 2018 +0300

add new vif_plug_contrail_vrouter to nova_vif driver

Change-Id: I5255ed44b3ace5de38e7377e4c0e5a8417230a57
Partial-Bug: #1773981

[Andrey Pavlov (apavlov-e)]

review 45505 fixes next error:

2018-08-30 16:00:49.627 7 ERROR os_vif [req-474a6358-be7e-4c27-aeab-9fc32caf4422 05138186edbb44819ff4778157bf29c4 b1b0008d503f472e9ffd0858580c7a40 - default default] Failed to plug vif VIFVHostUser(active=True,address=02:94:c3:dd:99:e1,has_traffic_filtering=False,id=94c3dd99-e177-4021-863f-eb6f2089b196,mode='server',network=Network(1aca4e19-63a6-4718-8a16-1ebdab91c86c),path='/var/run/vrouter/uvh_vif_tap94c3dd99-e1',plugin='contrail_vrouter',port_profile=<?>,preserve_on_delete=False,vif_name='tap94c3dd99-e1'): TypeError: Cannot serialize IPAddress('192.168.1.3')
2018-08-30 16:00:49.627 7 ERROR os_vif Traceback (most recent call last):
2018-08-30 16:00:49.627 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/os_vif/__init__.py", line 77, in plug
2018-08-30 16:00:49.627 7 ERROR os_vif plugin.plug(vif, instance_info)
2018-08-30 16:00:49.627 7 ERROR os_vif File "/opt/plugin/site-packages/vif_plug_vrouter/vrouter.py", line 210, in plug
2018-08-30 16:00:49.627 7 ERROR os_vif self._vrouter_port_add(instance_info, vif)
2018-08-30 16:00:49.627 7 ERROR os_vif File "/opt/plugin/site-packages/vif_plug_vrouter/vrouter.py", line 203, in _vrouter_port_add
2018-08-30 16:00:49.627 7 ERROR os_vif vhostuser_socket, vhostuser_mode)
2018-08-30 16:00:49.627 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/oslo_privsep/priv_context.py", line 207, in _wrap
2018-08-30 16:00:49.627 7 ERROR os_vif return self.channel.remote_call(name, args, kwargs)
2018-08-30 16:00:49.627 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/oslo_privsep/daemon.py", line 192, in remote_call
2018-08-30 16:00:49.627 7 ERROR os_vif result = self.send_recv((Message.CALL.value, name, args, kwargs))
2018-08-30 16:00:49.627 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/oslo_privsep/comm.py", line 163, in send_recv
2018-08-30 16:00:49.627 7 ERROR os_vif self.writer.send((myid, msg))
2018-08-30 16:00:49.627 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/oslo_privsep/comm.py", line 54, in send
2018-08-30 16:00:49.627 7 ERROR os_vif buf = msgpack.packb(msg, use_bin_type=True)
2018-08-30 16:00:49.627 7 ERROR os_vif File "/usr/lib64/python2.7/site-packages/msgpack/__init__.py", line 47, in packb
2018-08-30 16:00:49.627 7 ERROR os_vif return Packer(**kwargs).pack(o)
2018-08-30 16:00:49.627 7 ERROR os_vif File "/usr/lib64/python2.7/site-packages/msgpack/fallback.py", line 848, in pack
2018-08-30 16:00:49.627 7 ERROR os_vif self._pack(obj)
2018-08-30 16:00:49.627 7 ERROR os_vif File "/usr/lib64/python2.7/site-packages/msgpack/fallback.py", line 835, in _pack
2018-08-30 16:00:49.627 7 ERROR os_vif self._pack(obj[i], nest_limit - 1)
2018-08-30 16:00:49.627 7 ERROR os_vif File "/usr/lib64/python2.7/site-packages/msgpack/fallback.py", line 835, in _pack
2018-08-30 16:00:49.627 7 ERROR os_vif self._pack(obj[i], nest_limit - 1)
2018-08-30 16:00:49.627 7 ERROR os_vif File "/usr/lib64/python2.7/site-packages/msgpack/fallback.py", line 835, in _pack
2018-08-30 16:00:49.627 7 ERROR os_vif self._pack(obj[i], nest_limit - 1)
2018-08-30 16:00:49.627 7 ERROR os_vif File "/usr/lib64/python...

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45894
Submitter: Andrey Pavlov (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/45894
Committed: http://github.com/Juniper/contrail-nova-vif-driver/commit/65ecfdbfef0d3a581f3856034b66569fb4a145ea
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 65ecfdbfef0d3a581f3856034b66569fb4a145ea
Author: Andrey Pavlov <email address hidden>
Date: Thu Aug 30 19:28:57 2018 +0300

fix IPAddress serializing

nova passes netaddr.IPAddress to vif plugin that can't be
serialized by oslo/msgpack

Change-Id: I5a01f9c99088c285c606a5a49b47e96ed70ced74
Partial-Bug: #1773981