[DPDK] Network verification fails while adding new compute node

[Artem Panchenko]

Network verification fails after adding of new compute node with enabled DPDK to operational cluster:

[root@nailgun ~]# fuel task --task 54
id | status | name | cluster | progress | uuid
---+--------+-----------------+---------+----------+-------------------------------------
54 | error | verify_networks | 1 | 100 | 3905c182-dbe3-4e0b-a9be-f49a8a47c37e

Verification failed.
Node Name Node MAC address Node Interface Expected VLAN (not received)
slave-04_compute 64:80:9a:af:3b:ef enp0s6 1024, 1026, 1027, 1028, 1029, 1030, 1025, 1000, 1001, 1002, 1003, 1004, 1005, 1006, 1007, 1008, 1009, 1010, 1011, 1012, 1013, 1014, 1015, 1016, 1017, 1018, 1019, 1020, 1021, 1022, 1023

Steps to reproduce:

1. Deploy environment with 1 controller and N compute nodes with enabled DPDK (use VLAN and KVM, allocate HugePages for DPDK on computes)
2. Add new compute node and enable DPDK
3. Run network verification

Expected result: verification is passed

Actual result: verification fails, no tagged (private network) traffic is received on new compute from controller, because it's skipped

Here is the patch for nailgun which helps: http://paste.openstack.org/show/508459/

[Artem Panchenko (apanchenko-8)]

Igor K. helped me to investigate this issue and asked me to assign this bug to him.

[Dmitry Klenov (dklenov)]

Raised to high as basic operation (compute node addition) is not functioning properly and produces incorrect user experience.

[Ihor Kalnytskyi (ikalnytskyi)]

Dmitry K.,

You are wrong here. That would happened only and only iff:

* The cluster have precisely 1 node without DPDK
* User adds only DPDK enabled nodes

In that case, VLAN verification will fail. But that failure is not destructive. Everything is fine. Taking into account that (1) DPDK is considered experimental and (2) the case with precisely one DPDK disabled node (usually people have at least 3 controllers), I think that bug should be considered as Medium.

[Dmitry Klenov (dklenov)]

Igor, thanks for the investigation. Having that failure is not destructive and node will still deploy, lowering severity to medium and removing bug from 9.0.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/326550

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/326550
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=04443331a3ea4d0cfe2c22c4b7cb099aa1d29eb0
Submitter: Jenkins
Branch: master

commit 04443331a3ea4d0cfe2c22c4b7cb099aa1d29eb0
Author: Igor Kalnitsky <email address hidden>
Date: Tue Jun 7 17:31:44 2016 +0300

    Fix VLAN verification for discover nodes with DPDK

    In rare cases when we have 1 ready node without DPDK and 1+ discover
    nodes with DPDK, Nailgun has produced a wrong verification message.
    That as happened because Nailgun mistakenly assumes 1 ready node as DPDK
    capable and didn't serialize VLAN verification message for this node.

    Co-Authored-By: Artem Panchenko <email address hidden>
    Closes-Bug: #1589707

    Change-Id: Id01d8772707994ed6da8b0c3979693580a3c417f