R4.1 build 15 Netronome build 47: Port mirroring is not working

Bug #1797216 reported by Ankit Jain
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
Fix Released
Pieter Malan
Fix Committed
Pieter Malan

Bug Description

Following is the scenario:

Compute/VM: SRC: nodei8 /, -> DST: nodel9 / => ANALYZER: nodel9 /


1) Ping from count =5
2) Enable port mirroring on the port

Observation : Expecting 10 packets, received only 2

[ubuntu@] sudo: ping -s 1200 -c 5 -W 1
[ubuntu@] out: PING ( 1200(1228) bytes of data.
[ubuntu@] out: 1208 bytes from icmp_seq=1 ttl=63 time=1.45 ms
[ubuntu@] out: 1208 bytes from icmp_seq=2 ttl=63 time=0.522 ms
[ubuntu@] out: 1208 bytes from icmp_seq=3 ttl=63 time=0.550 ms
[ubuntu@] out: 1208 bytes from icmp_seq=4 ttl=63 time=0.671 ms
[ubuntu@] out: 1208 bytes from icmp_seq=5 ttl=63 time=0.671 ms
[ubuntu@] out:

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
19:08:50.524659 IP > UDP, length 1581
19:08:52.623990 IP > UDP, length 1309

Counters before and after:

root@nodel9:/home/ubuntu# /opt/netronome/libexec/vr_vif_cntrs -v 3
Obtaining VIF Counters for each VIF entry
VIF index 3
 VIF Input bytes=131239
 VIF Input packets=292
 VIF Input errors=10
 VIF Output bytes=3585182
 VIF Output packets=31305
 VIF Output errors=0
root@nodel9:/home/ubuntu# /opt/netronome/libexec/vr_vif_cntrs -v 3
Obtaining VIF Counters for each VIF entry
VIF index 3
 VIF Input bytes=137449
 VIF Input packets=297
 VIF Input errors=10
 VIF Output bytes=3593390
 VIF Output packets=31329
 VIF Output errors=0

Netronome team needs to debug it further. Without Netronome acceleration the mirroring works as expected.

Revision history for this message
Pieter Malan (pmalan) wrote :

I am not able to reproduce the mirroring issue locally.

We are seeing VRCNTR_WIRE_DROP_RPF_FAIL increments on the Ankit's setup.

            rpf_result = rpf_check(input_vif_index, VIF_TYPE_PHYSICAL);
            if (rpf_result == NH_SOURCE_SEND_TO_HOST) {
                goto send_to_vr;
            } else if (rpf_result == NH_SOURCE_INVALID) {
                /* Drop the packet. */
                goto drop;

input_vif_index is used to determine the rpf_result, this issue may be related to https://bugs.launchpad.net/juniperopenstack/+bug/1797358, where the input_vif_index is not valid.

Revision history for this message
Ankit Jain (ankitja) wrote :

Seeing the same issue even with nfp_p0 intf.

Ankit Jain (ankitja)
tags: added: blocker
Revision history for this message
Pieter Malan (pmalan) wrote :

Observation is that traffic is mirrored but the source IP is the underlay IP and not the IP of the port mirrored:

Expected Packet:
07:05:19.139958 IP > UDP, length 127

Received Packet:
07:04:11.445307 IP > UDP, length 100

We are busy investigating it internally and working on a fix.

Revision history for this message
Jeba Paulaiyan (jebap) wrote :

Hi Pieter,

The bug 1797216 is not seen in the latest Netronome build 82.

I’ve run the following tests to validate the same:

1) TestIntfMirror.test_intf_mirror_src_cn1vn1_dst_cn2vn2_analyzer_cn3vn3
2018-10-25 15:34:59,930 - INFO - END TEST : test_intf_mirror_src_cn1vn1_dst_cn2vn2_analyzer_cn3vn3 : PASSED[0:02:57]

2) TestIntfMirror.test_intf_mirror_with_subintf_src_cn1vn1_dst_cn2vn2_analyzer_cn3vn3
2018-10-255:31:37,749 - INFO - END TEST : test_intf_mirror_with_subintf_src_cn1vn1_dst_cn2vn2_analyzer_cn3vn3 : PASSED[0:03:04]

3) TestIntfMirror.test_intf_mirroring_disable_enable_scenarios
2018-10-25 15:26:41,529 - INFO - END TEST : test_intf_mirroring_disable_enable_scenarios : PASSED[0:06:52]

4) TestIntfMirror.test_juniper_header
2018-10-25 15:38:04,656 - INFO - END TEST : test_juniper_header : PASSED[0:04:43]

5) TestSVCV2MirrorIPv6.test_svc_v2_mirroring
2018-10-25 15:40:42,006 - INFO - END TEST : test_svc_v2_mirroring : PASSED[0:02:20]

6) TestSVCV2Mirror.test_svc_v2_mirroring
2018-10-25 15:43:49,733 - INFO - END TEST : test_svc_v2_mirroring : PASSED[0:02:11]


Revision history for this message
Jeba Paulaiyan (jebap) wrote :

Fixed in Netronome build 82

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.