【hns】PF failed(-22) to config VF's VLAN

[Fred Kimmy]

[Bug Description]
PF failed(-22) to config VF's VLAN
[Steps to Reproduce]
a) config vf
echo 1 > /sys/class/net/<ethx>/device/sriov_numvfs
enable VF
ip link set dev <vf_name> up
b）step:
        1、machine 1
        ip link set dev <pf_name> vf 0 vlan <id>
        2、
        #machine 2：mz <pf_name> -t tcp sp=5001,dp=5003 -b <dst_mac> -p 800 -c 1000 -Q <QinQ>
        rx=1000
        #machine 1：ifconfig <vf_name>
        3、
        #machine 2：mz <pf_name> -t tcp sp=5001,dp=5003 -b <dst_mac> -p 800 -c 1000 -Q
         rx=0
        #machine 1：ifconfig <vf_name>
        4、
        #machine 1 2：ip link set dev <pf_name> vf 0 vlan 0

[Actual Results]
PF failed(-22) to config VF's VLAN

[Expected Results]
        A、rx=1000
        B、rx=0

[Reproducibility]

nput cmd in tc: cat /etc/os-release | grep -w ID | awk -F'=' '{print $2}' | tr '[:upper:]' '[:lower:]'
cur os name cp: ubuntu
[comm_sys_env_clean.py:273] -------------cur ip a info:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp189s0f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
    link/ether 18:3d:5e:8e:e5:42 brd ff:ff:ff:ff:ff:ff
3: enp189s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 18:3d:5e:8e:e5:43 brd ff:ff:ff:ff:ff:ff
    inet 192.168.40.11/24 brd 192.168.40.255 scope global enp189s0f1
       valid_lft forever preferred_lft forever
4: enp125s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 18:3d:5e:8e:e5:41 brd ff:ff:ff:ff:ff:ff
    inet 192.168.10.11/24 brd 192.168.10.255 scope global enp125s0f0
       valid_lft forever preferred_lft forever
5: enp1s0f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether 9c:52:f8:60:e1:9e brd ff:ff:ff:ff:ff:ff
6: enp1s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 9c:52:f8:60:e1:9f brd ff:ff:ff:ff:ff:ff
    inet 192.168.71.252/24 brd 192.168.71.255 scope global enp1s0f1
       valid_lft forever preferred_lft forever
    inet6 fe80::9e52:f8ff:fe60:e19f/64 scope link
       valid_lft forever preferred_lft forever
2022-01-17 03:40:17,961 INFO [comm_sys_env_clean.py:277] -------------cur cp ip a info:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp189s0f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
    link/ether 18:3d:5e:8e:e4:4e brd ff:ff:ff:ff:ff:ff
3: enp189s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 18:3d:5e:8e:e4:4f brd ff:ff:ff:ff:ff:ff
    inet 192.168.40.12/24 brd 192.168.40.255 scope global enp189s0f1
       valid_lft forever preferred_lft forever
4: enp125s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 18:3d:5e:8e:e4:4d brd ff:ff:ff:ff:ff:ff
    inet 192.168.10.12/24 brd 192.168.10.255 scope global enp125s0f0
       valid_lft forever preferred_lft forever
5: enp1s0f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether f4:1d:6b:45:3a:8f brd ff:ff:ff:ff:ff:ff
6: enp1s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether f4:1d:6b:45:3a:90 brd ff:ff:ff:ff:ff:ff
    inet 192.168.71.251/24 brd 192.168.71.255 scope global enp1s0f1
       valid_lft forever preferred_lft forever
    inet6 fe80::f61d:6bff:fe45:3a90/64 scope link
       valid_lft forever preferred_lft forever
cmd:ethtool --show-fec enp189s0f1 | grep 'Active' | awk -F ': ' '{print $2}', result:Off
2022-01-17 03:40:29,621 INFO [comm_sys_base_info.py:219] input cmd in tc: ethtool --show-fec enp189s0f1 | grep 'Active' | awk -F ': ' '{print $2}', return: Off
2022-01-17 03:40:29,624 INFO [comm_sys_base_info.py:221] cmd:echo 'ping -c 2 192.168.10.12 > 192.168.10.12.log &' >> nic_check.sh, result:
2022-01-17 03:40:29,627 INFO [comm_sys_base_info.py:221] cmd:echo 'ping -c 2 192.168.40.12 > 192.168.40.12.log &' >> nic_check.sh, result:
2022-01-17 03:40:30,651 INFO [comm_sys_base_info.py:221] cmd:cat 192.168.10.12.log, result:PING 192.168.10.12 (192.168.10.12) 56(84) bytes of data.
64 bytes from 192.168.10.12: icmp_seq=1 ttl=64 time=0.103 ms
64 bytes from 192.168.10.12: icmp_seq=2 ttl=64 time=0.035 ms
--- 192.168.10.12 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1012ms
rtt min/avg/max/mdev = 0.035/0.069/0.103/0.034 ms
2022-01-17 03:40:30,655 INFO [comm_sys_base_info.py:221] cmd:rm -f 192.168.10.12.log, result:
2022-01-17 03:40:30,658 INFO [comm_sys_base_info.py:221] cmd:cat 192.168.40.12.log, result:PING 192.168.40.12 (192.168.40.12) 56(84) bytes of data.
64 bytes from 192.168.40.12: icmp_seq=1 ttl=64 time=0.055 ms
64 bytes from 192.168.40.12: icmp_seq=2 ttl=64 time=0.044 ms
--- 192.168.40.12 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1012ms
rtt min/avg/max/mdev = 0.044/0.049/0.055/0.005 ms
2022-01-17 03:40:30,662 INFO [comm_sys_base_info.py:221] cmd:rm -f 192.168.40.12.log, result:
2022-01-17 03:40:32,716 INFO [comm_sys_env_clean.py:830] end of checking environment
2022-01-17 03:40:32,716 INFO [comm_sys_env_clean.py:850] environment is clean, no need restore!!!
2022-01-17 03:40:32,721 INFO [module_nic_sut.py:319] cmd:ethtool -i enp189s0f0 | grep 'driver: hns', result:driver: hns3
2022-01-17 03:40:32,725 INFO [module_nic_sut.py:319] cmd:ethtool -i enp189s0f1 | grep 'driver: hns', result:driver: hns3
2022-01-17 03:40:32,729 INFO [module_nic_sut.py:319] cmd:ethtool -i enp125s0f0 | grep 'driver: hns', result:driver: hns3
2022-01-17 03:40:32,733 WARNING [module_nic_sut.py:319] cmd:ethtool -i enp1s0f0 | grep 'driver: hns', ret_code:1, result:
2022-01-17 03:40:32,737 WARNING [module_nic_sut.py:319] cmd:ethtool -i enp1s0f1 | grep 'driver: hns', ret_code:1, result:
2022-01-17 03:40:32,741 INFO [module_nic_sut.py:323] cmd:ethtool enp189s0f0, result:Settings for enp189s0f0:
 Supported ports: [ TP ]
 Supported link modes: 10baseT/Half 10baseT/Full
                         100baseT/Half 100baseT/Full
                         1000baseT/Full
 Supported pause frame use: Symmetric Receive-only
 Supports auto-negotiation: Yes
 Supported FEC modes: Not reported
 Advertised link modes: 10baseT/Half 10baseT/Full
                         100baseT/Half 100baseT/Full
                         1000baseT/Full
 Advertised pause frame use: Symmetric
 Advertised auto-negotiation: Yes
 Advertised FEC modes: Not reported
 Speed: Unknown!
 Duplex: Unknown! (255)
 Port: Twisted Pair
 PHYAD: 0
 Transceiver: internal
 Auto-negotiation: on
 MDI-X: Unknown
 Current message level: 0x00000036 (54)
          probe link ifdown ifup
 Link detected: no
2022-01-17 03:40:32,744 INFO [module_nic_sut.py:323] cmd:ethtool enp189s0f1, result:Settings for enp189s0f1:
 Supported ports: [ TP ]
 Supported link modes: 10baseT/Half 10baseT/Full
                         100baseT/Half 100baseT/Full
                         1000baseT/Full
 Supported pause frame use: Symmetric Receive-only
 Supports auto-negotiation: Yes
 Supported FEC modes: Not reported
 Advertised link modes: 10baseT/Half 10baseT/Full
                         100baseT/Half 100baseT/Full
                         1000baseT/Full
 Advertised pause frame use: Symmetric
 Advertised auto-negotiation: Yes
 Advertised FEC modes: Not reported
 Link partner advertised link modes: 10baseT/Half 10baseT/Full
                                      100baseT/Half 100baseT/Full
                                      1000baseT/Full
 Link partner advertised pause frame use: Symmetric
 Link partner advertised auto-negotiation: Yes
 Link partner advertised FEC modes: Not reported
 Speed: 1000Mb/s
 Duplex: Full
 Port: Twisted Pair
 PHYAD: 1
 Transceiver: internal
 Auto-negotiation: on
 MDI-X: Unknown
 Current message level: 0x00000036 (54)
          probe link ifdown ifup
 Link detected: yes
2022-01-17 03:40:32,746 INFO [comm_sys_base_info.py:332] get enp189s0f1 IP address : 192.168.40.11
2022-01-17 03:40:34,791 INFO [module_nic_sut.py:323] cmd:ethtool enp125s0f0, result:Settings for enp125s0f0:
 Supported ports: [ FIBRE ]
 Supported link modes: 25000baseCR/Full
                         25000baseSR/Full
                         50000baseCR2/Full
                         100000baseSR4/Full
                         100000baseCR4/Full
                         100000baseLR4_ER4/Full
                         50000baseSR2/Full
                         1000baseX/Full
                         10000baseCR/Full
                         10000baseSR/Full
                         10000baseLR/Full
                         50000baseLR_ER_FR/Full
 Supported pause frame use: Symmetric
 Supports auto-negotiation: No
 Supported FEC modes: None RS
 Advertised link modes: Not reported
 Advertised pause frame use: No
 Advertised auto-negotiation: No
 Advertised FEC modes: Not reported
 Speed: 100000Mb/s
 Duplex: Full
 Port: FIBRE
 PHYAD: 0
 Transceiver: internal
 Auto-negotiation: off
 Current message level: 0x00000036 (54)
          probe link ifdown ifup
 Link detected: yes
2022-01-17 03:40:34,792 INFO [comm_sys_base_info.py:332] get enp125s0f0 IP address : 192.168.10.11
2022-01-17 03:40:36,840 INFO [module_nic_sut.py:377] cmd:ethtool -i enp125s0f0 | grep 'bus-info' | awk '{print $NF}' | sed 's/ //g', result:0000:7d:00.0
2022-01-17 03:40:36,841 INFO [comm_sys_base_info.py:332] get enp125s0f0 IP address : 192.168.10.11
2022-01-17 03:40:38,214 INFO [comm_sys_base_info.py:221] cmd:cat /etc/os-release | grep -w ID | awk -F'=' '{print $2}' | tr '[:upper:]' '[:lower:]', result:ubuntu
2022-01-17 03:40:38,214 INFO [module_nic_sut.py:958] **************Start sut enp125s0f0 create vf**************
2022-01-17 03:40:38,418 INFO [module_nic_sut.py:960] cmd:echo 1 > /sys/class/net/enp125s0f0/device/sriov_numvfs, result:
2022-01-17 03:40:40,422 INFO [module_nic_sut.py:969] sut enp125s0f0 vf port list: ['enp125s0f0v0']
2022-01-17 03:40:40,426 INFO [module_nic_sut.py:981] cmd:ip addr add 172.201.10.3/24 dev enp125s0f0v0, result:
2022-01-17 03:40:40,508 INFO [module_nic_sut.py:982] cmd:ip link set enp125s0f0v0 up, result:
2022-01-17 03:40:40,508 INFO [module_nic_sut.py:986] sut vf port:enp125s0f0v0 vf ip:172.201.10.3
2022-01-17 03:40:40,508 INFO [module_nic_sut.py:987] **************Finish sut enp125s0f0 create vf**************
2022-01-17 03:40:40,509 INFO [module_nic_sut.py:988] sut vf port:['enp125s0f0v0']
2022-01-17 03:40:40,509 INFO [module_nic_sut.py:989] sut vf ip:['172.201.10.3']
2022-01-17 03:40:40,517 INFO [onboard_port_vlan_012.py:84] cmd:ip link set dev enp125s0f0 vf 0 vlan 1, ret_code:0
2022-01-17 03:40:40,666 INFO [onboard_port_vlan_012.py:84] cmd:ip link set dev enp125s0f0 vf 0 vlan 2, ret_code:0
2022-01-17 03:40:40,817 INFO [onboard_port_vlan_012.py:84] cmd:ip link set dev enp125s0f0 vf 0 vlan 3, ret_code:0
。。。。。
2022-01-17 03:50:16,339 INFO [onboard_port_vlan_012.py:84] cmd:ip link set dev enp125s0f0 vf 0 vlan 4094, ret_code:0
2022-01-17 03:50:16,477 INFO [onboard_port_vlan_012.py:84] cmd:ip link set dev enp125s0f0 vf 0 vlan 4095, ret_code:0
2022-01-17 03:50:16,481 ERROR [onboard_port_vlan_012.py:84] cmd:ip link set dev enp125s0f0 vf 0 vlan 4096, ret_code:255, error:Error: argument \"4096\" is wrong: Invalid \"vlan\" value
2022-01-17 03:50:16,617 INFO [onboard_port_vlan_012.py:90] cmd:ip a show enp125s0f0v0 | grep ether | awk '{print $2}', result:ee:f7:d9:13:55:6a
2022-01-17 03:50:16,754 INFO [onboard_port_vlan_012.py:91] cmd:ifconfig enp125s0f0v0 | grep 'RX packets', result:RX packets 0 bytes 0 (0.0 B)
2022-01-17 03:50:16,755 INFO [mz.py:50] check mz
2022-01-17 03:50:16,755 INFO [mz.py:40] install mz
2022-01-17 03:50:16,756 INFO [mz.py:87] mz enp125s0f0 -t tcp sp=5001,dp=5003 -b ee:f7:d9:13:55:6a -p 800 -c 1000 -Q 4095
2022-01-17 03:50:17,532 INFO [mz.py:90]
2022-01-17 03:50:17,537 INFO [onboard_port_vlan_012.py:94] cmd:ifconfig enp125s0f0v0 | grep 'RX packets', result:RX packets 1017 bytes 855241 (855.2 KB)
2022-01-17 03:50:17,538 INFO [mz.py:87] mz enp125s0f0 -t tcp sp=5001,dp=5003 -b ee:f7:d9:13:55:6a -p 800 -c 1000 -Q 4094
2022-01-17 03:50:18,320 INFO [mz.py:90]
2022-01-17 03:50:18,325 INFO [onboard_port_vlan_012.py:97] cmd:ifconfig enp125s0f0v0 | grep 'RX packets', result:RX packets 1017 bytes 855241 (855.2 KB)
2022-01-17 03:50:18,326 INFO [onboard_port_vlan_012.py:99] mz test pass
2022-01-17 03:50:18,331 INFO [comm_sys_base_info.py:221] cmd:cat /etc/os-release | grep -w ID | awk -F'=' '{print $2}' | tr '[:upper:]' '[:lower:]', result:ubuntu
2022-01-17 03:50:18,374 INFO [comm_sys_base_info.py:221] cmd:dmesg | grep -vE \"Hardware Error|-16|warning: setlocale|unknown memory\" | grep -iE \"error|fail|call trace\", result:[ 4081.114536] hns3 0000:7d:00.0: Add port vlan failed, vport 1 is already in vlan 1
[ 4081.122015] hns3 0000:7d:00.0: PF failed(-22) to config VF's VLAN
[ 4081.267084] hns3 0000:7d:00.0: Add port vlan failed, vport 1 is already in vlan 2
[ 4081.274559] hns3 0000:7d:00.0: PF failed(-22) to config VF's VLAN
[ 4081.418606] hns3 0000:7d:00.0: Add port vlan failed, vport 1 is already in vlan 3
[ 4081.426079] hns3 0000:7d:00.0: PF failed(-22) to config VF's VLAN
[ 4081.571115] hns3 0000:7d:00.0: Add port vlan failed, vport 1 is already in vlan 4
................
[ 4301.188929] hns3 0000:7d:00.0: PF failed(-22) to config VF's VLAN
[ 4301.333045] hns3 0000:7d:00.0: Add port vlan failed, vport 1 is already in vlan 1531
[ 4301.340777] hns3 0000:7d:00.0: PF failed(-22) to config VF's VLAN
[ 4301.752910] hns3 0000:7d:00.0: Add port vlan failed, vport 1 is already in vlan 1534
[ 4301.760643] hns3 0000:7d:00.0: PF failed(-22) to config VF's VLAN
[ 4301.905169] hns3 0000:7d:00.0: Add port vlan failed, vport 1 is already in vlan 1535
[ 4301.912902] hns3 0000:7d:00.0: PF failed(-22) to config VF's VLAN
2022-01-17 03:50:18,381 ERROR [comm_sys_base_info.py:450] check dmesg fail
2022-01-17 03:50:18,423 INFO [conftest.py:196] case: test_onboard_port_vlan_012 --- result: failed
[Additional information]
(Firmware version, kernel version, affected hardware, etc. if required2022011908212):

[Resolution]
commit 132023de7149e7dde4b457328cb233dc58561b54
Author: Jian Shen <email address hidden>
Date: Mon May 31 10:38:40 2021 +0800

    net: hns3: remove unnecessary updating port based VLAN

    For the PF have called hclge_update_port_base_vlan_cfg() already
    before notify VF, it's unnecessary to update port based VLAN again
    when received mailbox request from VF.

    Signed-off-by: Jian Shen <email address hidden>
    Signed-off-by: Huazhong Tan <email address hidden>
    Signed-off-by: Jakub Kicinski <email address hidden>

diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mbx.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mbx.c
index 16b42cee778e..3f7d1f2cbe2d 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mbx.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mbx.c
@@ -360,15 +360,6 @@ static int hclge_set_vf_vlan_cfg(struct hclge_vport *vport,
                bool en = msg_cmd->is_kill ? true : false;

                status = hclge_en_hw_strip_rxvtag(handle, en);
- } else if (msg_cmd->subcode == HCLGE_MBX_PORT_BASE_VLAN_CFG) {
- struct hclge_vlan_info *vlan_info;
- u16 *state;
-
- state = (u16 *)&mbx_req->msg.data[HCLGE_MBX_VLAN_STATE_OFFSET];
- vlan_info = (struct hclge_vlan_info *)
- &mbx_req->msg.data[HCLGE_MBX_VLAN_INFO_OFFSET];
- status = hclge_update_port_base_vlan_cfg(vport, *state,
- vlan_info);
        } else if (msg_cmd->subcode == HCLGE_MBX_GET_PORT_BASE_VLAN_STATE) {
                struct hnae3_ae_dev *ae_dev = pci_get_drvdata(vport->nic.pdev);
                /* vf does not need to know about the port based VLAN state

https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/commit/drivers/net/ethernet/hisilicon?id=132023de7149e7dde4b457328cb233dc58561b54

[Ike Panhc (ikepanhc)]

Patch 132023de71 ("net: hns3: remove unnecessary updating port based VLAN") hits mainline kernel since v5.14 and not yet merged into focal GA or HWE kernel.

Try to cherry-pick and can be clean cherry-pick to focal HWE (5.13) but not GA (5.4). Next step is to reproduce the failure.

[Ike Panhc (ikepanhc)]

In commit 92f11ea177cd ("net: hns3: fix set port based VLAN issue for VF"), it introduce the snippet for HCLGE_MBX_PORT_BASE_VLAN_CFG[1] and is also included in 5.4 kernel. Is that necessary to remove them from 5.4 kernel?

[1] git show 92f11ea177cd
@@ -310,11 +326,22 @@ static int hclge_set_vf_vlan_cfg(struct hclge_vport *vport,
                bool en = mbx_req->msg[2] ? true : false;

                status = hclge_en_hw_strip_rxvtag(handle, en);
+ } else if (mbx_req->msg[1] == HCLGE_MBX_PORT_BASE_VLAN_CFG) {
+ struct hclge_vlan_info *vlan_info;
+ u16 *state;
+
+ state = (u16 *)&mbx_req->msg[2];
+ vlan_info = (struct hclge_vlan_info *)&mbx_req->msg[4];
+ status = hclge_update_port_base_vlan_cfg(vport, *state,
+ vlan_info);
+ } else if (mbx_req->msg[1] == HCLGE_MBX_GET_PORT_BASE_VLAN_STATE) {
+ u8 state;
+
+ state = vport->port_base_vlan_cfg.state;
+ status = hclge_gen_resp_to_vf(vport, mbx_req, 0, &state,
+ sizeof(u8));
        }

- if (gen_resp)
- status = hclge_gen_resp_to_vf(vport, mbx_req, status, NULL, 0);
-
        return status;
 }

[Ike Panhc (ikepanhc)]

I will try to reproduce this issue and see if we need backport.

[Ike Panhc (ikepanhc)]

Fix already applied to Ubuntu 5.15 kernel and I can not reproduce on Ubuntu 5.4 kernel.