Comment 18 for bug 1692538

Hi Siva,

Thank you for your quick and thoughtful response.

I will ask about the default MTU for the veth interface to see if the user increased it themselves.

I'm not sure I completely understand what you mean about largesend offload being disabled after retransmits. I'm also not completely sure if it's largesend offload or just large packets that are causing issues. If I have understood correctly (e.g. https://www.ibm.com/support/knowledgecenter/en/ssw_aix_72/com.ibm.aix.performance/tcp_large_send_offload.htm) large-send offload is what Linux would call TCP Segmentation Offload (TSO) - does that match your understanding?

Here's my concern. The code I'm looking at (let's look at Zesty, so v4.10) is in ibmveth.c, ibmveth_poll(). There we see:

 if (length > netdev->mtu + ETH_HLEN) {
 ibmveth_rx_mss_helper(skb, mss, lrg_pkt);
 adapter->rx_large_packets++;
 }

Then ibmveth_rx_mss_helper() has the following - setting GSO on regardless of the large_pkt bit:

 /* if mss is not set through Large Packet bit/mss in rx buffer,
  * expect that the mss will be written to the tcp header checksum.
  */
 tcph = (struct tcphdr *)(skb->data + offset);
 if (lrg_pkt) {
 skb_shinfo(skb)->gso_size = mss;
 } else if (offset) {
 skb_shinfo(skb)->gso_size = ntohs(tcph->check);
 tcph->check = 0;
 }

It looks to me that Linux will interpret a packet from the veth adaptor as a GSO/GRO packet based only on whether or not the size of the received packet is greater than the linux-side MTU plus the header size - not based on whether AIX thinks it is transmitting a LSO packet.

To put it another way - if I have understood correctly - there are two ways we could end up with a GSO/GRO packet coming out of a veth adaptor. The ibmveth_rx_mss_helper path is taken when the size of the packet is greater than MTU+ETH_HLEN, which can happen when:

 1) The AIX end has turned on LSO, so the large_packet bit is set
 2) Large-send is off in AIX but there is a mis-matched MTU between AIX and Linux

In the first case case, you say that AIX will turn off largesend, which will fix the issue. But in the second case, if I have understood correctly, AIX will not be able to do anything. Unless you are saying that AIX will dynamically reduce the MTU for a connection in the presence of a number of re-transmits?

This isn't necessarily wrong behaviour from AIX - Linux can't do anything in this situation either; a 'hop' that can participate in Path MTU Discovery would be needed.

If I understand it, then, the optimal configuration would be for the AIX LPAR to set an MTU of 1500/9000 and turn on LSO for veth on the AIX side - does that sound right?

Thanks again!
Regards,
Daniel