Debian: Kernel's eBPF test cases fail due to a behaviour change/bug in "ping"

Bug #1977849 reported by M. Vefa Bicakci
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
StarlingX
Fix Released
Low
M. Vefa Bicakci

Bug Description

Brief Description
-----------------

The version of ping in Debian 11/Bullseye does not print information about ICMP replies originating from a host other than the one being pinged.

In other words, if a user runs "ping 192.168.1.1" and the responses come from "192.168.1.3", then the version of ping in Debian does not report these responses. This in turn causes the kernel's eBPF test cases to fail. To be specific, the "test_lwt_bpf.sh" test case fails, because this test case uses eBPF to rewrite the destination or source of ICMP packets.

Further research showed that this is (or was) a known issue in ping. The bug is introduced by this commit in the upstream iputils repository:
  https://github.com/iputils/iputils/commit/5e052ada96c3

And the bug is fixed by this commit:
  https://github.com/iputils/iputils/commit/5f6bec5ab57c

Severity
--------
Minor: The only known effect is the failure of eBPF test cases, at least for now.

Steps to Reproduce
------------------
The steps to reproduce this issue are unfortunately very complex, so I will not be able to document them here.

Expected Behavior
------------------
test_lwt_bpf.sh test case passes.

Actual Behavior
----------------
test_lwt_bpf.sh test case fails due to a behaviour change in ping, compared to CentOS 7's version of ping.

Reproducibility
---------------
100%

System Configuration
--------------------
Not applicable, but this issue was reproduced in a VM running Debian-based StarlingX in All-in-One Simplex mode with a standard kernel (i.e., non-low-latency).

Branch/Pull Time/Commit
-----------------------
Day 1 issue with Debian-based StarlingX.

Last Pass
---------
CentOS 7-based StarlingX does not have this issue.

Timestamp/Logs
--------------
(I do not have any failure logs available, unfortunately.)

Test Activity
-------------
Feature and Regression Testing

Workaround
----------
None.

Changed in starlingx:
assignee: nobody → M. Vefa Bicakci (vbicakci)
status: New → Confirmed
status: Confirmed → New
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to integ (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/integ/+/844976

Changed in starlingx:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tools (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/tools/+/844977

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to integ (master)

Reviewed: https://review.opendev.org/c/starlingx/integ/+/844976
Committed: https://opendev.org/starlingx/integ/commit/829e0c21e7bfa66fb6c5266a0e425a39c409eae1
Submitter: "Zuul (22348)"
Branch: master

commit 829e0c21e7bfa66fb6c5266a0e425a39c409eae1
Author: M. Vefa Bicakci <email address hidden>
Date: Mon May 30 19:49:32 2022 +0000

    Debian: Patch ping for replies with wrong source

    This commit patches iputils-ping in Debian-based StarlingX so that ping
    prints out ICMP echo replies with unexpected/"wrong" source IP addresses
    as was done with CentOS 7's version of ping and as is done with more
    recent versions of the iputils package. As an example, if the user calls
    "ping 192.168.1.1" and the ICMP echo responses are sent by 192.168.1.3,
    then the version of ping in Debian 11 currently assumes that no
    responses have been received. This issue was discovered while running
    the eBPF test cases in the kernel's samples/bpf source sub-directory.

    This issue was introduced upstream with the following commit:
      5e052ada96c3 ("ping: discard packets with wrong source address")
      https://github.com/iputils/iputils/commit/5e052ada96c3

    The following commit fixes this issue:
      5f6bec5ab57c ("ping: Print reply with wrong source with warning")
      https://github.com/iputils/iputils/commit/5f6bec5ab57c

    Due to a number of contextual dependencies, we unfortunately cannot
    directly cherry-pick the commit that fixes this issue. Hence, we
    cherry-pick four patches in total. We did not opt for completely
    uprevisioning the iputils package to be able to inherit bug fixes that
    Debian may introduce in the future.

    We checked the list of commits between the two tags 20210202 and
    20211215 using the following command:

      git log --oneline refs/tags/20210202..origin/master -- 'ping/*.[ch]'

    And we did not find any commits that fix issues in the commits that we
    are cherry-picking with this commit.

    Testing:
    - An ISO image with the standard kernel was successfully incrementally
      built with this commit.
    - The resulting ISO image was installed onto a qemu/KVM-based virtual
      machine and the resulting system was bootstrapped with Ansible and
      unlocked.
    - The test cases shipped in the kernel repository's samples/bpf
      sub-directory were successfully executed in the virtual machine. These
      test cases make use of ping in IPv4 and IPv6 mode, which ought to be a
      good sanity test.

    Partial-Bug: 1977849

    Change-Id: I25b42b8de80deec86be0eb504442992863a8d833
    Signed-off-by: M. Vefa Bicakci <email address hidden>

Changed in starlingx:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tools (master)

Reviewed: https://review.opendev.org/c/starlingx/tools/+/844977
Committed: https://opendev.org/starlingx/tools/commit/1a90124291c58aeae520bd974464235494033763
Submitter: "Zuul (22348)"
Branch: master

commit 1a90124291c58aeae520bd974464235494033763
Author: M. Vefa Bicakci <email address hidden>
Date: Mon May 30 20:01:58 2022 +0000

    Debian: Use patched/downstream version of iputils

    This commit instructs the build system to make use of the iputils
    packages built from source code as part of the StarlingX build instead
    of using the upstream (Debian) versions.

    iputils is patched with patches acquired from upstream to let ping
    report ICMP echo replies originate from a host other than the one
    specified to ping on its command line.

    Verification:
    - An ISO image with the standard kernel was successfully built in an
      incremental manner.
    - The resulting ISO image was installed onto a qemu/KVM-based virtual
      machine.
    - The existence of the "ping", "arping" and "tracepath" utilities in the
      installed system was confirmed.

    Closes-Bug: 1977849
    Depends-On: https://review.opendev.org/c/starlingx/integ/+/844976

    Change-Id: I3d79b07ce8f23c26d2ba1faa5724592a82f659c1
    Signed-off-by: M. Vefa Bicakci <email address hidden>

Ghada Khalil (gkhalil)
Changed in starlingx:
importance: Undecided → Low
tags: added: stx.7.0 stx.debian stx.distro.other
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.