Comment 1 for bug 1761327

Thanks for the repro steps, to add to that is only triggers if the HW REALLY can do HW Timestamping (which likely is the reason it was missed as e.g. virtio devs can't).
You can check with:
  ethtool -T <dev>

From man chrony.conf
This directive is supported on Linux 3.19 and newer. The NIC must support HW timestamping, which can be verified with the ethtool -T command. The list of capabilities should include
SOF_TIMESTAMPING_RAW_HARDWARE, SOF_TIMESTAMPING_TX_HARDWARE, and SOF_TIMESTAMPING_RX_HARDWARE. Receive filter HWTSTAMP_FILTER_ALL, or HWTSTAMP_FILTER_NTP_ALL, is necessary for timestamping of received packets. Timestamping of packets received from bridged and bonded interfaces is supported on Linux 4.13 and newer. When chronyd is running, no other process (e.g. a PTP daemon) should be working with the NIC clock.

In a KVM guest it will run with SW timestamping which works.
From chronc > ntpdata
[...]
TX timestamping : Kernel
RX timestamping : Kernel