TPACKET_V3 in Linux before 3.19 causes packet loss in libpcap

[Daniel Miller]

Observed packet loss and major slowdown in Nmap on Ubuntu 14.04: https://github.com/nmap/nmap/issues/34

Traced to this bug report for libpcap 1.5.3: https://github.com/the-tcpdump-group/libpcap/issues/380

Underlying issue is a bug in the Linux kernel which was fixed in 3.19: https://github.com/torvalds/linux/commit/da413ee

Please backport this patch to Ubuntu 14.04 LTS kernel.

ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: linux-image-3.13.0-53-generic 3.13.0-53.89
ProcVersionSignature: Ubuntu 3.13.0-53.89-generic 3.13.11-ckt19
Uname: Linux 3.13.0-53-generic x86_64
ApportVersion: 2.14.1-0ubuntu3.10
Architecture: amd64
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC0: nmap 12401 F.... pulseaudio
 /dev/snd/pcmC0D0c: nmap 12401 F...m pulseaudio
 /dev/snd/pcmC0D0p: nmap 12401 F...m pulseaudio
 /dev/snd/timer: nmap 12401 f.... pulseaudio
CurrentDmesg: [ 25.541696] init: plymouth-upstart-bridge main process ended, respawning
Date: Thu May 21 07:42:08 2015
HibernationDevice: RESUME=UUID=e4b6770e-dcc5-445d-a445-25d9a244e204
InstallationDate: Installed on 2015-01-01 (139 days ago)
InstallationMedia: Ubuntu 14.04.1 LTS "Trusty Tahr" - Release amd64 (20140722.2)
IwConfig:
 eth0 no wireless extensions.

 lo no wireless extensions.
Lsusb:
 Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
 Bus 002 Device 002: ID 80ee:0021 VirtualBox USB Tablet
 Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
MachineType: innotek GmbH VirtualBox
ProcEnviron:
 TERM=linux
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcFB: 0 VESA VGA
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-3.13.0-53-generic root=UUID=032922f2-3fe1-4228-8c93-8c1e9afc4289 ro quiet splash vt.handoff=7
RelatedPackageVersions:
 linux-restricted-modules-3.13.0-53-generic N/A
 linux-backports-modules-3.13.0-53-generic N/A
 linux-firmware 1.127.12
RfKill:

SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 12/01/2006
dmi.bios.vendor: innotek GmbH
dmi.bios.version: VirtualBox
dmi.board.name: VirtualBox
dmi.board.vendor: Oracle Corporation
dmi.board.version: 1.2
dmi.chassis.type: 1
dmi.chassis.vendor: Oracle Corporation
dmi.modalias: dmi:bvninnotekGmbH:bvrVirtualBox:bd12/01/2006:svninnotekGmbH:pnVirtualBox:pvr1.2:rvnOracleCorporation:rnVirtualBox:rvr1.2:cvnOracleCorporation:ct1:cvr:
dmi.product.name: VirtualBox
dmi.product.version: 1.2
dmi.sys.vendor: innotek GmbH

[Brad Figg (brad-figg)]

This change was made by a bot.

[Mike Pontillo (mpontillo)]

I have tested this on kernel 4.4.0 (in Xenial) and I don't think the fix that went into 3.19 fixed the entire issue.

My understanding is that since nmap uses a relatively long pcap_select() timeout, and the packet buffers in the kernel aren't filling up fast enough, the packet loss can still be seen.

I see that you have worked around this in nmap by disabling TPACKET_v3:

https://github.com/nmap/nmap/commit/83b9c4c939094e673705b7c270533708853891f0

I noticed that another project (which I found linked to the libpcap bug) worked around this issue by using libpcap's immediate_mode:

https://github.com/teclo/flow-disruptor/commit/3376d049fc5280aedccc42fc6e090c32c0c4a128

Would you consider trying this same workaround in nmap, since it seems more general than forcing TPACKET_V3 to be disabled? (That is, only users who compile nmap from source and use "./configure --with-libpcap=included" can currently use the workaround.)

[Mike Pontillo (mpontillo)]

Inspired by the flow-disruptor workaround, I did a proof-of-concept nmap workaround as follows:

http://paste.ubuntu.com/23176851/

Obviously, this is nowhere near production-ready code, but I wanted to convince myself that the pcap_set_immediate_mode() workaround could work in nmap as well.