Ignoring the default NAT when using the virtio adapter
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
libvirt (Ubuntu) |
Expired
|
Undecided
|
Unassigned |
Bug Description
I am using libvirt with KVM on a UBUNTU 18.04.3 LTS
I have the following topology inside libvirt/KVM
The default virtual network with IP range 10.0.0.0/24 on virbr0 interface with IP 10.0.0.1 and DHCP enabled
Another virtual network named "confinada" with IP range 192.168.254.0/24 on virbr1 interface with no IP and DHCP disabled
I have one VM with pfSense that is connected to the two networks and is acting as a gateway to the others VMs.
I have one VM with UBUNTU 18.04 and one with Windows 7
The topology is as following:
INTERNET
|
| 192.168.11.201
HOST
| 10.0.0.1
(default)|
| 10.0.0.138
pfSense
| 192.168.254.1
+-
| (confinada) |
| 192.168.254.2 | 192.168.254.103
UBUNTU Windows 7
I have sucess accessing Internet on the Windows 7 VM but not on the UBUNTU machine.
During debug i found that the packets from the UBUNTU machine are not being NAT'ed correctly when leaving the host machine.
I compared the two VM and found that the UBUNTU VM is using device model "virtio" and the Windows VM is using "rtl8139".
When i changed the device model of the UBUNTU VM to "rtl8139" it start accessing the Internet.
The pfSense VM is using the device model "virtio" on both interfaces.
I tried to acess www.google.com (172.217.28.68) on the VM and used tcpdump on the host interface.
I have the following results:
1 - Using "virtio" device model:
# tcpdump -nN -i enp2s0 host 172.217.28.68
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on enp2s0, link-type EN10MB (Ethernet), capture size 262144 bytes
12:16:02.347830 IP 10.0.0.138.47764 > 172.217.28.68.80: Flags [S], seq 2073890688, win 29200, options [mss 1460,sackOK,TS val 3101189393 ecr 0,nop,wscale 7], length 0
12:16:03.359092 IP 10.0.0.138.47764 > 172.217.28.68.80: Flags [S], seq 2073890688, win 29200, options [mss 1460,sackOK,TS val 3101190405 ecr 0,nop,wscale 7], length 0
12:16:05.375124 IP 10.0.0.138.47764 > 172.217.28.68.80: Flags [S], seq 2073890688, win 29200, options [mss 1460,sackOK,TS val 3101192421 ecr 0,nop,wscale 7], length 0
12:16:09.631218 IP 10.0.0.138.47764 > 172.217.28.68.80: Flags [S], seq 2073890688, win 29200, options [mss 1460,sackOK,TS val 3101196677 ecr 0,nop,wscale 7], length 0
2 - Using "rtl8139" device model:
# tcpdump -nN -i enp2s0 host 172.217.28.68
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on enp2s0, link-type EN10MB (Ethernet), capture size 262144 bytes
12:17:15.206181 IP 192.168.11.201.6085 > 172.217.28.68.80: Flags [S], seq 733025182, win 29200, options [mss 1460,sackOK,TS val 550129346 ecr 0,nop,wscale 7], length 0
12:17:15.226156 IP 172.217.28.68.80 > 192.168.
12:17:15.227137 IP 192.168.11.201.6085 > 172.217.28.68.80: Flags [.], ack 1, win 229, options [nop,nop,TS val 550129367 ecr 1081017811], length 0
12:17:15.228442 IP 192.168.11.201.6085 > 172.217.28.68.80: Flags [P.], seq 1:142, ack 1, win 229, options [nop,nop,TS val 550129368 ecr 1081017811], length 141: HTTP: GET / HTTP/1.1
VIRTUAL NETWORK INFORMATION
virsh # net-dumpxml default
<network connections='1'>
<name>
<uuid>
<forward dev='enp2s0' mode='nat'>
<nat>
<port start='1024' end='65535'/>
</nat>
<interface dev='enp2s0'/>
</forward>
<bridge name='virbr0' stp='on' delay='0'/>
<mac address=
<domain name='default'/>
<ip address='10.0.0.1' netmask=
<dhcp>
<range start='10.0.0.128' end='10.0.0.254'/>
</dhcp>
</ip>
</network>
virsh #
virsh # net-dumpxml confinada
<network connections='2'>
<name>
<uuid>
<bridge name='virbr1' stp='off' delay='0'/>
<mac address=
<domain name='confinada'/>
</network>
virsh #
HOST SYSTEM INFORMATION:
root@jlbastos-
Description: Ubuntu 18.04.3 LTS
Release: 18.04
root@jlbastos-
root@jlbastos-
libvirt-bin:
Instalado: 4.0.0-1ubuntu8.13
Candidato: 4.0.0-1ubuntu8.13
Tabela de versão:
*** 4.0.0-1ubuntu8.13 500
500 http://
100 /var/lib/
4.
500 http://
4.0.0-1ubuntu8 500
500 http://
libvirt-clients:
Instalado: 4.0.0-1ubuntu8.13
Candidato: 4.0.0-1ubuntu8.13
Tabela de versão:
*** 4.0.0-1ubuntu8.13 500
500 http://
100 /var/lib/
4.
500 http://
4.0.0-1ubuntu8 500
500 http://
libvirt-daemon:
Instalado: 4.0.0-1ubuntu8.13
Candidato: 4.0.0-1ubuntu8.13
Tabela de versão:
*** 4.0.0-1ubuntu8.13 500
500 http://
100 /var/lib/
4.
500 http://
4.0.0-1ubuntu8 500
500 http://
libvirt0:
Instalado: 4.0.0-1ubuntu8.13
Candidato: 4.0.0-1ubuntu8.13
Tabela de versão:
*** 4.0.0-1ubuntu8.13 500
500 http://
100 /var/lib/
4.
500 http://
4.0.0-1ubuntu8 500
500 http://
qemu-kvm:
Instalado: 1:2.11+
Candidato: 1:2.11+
Tabela de versão:
*** 1:2.11+
500 http://
500 http://
100 /var/lib/
1:
500 http://
root@jlbastos-
description: | updated |
Hi,
interesting but nothing obvious comes directly to my mind.
It is great that you identified that the device driver virtio/rtl8139 makes a difference.
Lets start there to think about it.
Traffic for rtl8139 will always go to host-userspace for device emulation and flow on from there.
With virtio in comparison I think the default if available will be vhost-net which will handle things in the host kernel and should not (tm) but might behave differently.
While vhost-net is usually better for efficiency and performance lets check if that makes the difference.
Please set back the Ubuntu VM to virtio but then in addition to
<model type="virtio"/>
also set
<driver name="qemu"/>
The default is "try vhost but if failing (e.g. module not present) fall back to qemu"; but setting qemu explicitly there will force it to use the userspace backend.
Please let me know how virtio behaves in that case.
If we can move this from rtl8139-vs-virtio to virtio with/without vhost that already would be a good step to get closer to the root cause.