> Do you have the full ovs-agent log? I say that because the one attached is very limited.
Atached new logs neutron-openvswitch-agent-full.log (search 'is not a positive integer' for ~ timestamp)
> I have one more question: why are you stopping the VM manually? Instead of using the OpenStack commands: server stop, server start, etc.
Actually I stop the VM using openstack CLI. In the issue description I am stopping it manually to show reproduction by steps (to elliminate race between libvirt and rcp_loop calls).
I wrote a small bash script using witch I managed to get reproduction.
So, I have two VMs:
[root@sqvm2-0506 ~]# openstack server list
+--------------------------------------+------+--------+---------------------+-------+--------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+------+--------+---------------------+-------+--------+
| 968eb966-77d7-4668-a2bf-226284661ee8 | vm-2 | ACTIVE | net1=192.168.128.18 | | tiny |
| ef1afb42-ab8f-448b-876c-b1705e718ec5 | vm-1 | ACTIVE | net1=192.168.128.13 | | tiny |
+--------------------------------------+------+--------+---------------------+-------+--------+
[root@sqvm2-0506 ~]#
[root@sqvm2-0506 ~]# openstack port list | grep 192.168.128.13
| fce15578-ba1c-4e96-a8ba-1612b389aada | | fa:16:3e:d5:10:62 | ip_address='192.168.128.13', subnet_id='868b4265-f7b9-4bb0-845c-6a05a56a4eaa' | ACTIVE |
[root@sqvm2-0506 ~]#
while :; do
echo "Step N=${step}" && ((step++)) || true
openstack --insecure server add network vm-2 net2 1>/dev/null &
pid=$!
openstack --insecure server stop vm-1 1>/dev/null
wait "${pid}"
sleep 30 # give time to VM stopping and rpc_loop ports processing
ovs-ofctl dump-flows br-int | grep fa:16:3e:d5:10:62 && { echo "Got it, exit"; exit 1; }
openstack --insecure server remove network vm-2 net2 1>/dev/null
openstack --insecure server start vm-1 1>/dev/null
sleep 30 # give time to VM start
done
[root@sqvm2-0506 ~]#
Hello Rodolfo
> Do you have the full ovs-agent log? I say that because the one attached is very limited. openvswitch- agent-full. log (search 'is not a positive integer' for ~ timestamp)
Atached new logs neutron-
> I have one more question: why are you stopping the VM manually? Instead of using the OpenStack commands: server stop, server start, etc.
Actually I stop the VM using openstack CLI. In the issue description I am stopping it manually to show reproduction by steps (to elliminate race between libvirt and rcp_loop calls).
I wrote a small bash script using witch I managed to get reproduction.
So, I have two VMs:
[root@sqvm2-0506 ~]# openstack server list ------- ------- ------- ------- ----+-- ----+-- ------+ ------- ------- ------- +------ -+----- ---+ ------- ------- ------- ------- ----+-- ----+-- ------+ ------- ------- ------- +------ -+----- ---+ 77d7-4668- a2bf-226284661e e8 | vm-2 | ACTIVE | net1=192.168.128.18 | | tiny | ab8f-448b- 876c-b1705e718e c5 | vm-1 | ACTIVE | net1=192.168.128.13 | | tiny | ------- ------- ------- ------- ----+-- ----+-- ------+ ------- ------- ------- +------ -+----- ---+
+------
| ID | Name | Status | Networks | Image | Flavor |
+------
| 968eb966-
| ef1afb42-
+------
[root@sqvm2-0506 ~]#
[root@sqvm2-0506 ~]# openstack port list | grep 192.168.128.13 ba1c-4e96- a8ba-1612b389aa da | | fa:16:3e:d5:10:62 | ip_address= '192.168. 128.13' , subnet_ id='868b4265- f7b9-4bb0- 845c-6a05a56a4e aa' | ACTIVE |
| fce15578-
[root@sqvm2-0506 ~]#
Mac 'fa:16:3e:d5:10:62' is the address of vm-1
[root@sqvm2-0506 ~]# cat test.sh
#!/usr/bin/env bash
set -e
step=0
while :; do
echo "Step N=${step}" && ((step++)) || true
openstack --insecure server add network vm-2 net2 1>/dev/null &
pid=$!
openstack --insecure server stop vm-1 1>/dev/null
wait "${pid}"
sleep 30 # give time to VM stopping and rpc_loop ports processing
ovs-ofctl dump-flows br-int | grep fa:16:3e:d5:10:62 && { echo "Got it, exit"; exit 1; }
openstack --insecure server remove network vm-2 net2 1>/dev/null
openstack --insecure server start vm-1 1>/dev/null
sleep 30 # give time to VM start
done
[root@sqvm2-0506 ~]#
[root@sqvm2-0506 ~]# /bin/bash test.sh 0x9e2f137c9b53b 412, duration=239.247s, table=60, n_packets=67, n_bytes=7398, idle_age=329, priority= 90,dl_vlan= 1,dl_dst= fa:16:3e: d5:10:62 actions= load:0x24- >NXM_NX_ REG5[], load:0x1- >NXM_NX_ REG6[], strip_vlan, resubmit( ,81) 0x9e2f137c9b53b 412, duration=239.246s, table=73, n_packets=0, n_bytes=0, idle_age=357, priority= 100,reg6= 0x1,dl_ dst=fa: 16:3e:d5: 10:62 actions= load:0x24- >NXM_NX_ REG5[], resubmit( ,81) ------- ------- ------- ------- ----+-- ----+-- ------- +------ ------- ------- ------- ------- ------+ ------- +------ --+ ------- ------- ------- ------- ----+-- ----+-- ------- +------ ------- ------- ------- ------- ------+ ------- +------ --+ 77d7-4668- a2bf-226284661e e8 | vm-2 | ACTIVE | net2=192.168.5.15; net1=192.168.128.18 | | tiny | ab8f-448b- 876c-b1705e718e c5 | vm-1 | SHUTOFF | net1=192.168.128.13 | | tiny | ------- ------- ------- ------- ----+-- ----+-- ------- +------ ------- ------- ------- ------- ------+ ------- +------ --+ 0x9e2f137c9b53b 412, duration=584.966s, table=60, n_packets=67, n_bytes=7398, idle_age=675, priority= 90,dl_vlan= 1,dl_dst= fa:16:3e: d5:10:62 actions= load:0x24- >NXM_NX_ REG5[], load:0x1- >NXM_NX_ REG6[], strip_vlan, resubmit( ,81) 0x9e2f137c9b53b 412, duration=584.965s, table=73, n_packets=0, n_bytes=0, idle_age=702, priority= 100,reg6= 0x1,dl_ dst=fa: 16:3e:d5: 10:62 actions= load:0x24- >NXM_NX_ REG5[], resubmit( ,81)
Step N=0
cookie=
cookie=
Got it, exit
[root@sqvm2-0506 ~]#
[root@sqvm2-0506 ~]# openstack server list
+------
| ID | Name | Status | Networks | Image | Flavor |
+------
| 968eb966-
| ef1afb42-
+------
[root@sqvm2-0506 ~]#
[root@sqvm2-0506 ~]# ovs-ofctl dump-flows br-int | grep fa:16:3e:d5:10:62
cookie=
cookie=
[root@sqvm2-0506 ~]#
Looks like I was lucky to get it on the first iter.