Comment 11 for bug 1788006

> "the instance has a pending task (spawning). Skip".

That's unrelated from a periodic task that runs in nova-compute to look for discrepancies between what the hypervisor reports as guests running on it vs what's in the nova database. As noted in the log message, it skips instances that have a task_state set (spawning in this case).

--

Looking at the original failed job link, nova-compute starts waiting for the network-vif-plugged event here:

http://logs.openstack.org/74/591074/4/check/neutron-tempest-plugin-designate-scenario/d02f171/controller/logs/screen-n-cpu.txt.gz#_Aug_20_12_52_04_659588

Aug 20 12:52:04.659588 ubuntu-xenial-ovh-gra1-0001411231 nova-compute[27062]: DEBUG nova.compute.manager [None req-e3fd2700-621a-4a80-b16b-03d6018efbf2 tempest-DNSIntegrationTests-199337017 tempest-DNSIntegrationTests-199337017] [instance: 9797a641-8925-4f74-abbf-c6111d47f91d] Preparing to wait for external event network-vif-plugged-e14bdbb1-ca9a-44b8-9f3b-a296f01a71fd {{(pid=27062) prepare_for_instance_event /opt/stack/nova/nova/compute/manager.py:328}}

The libvirt driver then plugs the vif on the host:

vif={"profile": {}, "ovs_interfaceid": "e14bdbb1-ca9a-44b8-9f3b-a296f01a71fd", "preserve_on_delete": true, "network": {"bridge": "br-int", "subnets": [{"ips": [{"meta": {}, "version": 4, "type": "fixed", "floating_ips": [], "address": "10.1.0.6"}], "version": 4, "meta": {"dhcp_server": "10.1.0.2"}, "dns": [], "routes": [], "cidr": "10.1.0.0/28", "gateway": {"meta": {}, "version": 4, "type": "gateway", "address": "10.1.0.1"}}], "meta": {"injected": false, "tunneled": true, "tenant_id": "df130fc6f8a64ab1b877c39e32e8a1d6", "physical_network": null, "mtu": 1400}, "id": "f7495a16-b458-49e3-b219-b5baa539df75", "label": "tempest-test-network--304750879"}, "devname": "tape14bdbb1-ca", "vnic_type": "normal", "qbh_params": null, "meta": {}, "details": {"port_filter": true, "datapath_type": "system", "ovs_hybrid_plug": false}, "address": "fa:16:3e:7a:37:d1", "active": false, "type": "ovs", "id": "e14bdbb1-ca9a-44b8-9f3b-a296f01a71fd", "qbg_params": null}

n-cpu receives the network-vif-plugged event here:

http://logs.openstack.org/74/591074/4/check/neutron-tempest-plugin-designate-scenario/d02f171/controller/logs/screen-n-cpu.txt.gz#_Aug_20_12_52_07_626394

Aug 20 12:52:07.626394 ubuntu-xenial-ovh-gra1-0001411231 nova-compute[27062]: DEBUG nova.compute.manager [None req-633098b6-6b29-473b-a7d1-76fbf8648e7b service nova] [instance: 9797a641-8925-4f74-abbf-c6111d47f91d] Received event network-vif-plugged-e14bdbb1-ca9a-44b8-9f3b-a296f01a71fd {{(pid=27062) external_instance_event /opt/stack/nova/nova/compute/manager.py:8077}}

And then after that yeah I don't see nova saying the VM is active...I wonder if we're dropping the event somehow with an eventlet greenthread switch or something - there were some recent eventlet updates in upper-constraints, but I wouldn't think those are also on stable branches.

Do we know when this started occurring in the DNS job?