test_reboot_server_hard failure in grenade job

[YAMAMOTO Takashi]

eg.
http://logs.openstack.org/13/551013/1/check/networking-midonet-grenade-ml2/af704de/logs/
http://logs.openstack.org/19/551419/1/check/networking-midonet-grenade-ml2/86f5bd2/logs/

[YAMAMOTO Takashi (yamamoto)]

VirtualInterfaceCreateException caused by vif-plugged timeout it seems.

eg. http://logs.openstack.org/69/553069/2/check/networking-midonet-grenade-ml2/0c364ef/logs/screen-n-cpu.txt.gz#_Mar_19_03_00_14_585858

Mar 19 03:00:14.585858 ubuntu-xenial-ovh-bhs1-0003046183 nova-compute[17330]: WARNING nova.virt.libvirt.driver [None req-da6d9470-be12-426c-aec8-25f121481e95 service nova] [instance: 61309612-e8e7-4693-9d1d-b442e7a46271] Timeout waiting for [('network-vif-plugged', u'e7ddb107-2a4f-4bb5-84e1-d95da46415ae')] for instance with vm_state active and task_state reboot_started_hard.: Timeout: 300 seconds
Mar 19 03:00:14.815753 ubuntu-xenial-ovh-bhs1-0003046183 nova-compute[17330]: DEBUG nova.virt.libvirt.vif [None req-da6d9470-be12-426c-aec8-25f121481e95 service nova] vif_type=midonet instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2018-03-19T02:54:37Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-2512996',display_name='tempest-ServerActionsTestJSON-server-2512996',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),host='ubuntu-xenial-ovh-bhs1-0003046183',hostname='tempest-serveractionstestjson-server-2512996',id=2,image_ref='06d1e26e-0b8c-4af5-898d-60bc947997be',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdB8TLRZ3QsAlXMBNh/2IRo2QikCcYkUVWPVCJLh3TPIoEneJYb/4YY3gLp7+oAtKKLHacqWk/O+drck+TWj0jGQrRIkT0Tf4laugjD4gurKbvy7g/gIuHhbj/B6denncd0V/3jAkywPYQzU9eQHlrW87CT0SwW7tJAH22lZI09Y8rY9+TJ9S7KyY5eG2Eii304Hte+TDa3HzbJoDZhNkBI6S2TIglfqSuG+3q6KCF13HNHrdT+e7Ra1U5sAST2d+ydToeHmFHkfq33qCHmSIyFAKVLfw8QQXE1lAdJZ37C3eusMW96x59EFx9tt9yR8i6cqm+7COuPD2WL5I/fJTl Generated-by-Nova',key_name='tempest-keypair-244702523',keypairs=<?>,launch_index=0,launched_at=2018-03-19T02:54:44Z,launched_on='ubuntu-xenial-ovh-bhs1-0003046183',locked=False,locked_by=None,memory_mb=64,metadata={},migration_context=<?>,new_flavor=None,node='ubuntu-xenial-ovh-bhs1-0003046183',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=<?>,power_state=1,progress=0,project_id='621f30444ee441c980636a2137e449ed',ramdisk_id='',reservation_id='r-nzh7a7s5',root_device_name='/dev/vda',root_gb=0,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='Member',image_base_image_ref='06d1e26e-0b8c-4af5-898d-60bc947997be',image_container_format='bare',image_disk_format='q
Mar 19 03:00:14.816815 ubuntu-xenial-ovh-bhs1-0003046183 nova-compute[17330]: cow2',image_min_disk='0',image_min_ram='0',owner_project_name='tempest-ServerActionsTestJSON-1272777388',owner_user_name='tempest-ServerActionsTestJSON-1272777388'},tags=<?>,task_state='reboot_started_hard',terminated_at=None,updated_at=2018-03-19T02:55:13Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='31b211ed48bf4d1494f806667...

[YAMAMOTO Takashi (yamamoto)]

maybe due to nova change? Ib08afad3822f2ca95cfeea18d7f4fc4cb407b4d6

[YAMAMOTO Takashi (yamamoto)]

https://review.openstack.org/#/c/553035/

[YAMAMOTO Takashi (yamamoto)]

the nova change in question has been reverted in master and queens