Comment 13 for bug 1323658

Revision history for this message
Kyle Mestery (mestery) wrote : Re: SSH EOFError - Public network connectivity check failed

I'm now able to reproduce this locally, and in each case (either restart or resize), the guest VM itself appears to not be booted. I've added code to dump the console of the guest during the failure, and in each case the recall from Nova comes back with no data in the response body (see below). This leads me to believe this is a problem with the guest after the restart/resize:

2014-06-09 16:14:32.640 30408 TRACE tempest.scenario.test_network_advanced_server_ops AssertionError: Timed out waiting for 172.24.4.16 to become reachable
2014-06-09 16:14:32.640 30408 TRACE tempest.scenario.test_network_advanced_server_ops
2014-06-09 16:14:32.642 30408 DEBUG tempest.scenario.manager [-] Console output for 86e7e749-83ec-4e39-9a39-3042213569b5 _log_console_output tempest/scenario/manager.py:397
2014-06-09 16:14:32.642 30408 DEBUG novaclient.client [-]
REQ: curl -i 'http://173.39.225.77:8774/v2/54a724dc112c482e84975fc6c1e3014e/servers/86e7e749-83ec-4e39-9a39-3042213569b5/action' -X POST -H "X-Auth-Project-Id: TestNetworkAdvancedServerOps-1411494319" -H "User-Agent: python-novaclient" -H "Content-Type: application/json" -H "Accept: application/json" -H "X-Auth-Token: MIIRxAYJKoZIhvcNAQcCoIIRtTCCEbECAQExCTAHBgUrDgMCGjCCEBoGCSqGSIb3DQEHAaCCEAsEghAHeyJhY2Nlc3MiOiB7InRva2VuIjogeyJpc3N1ZWRfYXQiOiAiMjAxNC0wNi0wOVQxNjowNDoyNS41MTM2ODkiLCAiZXhwaXJlcyI6ICIyMDE0LTA2LTA5VDE3OjA0OjI1WiIsICJpZCI6ICJwbGFjZWhvbGRlciIsICJ0ZW5hbnQiOiB7ImRlc2NyaXB0aW9uIjogIlRlc3ROZXR3b3JrQWR2YW5jZWRTZXJ2ZXJPcHMtMTQxMTQ5NDMxOS1kZXNjIiwgImVuYWJsZWQiOiB0cnVlLCAiaWQiOiAiNTRhNzI0ZGMxMTJjNDgyZTg0OTc1ZmM2YzFlMzAxNGUiLCAibmFtZSI6ICJUZXN0TmV0d29ya0FkdmFuY2VkU2VydmVyT3BzLTE0MTE0OTQzMTkifX0sICJzZXJ2aWNlQ2F0YWxvZyI6IFt7ImVuZHBvaW50cyI6IFt7ImFkbWluVVJMIjogImh0dHA6Ly8xNzMuMzkuMjI1Ljc3Ojg3NzQvdjIvNTRhNzI0ZGMxMTJjNDgyZTg0OTc1ZmM2YzFlMzAxNGUiLCAicmVnaW9uIjogIlJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTczLjM5LjIyNS43Nzo4Nzc0L3YyLzU0YTcyNGRjMTEyYzQ4MmU4NDk3NWZjNmMxZTMwMTRlIiwgImlkIjogIjEyYWVlMDc2M2E2ZTRkMDFiMTc0ZTMyMWQ0NTIxZjRiIiwgInB1YmxpY1VSTCI6ICJodHRwOi8vMTczLjM5LjIyNS43Nzo4Nzc0L3YyLzU0YTcyNGRjMTEyYzQ4MmU4NDk3NWZjNmMxZTMwMTRlIn1dLCAiZW5kcG9pbnRzX2xpbmtzIjogW10sICJ0eXBlIjogImNvbXB1dGUiLCAibmFtZSI6ICJub3ZhIn0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzE3My4zOS4yMjUuNzc6OTY5Ni8iLCAicmVnaW9uIjogIlJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTczLjM5LjIyNS43Nzo5Njk2LyIsICJpZCI6ICIzOTJjY2Y3MzAwYzc0ZThhYjI5ZTA2YTUyZTFlOWEwYyIsICJwdWJsaWNVUkwiOiAiaHR0cDovLzE3My4zOS4yMjUuNzc6OTY5Ni8ifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAibmV0d29yayIsICJuYW1lIjogIm5ldXRyb24ifSwgeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTczLjM5LjIyNS43Nzo4Nzc2L3YyLzU0YTcyNGRjMTEyYzQ4MmU4NDk3NWZjNmMxZTMwMTRlIiwgInJlZ2lvbiI6ICJSZWdpb25PbmUiLCAiaW50ZXJuYWxVUkwiOiAiaHR0cDovLzE3My4zOS4yMjUuNzc6ODc3Ni92Mi81NGE3MjRkYzExMmM0ODJlODQ5NzVmYzZjMWUzMDE0ZSIsICJpZCI6ICIwY2ZkMTkyMjNmYzc0YTkzYjAyZDI5MTg3ODkxMzc1ZSIsICJwdWJsaWNVUkwiOiAiaHR0cDovLzE3My4zOS4yMjUuNzc6ODc3Ni92Mi81NGE3MjRkYzExMmM0ODJlODQ5NzVmYzZjMWUzMDE0ZSJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJ2b2x1bWV2MiIsICJuYW1lIjogImNpbmRlcnYyIn0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzE3My4zOS4yMjUuNzc6ODc3NC92MyIsICJyZWdpb24iOiAiUmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xNzMuMzkuMjI1Ljc3Ojg3NzQvdjMiLCAiaWQiOiAiOWMzNzQ2ZDdlZGViNDc2MWE1Njk5NzJmNmQzZjhkNTAiLCAicHVibGljVVJMIjogImh0dHA6Ly8xNzMuMzkuMjI1Ljc3Ojg3NzQvdjMifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAiY29tcHV0ZXYzIiwgIm5hbWUiOiAibm92YXYzIn0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzE3My4zOS4yMjUuNzc6MzMzMyIsICJyZWdpb24iOiAiUmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xNzMuMzkuMjI1Ljc3OjMzMzMiLCAiaWQiOiAiMWUyYjcyMDAzYTIzNDYwM2IzYTRhM2M0Mjc5OTk5YjYiLCAicHVibGljVVJMIjogImh0dHA6Ly8xNzMuMzkuMjI1Ljc3OjMzMzMifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAiczMiLCAibmFtZSI6ICJzMyJ9LCB7ImVuZHBvaW50cyI6IFt7ImFkbWluVVJMIjogImh0dHA6Ly8xNzMuMzkuMjI1Ljc3OjkyOTIiLCAicmVnaW9uIjogIlJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTczLjM5LjIyNS43Nzo5MjkyIiwgImlkIjogIjFmZDc4MWQ0MTkxMzQwOTM5MTYxYzdiNTI3ODVhZDZkIiwgInB1YmxpY1VSTCI6ICJodHRwOi8vMTczLjM5LjIyNS43Nzo5MjkyIn1dLCAiZW5kcG9pbnRzX2xpbmtzIjogW10sICJ0eXBlIjogImltYWdlIiwgIm5hbWUiOiAiZ2xhbmNlIn0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzE3My4zOS4yMjUuNzc6ODAwMC92MSIsICJyZWdpb24iOiAiUmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xNzMuMzkuMjI1Ljc3OjgwMDAvdjEiLCAiaWQiOiAiNjBkNDFiNDFkY2QxNDk3OWIyZDJjMjBjNmZhMjlmYjgiLCAicHVibGljVVJMIjogImh0dHA6Ly8xNzMuMzkuMjI1Ljc3OjgwMDAvdjEifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAiY2xvdWRmb3JtYXRpb24iLCAibmFtZSI6ICJoZWF0In0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzE3My4zOS4yMjUuNzc6ODc3Ni92MS81NGE3MjRkYzExMmM0ODJlODQ5NzVmYzZjMWUzMDE0ZSIsICJyZWdpb24iOiAiUmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xNzMuMzkuMjI1Ljc3Ojg3NzYvdjEvNTRhNzI0ZGMxMTJjNDgyZTg0OTc1ZmM2YzFlMzAxNGUiLCAiaWQiOiAiNDA4ZWI5ZWE2MzA1NGEwNWJlZGIzYzFlNzE3NjBjZjAiLCAicHVibGljVVJMIjogImh0dHA6Ly8xNzMuMzkuMjI1Ljc3Ojg3NzYvdjEvNTRhNzI0ZGMxMTJjNDgyZTg0OTc1ZmM2YzFlMzAxNGUifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAidm9sdW1lIiwgIm5hbWUiOiAiY2luZGVyIn0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzE3My4zOS4yMjUuNzc6ODc3My9zZXJ2aWNlcy9BZG1pbiIsICJyZWdpb24iOiAiUmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xNzMuMzkuMjI1Ljc3Ojg3NzMvc2VydmljZXMvQ2xvdWQiLCAiaWQiOiAiMjVlNjU4YTMyNDBhNDIzMjhlNGMzMjYxMTk3YTFkYmUiLCAicHVibGljVVJMIjogImh0dHA6Ly8xNzMuMzkuMjI1Ljc3Ojg3NzMvc2VydmljZXMvQ2xvdWQifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAiZWMyIiwgIm5hbWUiOiAiZWMyIn0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzE3My4zOS4yMjUuNzc6ODAwNC92MS81NGE3MjRkYzExMmM0ODJlODQ5NzVmYzZjMWUzMDE0ZSIsICJyZWdpb24iOiAiUmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xNzMuMzkuMjI1Ljc3OjgwMDQvdjEvNTRhNzI0ZGMxMTJjNDgyZTg0OTc1ZmM2YzFlMzAxNGUiLCAiaWQiOiAiMzQzN2U5MWRmMTc5NDJlNzk0NTU1YjAyYTA5NGFhYjYiLCAicHVibGljVVJMIjogImh0dHA6Ly8xNzMuMzkuMjI1Ljc3OjgwMDQvdjEvNTRhNzI0ZGMxMTJjNDgyZTg0OTc1ZmM2YzFlMzAxNGUifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAib3JjaGVzdHJhdGlvbiIsICJuYW1lIjogImhlYXQifSwgeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTczLjM5LjIyNS43NzozNTM1Ny92Mi4wIiwgInJlZ2lvbiI6ICJSZWdpb25PbmUiLCAiaW50ZXJuYWxVUkwiOiAiaHR0cDovLzE3My4zOS4yMjUuNzc6NTAwMC92Mi4wIiwgImlkIjogIjFmOTdiNDA1NjRjOTQzZDU5MjcyMTA0YjgwZTg1NDQ4IiwgInB1YmxpY1VSTCI6ICJodHRwOi8vMTczLjM5LjIyNS43Nzo1MDAwL3YyLjAifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAiaWRlbnRpdHkiLCAibmFtZSI6ICJrZXlzdG9uZSJ9XSwgInVzZXIiOiB7InVzZXJuYW1lIjogIlRlc3ROZXR3b3JrQWR2YW5jZWRTZXJ2ZXJPcHMtMzQ5NTc3NjgyIiwgInJvbGVzX2xpbmtzIjogW10sICJpZCI6ICI1Mzg3YjdjM2YzMDA0ZjY1YTg2YTg0YmNmMDdjOTNhZCIsICJyb2xlcyI6IFt7Im5hbWUiOiAiX21lbWJlcl8ifV0sICJuYW1lIjogIlRlc3ROZXR3b3JrQWR2YW5jZWRTZXJ2ZXJPcHMtMzQ5NTc3NjgyIn0sICJtZXRhZGF0YSI6IHsiaXNfYWRtaW4iOiAwLCAicm9sZXMiOiBbIjlmZTJmZjllZTQzODRiMTg5NGE5MDg3OGQzZTkyYmFiIl19fX0xggGBMIIBfQIBATBcMFcxCzAJBgNVBAYTAlVTMQ4wDAYDVQQIDAVVbnNldDEOMAwGA1UEBwwFVW5zZXQxDjAMBgNVBAoMBVVuc2V0MRgwFgYDVQQDDA93d3cuZXhhbXBsZS5jb20CAQEwBwYFKw4DAhowDQYJKoZIhvcNAQEBBQAEggEAAnVjb6K7gbLqwgl5IROR5CHkaIu+NvcGkTXt7sUjU98CCcYP4uEClYb5ivxRQpzJv44+IOArB439Cr9hPOEJD+dAXD6V3DuNXVYvAypcL72TMkOV77LDMTKIYUNvJPY0K9F8dWGxA-jaLI6y-Dek2VSzwiKUcL7xDhzj9srnXL7lks12DZ0LW8SKQb2F1jxL6vcJzOaz5NGSt5NyPtK4gtjg+-W30gZEriBHzwYX+lKS3V4J5UrwoatkA4I-BJ+SUfMNGlKxKeUm7akccMSrQ7iD2aXQcNzYQ8SIG3bTS0zhtkRKcDFdyRUgTMV4Kpk5XgG5yodANeMbwXoK7aQgsg==" -d '{"os-getConsoleOutput": {"length": null}}'
 http_log_req /opt/stack/python-novaclient/novaclient/client.py:181
2014-06-09 16:14:32.758 30408 DEBUG novaclient.client [-] RESP: [200] CaseInsensitiveDict({'date': 'Mon, 09 Jun 2014 16:14:32 GMT', 'content-length': '14', 'content-type': 'application/json', 'x-compute-request-id': 'req-f24c4321-5cfa-46f5-93a6-636fe19f7daf'})
RESP BODY: {"output": ""}