nova-hyperv: nova-hyperv cannot download installer file after vmswitch is created on management NIC when DHCP enabled

[Larry Michel]

On systems for which the vmswitch is created over the management physical NIC (single connection), then trying to download the installer file immediately after the creation of the vmswitch fails consistently. These are the current high level installation steps for the charm:

1) Create vmswitch
2) Download installer file
3) Install nova compute

The reason that 2) would fail is because because the vethernet interface is still waiting for DHCP response by time attempt is made to download installer file. As can be seen in log below, there are also a lot of other failures due to dropped connections. The DownloadFile webrequest exception is at the very end:

2015-02-23 03:44:09 INFO worker.uniter.jujuc server.go:110 running hook tool "config-get.exe" ["--format=json" "vmswitch-name"]
2015-02-23 03:44:09 DEBUG worker.uniter.jujuc server.go:111 hook context id "nova-hyperv/0-install-751309805779905445"; dir "C:\\Juju\\lib\\juju\\agents\\unit-nova-hyperv-0\\charm"
2015-02-23 03:44:18 ERROR juju.api.watcher watcher.go:83 error trying to stop watcher: connection is shut down
... repeated x times
2015-02-23 03:44:18 ERROR juju.api.watcher watcher.go:83 error trying to stop watcher: connection is shut down
2015-02-23 03:44:18 DEBUG juju.worker.uniter.filter filter.go:358 got unit change
2015-02-23 03:44:18 ERROR juju.api.watcher watcher.go:83 error trying to stop watcher: connection is shut down
... repeated x times
2015-02-23 03:44:18 ERROR juju.worker.uniter.filter filter.go:132 error receiving message: WSARecv tcp 10.245.0.233:49186: An established connection was aborted by the software in your host machine.
2015-02-23 03:44:18 DEBUG juju.worker.uniter uniter.go:871 new environment change
2015-02-23 03:44:18 INFO juju.cmd.jujud agent.go:202 error pinging *api.State: connection is shut down
2015-02-23 03:44:18 ERROR juju.worker runner.go:208 fatal "apiaddressupdater": error receiving message: WSARecv tcp 10.245.0.233:49186: An established connection was aborted by the software in your host machine.
2015-02-23 03:44:18 DEBUG juju.worker runner.go:242 killing "upgrader"
2015-02-23 03:44:18 DEBUG juju.worker runner.go:242 killing "logger"
2015-02-23 03:44:18 DEBUG juju.worker runner.go:242 killing "uniter"
2015-02-23 03:44:18 DEBUG juju.worker runner.go:242 killing "rsyslog"
2015-02-23 03:44:18 INFO juju.cmd.jujud agent.go:202 error pinging *api.State: connection is shut down
2015-02-23 03:44:18 ERROR juju.worker runner.go:208 fatal "rsyslog": error receiving message: WSARecv tcp 10.245.0.233:49186: An established connection was aborted by the software in your host machine.
2015-02-23 03:44:18 INFO juju.cmd.jujud agent.go:202 error pinging *api.State: connection is shut down
2015-02-23 03:44:18 ERROR juju.worker runner.go:208 fatal "logger": error receiving message: WSARecv tcp 10.245.0.233:49186: An established connection was aborted by the software in your host machine.
rted by the software in your host machine.
2015-02-23 03:44:20 INFO unit.nova-hyperv/0.install logger.go:40
2015-02-23 03:44:20 INFO worker.uniter.jujuc server.go:110 running hook tool "config-get.exe" ["--format=json" "openstack-origin"]
2015-02-23 03:44:20 DEBUG worker.uniter.jujuc server.go:111 hook context id "nova-hyperv/0-install-751309805779905445"; dir "C:\\Juju\\lib\\juju\\agents\\unit-nova-hyperv-0\\charm"
2015-02-23 03:44:20 INFO worker.uniter.jujuc server.go:110 running hook tool "config-get.exe" ["--format=json" "installer-url"]
2015-02-23 03:44:20 DEBUG worker.uniter.jujuc server.go:111 hook context id "nova-hyperv/0-install-751309805779905445"; dir "C:\\Juju\\lib\\juju\\agents\\unit-nova-hyperv-0\\charm"
2015-02-23 03:44:20 INFO worker.uniter.jujuc server.go:110 running hook tool "juju-log.exe" ["Calling Download with http://10.245.0.10/download/HyperVNovaCompute_Icehouse_2014_1_3.msi and C:\\Users\\jujud\\AppData\\Local\\Temp\\HyperVNovaCompute_Icehouse_2014_1_3.msi"]
2015-02-23 03:44:20 DEBUG worker.uniter.jujuc server.go:111 hook context id "nova-hyperv/0-install-751309805779905445"; dir "C:\\Juju\\lib\\juju\\agents\\unit-nova-hyperv-0\\charm"
2015-02-23 03:44:20 INFO unit.nova-hyperv/0.juju-log cmd.go:247 Calling Download with http://10.245.0.10/download/HyperVNovaCompute_Icehouse_2014_1_3.msi and C:\Users\jujud\AppData\Local\Temp\HyperVNovaCompute_Icehouse_2014_1_3.msi
2015-02-23 03:44:22 INFO unit.nova-hyperv/0.install logger.go:40 Exception calling "DownloadFile" with "2" argument(s): "Unable to connect to
2015-02-23 03:44:22 INFO unit.nova-hyperv/0.install logger.go:40 the remote server"
2015-02-23 03:44:22 INFO unit.nova-hyperv/0.install logger.go:40 At C:\Juju\lib\juju\agents\unit-nova-hyperv-0\charm\hooks\install.ps1:37
2015-02-23 03:44:22 INFO unit.nova-hyperv/0.install logger.go:40 char:17
2015-02-23 03:44:22 INFO unit.nova-hyperv/0.install logger.go:40 + ExecRetry { (new-object System.Net.WebClient).DownloadFile($url,
2015-02-23 03:44:22 INFO unit.nova-hyperv/0.install logger.go:40 $download_l ...
2015-02-23 03:44:22 INFO unit.nova-hyperv/0.install logger.go:40 +
2015-02-23 03:44:22 INFO unit.nova-hyperv/0.install logger.go:40 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2015-02-23 03:44:22 INFO unit.nova-hyperv/0.install logger.go:40 + CategoryInfo : NotSpecified: (:) [], ParentContainsErrorRecordE
2015-02-23 03:44:22 INFO unit.nova-hyperv/0.install logger.go:40 xception
2015-02-23 03:44:22 INFO unit.nova-hyperv/0.install logger.go:40 + FullyQualifiedErrorId : WebException
2015-02-23 03:44:22 INFO unit.nova-hyperv/0.install logger.go:40
2015-02-23 03:44:22 INFO juju.worker.uniter.context context.go:304 handling reboot
2015-02-23 03:44:22 INFO juju.worker.uniter uniter.go:617 ran "install" hook
2015-02-23 03:44:22 INFO juju.worker.uniter uniter.go:628 committing "install" hook
2015-02-23 03:44:22 INFO juju.worker.uniter uniter.go:643 committed "install" hook
2015-02-23 03:44:22 DEBUG juju.worker.uniter modes.go:430 ModeContinue exiting
2015-02-23 03:44:22 INFO juju.worker.uniter uniter.go:159 unit "nova-hyperv/0" shutting down: tomb: dying

If installer file is downloaded prior to making networking change, then this issue can be avoided.

For example:

1) Download installer file
2) Create vmswitch and wait x seconds for networking to settle
3) Install nova compute

If installation steps are executed in this order, then the network errors still happened but the msi file is safely downloaded before the network goes down. Following creation of the vmswitch, msi nova compute installation is able to complete.

nova-hyperv-0\\charm"
2015-02-23 07:17:47 INFO worker.uniter.jujuc server.go:110 running hook tool "config-get.exe" ["--format=json" "installer-url"]
2015-02-23 07:17:47 DEBUG worker.uniter.jujuc server.go:111 hook context id "nova-hyperv/0-install-1660459888648598056"; dir "C:\\Juju\\lib\\juju\\agents\\unit-nova-hyperv-0\\charm"
2015-02-23 07:17:49 INFO worker.uniter.jujuc server.go:110 running hook tool "config-get.exe" ["--format=json" "vmswitch-name"]
2015-02-23 07:17:49 DEBUG worker.uniter.jujuc server.go:111 hook context id "nova-hyperv/0-install-1660459888648598056"; dir "C:\\Juju\\lib\\juju\\agents\\unit-nova-hyperv-0\\charm"
2015-02-23 07:17:58 ERROR juju.api.watcher watcher.go:83 error trying to stop watcher: connection is shut down
2015-02-23 07:17:58 ERROR juju.api.watcher watcher.go:83 error trying to stop watcher: connection is shut down
... errors repeated
2015-02-23 07:17:58 INFO juju.cmd.jujud agent.go:202 error pinging *api.State: connection is shut down
2015-02-23 07:17:58 ERROR juju.worker runner.go:208 fatal "logger": error receiving message: WSARecv tcp 10.245.0.243:49186: An established connection was aborted by the software in your host machine.
2015-02-23 07:18:00 INFO unit.nova-hyperv/0.install logger.go:40
2015-02-23 07:18:45 INFO worker.uniter.jujuc server.go:110 running hook tool "juju-log.exe" ["Running install"]
2015-02-23 07:18:45 DEBUG worker.uniter.jujuc server.go:111 hook context id "nova-hyperv/0-install-1660459888648598056"; dir "C:\\Juju\\lib\\juju\\agents\\unit-nova-hyperv-0\\charm"
2015-02-23 07:18:45 INFO unit.nova-hyperv/0.juju-log cmd.go:247 Running install
2015-02-23 07:18:45 INFO worker.uniter.jujuc server.go:110 running hook tool "juju-log.exe" ["Installing from C:\\Users\\jujud\\AppData\\Local\\Temp\\HyperVNovaCompute_Icehouse_2014_1_3.msi"]
2015-02-23 07:18:45 DEBUG worker.uniter.jujuc server.go:111 hook context id "nova-hyperv/0-install-1660459888648598056"; dir "C:\\Juju\\lib\\juju\\agents\\unit-nova-hyperv-0\\charm"
2015-02-23 07:18:45 INFO unit.nova-hyperv/0.juju-log cmd.go:247 Installing from C:\Users\jujud\AppData\Local\Temp\HyperVNovaCompute_Icehouse_2014_1_3.msi

[Larry Michel (lmic)]

This issue does not exist in latest version of the charm.