The container networking test does not support xenial

[Curtis Hovey]

The recently updated container networking test fails when run with xenial containers. Xenial's network commands and setup is unlike trusty. New development is need to keep the test current.

[Curtis Hovey (sinzui)]

2016-08-30 14:37:30 INFO Assessing address range.
2016-08-30 14:37:30 INFO Assessing address range for 0/lxd/1.
2016-08-30 14:37:31 INFO Default route from 0: default via 10.0.30.1 dev br-eth1 onlink

2016-08-30 14:37:31 INFO Fetching the device IP of onlink
2016-08-30 14:37:32 ERROR Command '('juju', '--show-log', 'ssh', '-m', 'functional-container-networking:functional-container-networking', '--proxy', '0', 'ip -4 -o addr show onlink')' returned non-zero exit status 1
2016-08-30 14:37:32 INFO Output from exception:
stdout:
Device "onlink" does not exist.

[Curtis Hovey (sinzui)]

Hi Andrew can you look into a fix for this? The container networking test test can only be run with the maas 1.9 or 2.0 substrate. You can contrive some way to test test locally, but to really be certain the fix work, we need to run the fixed script on munna or finfolk.

I think we need to find the xenial replacement commands. The script needs to discover which commands to use sicne we still want to test on trusty.

I discovered thie will working on a bug in the reboot call. I think I was running a running a command like
    assess_container_networking.py parallel-munna-vmaas /usr/bin/juju ./logs cn-test --series xenial --machine-type=lxd
but I am sure I was on my local host when I found it. Maybe parallel-lxd env will work for part of the test, if not I must have been uploading my my changes to the mass machine and running it in a terminal.

[Curtis Hovey (sinzui)]

We want this fixed right away to verify the fixed lxd package in xenial.

[Curtis Hovey (sinzui)]

This is still not fixes. Xenial's shutdown behavior is different. The first TB implies all is well until the broken pipe. the second TB indicates ssh was terminated. Xenial might reboot too quick for ssh to disconnect and report success.

2016-12-12 15:24:25 INFO Error running shutdown:
stdout:
stderr: 15:24:25 INFO juju.cmd supercommand.go:63 running juju [2.2-alpha1 gc go1.6.2]
15:24:25 INFO juju.juju api.go:72 connecting to API addresses: [10.0.200.3:17070]
15:24:25 INFO juju.api apiclient.go:570 dialing "wss://10.0.200.3:17070/model/ee7174a2-0c02-442a-8cf0-915070ebb573/api"
15:24:25 INFO juju.api apiclient.go:501 connection established to "wss://10.0.200.3:17070/model/ee7174a2-0c02-442a-8cf0-915070ebb573/api"
15:24:25 INFO juju.cmd.juju.commands ssh_common.go:326 proxy-ssh enabled but ignored: trying all addresses of target "0"
15:24:25 INFO juju.network hostport.go:274 dialed "10.0.200.3:22" successfully
Connection to 10.0.200.3 closed by remote host.
packet_write_wait: Connection to UNKNOWN port 0: Broken pipe
15:24:25 INFO cmd supercommand.go:465 command finished

Traceback (most recent call last):
  File "/var/lib/jenkins/juju-ci-tools/utility.py", line 553, in logged_exception
    yield
  File "/var/lib/jenkins/juju-ci-tools/deploy_stack.py", line 871, in runtime_context
    yield
  File "/var/lib/jenkins/juju-ci-tools/assess_container_networking.py", line 382, in cleaned_bootstrap_context
    yield
  File "/var/lib/jenkins/juju-ci-tools/assess_container_networking.py", line 412, in main
    assess_container_networking(bs_manager.client, machine_types)
  File "/var/lib/jenkins/juju-ci-tools/assess_container_networking.py", line 345, in assess_container_networking
    ssh(controller_client, '0', 'sudo shutdown -r now')
  File "/var/lib/jenkins/juju-ci-tools/assess_container_networking.py", line 75, in ssh
    return client.get_juju_output('ssh', '--proxy', machine, cmd)
  File "/var/lib/jenkins/juju-ci-tools/jujupy.py", line 1786, in get_juju_output
    model, user_name=self.env.user_name, **pass_kwargs)
  File "/var/lib/jenkins/juju-ci-tools/jujupy.py", line 1218, in get_juju_output
    raise e
CalledProcessError: Command '('juju', '--show-log', 'ssh', '-m', 'functional-container-networking-maas-2-1:controller', '--proxy', '0', 'sudo shutdown -r now')' returned non-zero exit status 255

[Curtis Hovey (sinzui)]

The network-health tests can test with xenial and they do some, maybe all of what the container networking test does