After further review of the test, there appear to be two scripts which to the bulk of the work, smoke_test_old.sh and smoke_test-0.13.0.sh In all cases, smoke_test_old.sh is executed across all of our tests , due to this if statement. if dpkg --compare-versions $FAN_VERSION lt 0.13; then echo "Testing Fan Networking (pre-0.13.0 API)" $RUN_DIR/smoke_test_old.sh "$@" RC=$? else echo "Testing Fan Networking (0.13.0+ API)" $RUN_DIR/smoke_test-0.13.0.sh "$@" RC=$? fi The latest ubuntu-fan is 0.12.13 which will always be less than 0.13. So this in itself is broken, Looking at the two scripts the newer one appears to be more fluid. However, it needs to be rewritten. To accommodate some changes as fanctl has changed in its output a bit. The older script appears to test just that basics, -Fanup/FanDown -Fan + Docker That's it, we don't include any of the testing for - Fan + LXD - Fan + Ping Remote Host Remote host could be anything, but obviously when firewalled across labs. that makes this test quite a headache to run. I'm assuming this would be riddled with if statements to identify if we are in the cloud or a metal lab. To fix this "FOR NOW" so stuff just works, then the function in comment #7 needs to be changed to something like this. def determine_underlay(self): cmd = 'ip route' output = utils.system_output(cmd, retain_output=False) m = output.split(" ")[2] underlay = m[:-1]+'0'+'/16' This way, we use the interface/subnet of whatever our default route is. As opposed to the previous method , which used regex to match strings, and in some cases it would fail. This Works in the clouds as tested on google. 18:43:33 INFO | Writing results to /home/jenkins/autotest/client/results/default 18:43:33 DEBUG| Initializing the state engine 18:43:33 DEBUG| Persistent state client.steps now set to [] 18:43:33 DEBUG| Persistent option harness now set to None 18:43:33 DEBUG| Persistent option harness_args now set to None 18:43:33 DEBUG| Selected harness: standalone 18:43:33 INFO | START ---- ---- timestamp=1587062613 localtime=Apr 16 18:43:33 18:43:33 DEBUG| Persistent state client._record_indent now set to 1 18:43:33 INFO | START ubuntu_fan_smoke_test.fan-smoke-test ubuntu_fan_smoke_test.fan-smoke-test timestamp=1587062613 localtime=Apr 16 18:43:33 18:43:33 DEBUG| Persistent state client._record_indent now set to 2 18:43:33 DEBUG| Persistent state client.unexpected_reboot now set to ('ubuntu_fan_smoke_test.fan-smoke-test', 'ubuntu_fan_smoke_test.fan-smoke-test') 18:43:33 DEBUG| Running 'ip route' 18:43:33 DEBUG| Running './ubuntu_fan_smoke_test.sh 10.240.0.0/16' 18:43:33 ERROR| [stderr] + awk /Installed:/{print $2} 18:43:33 ERROR| [stderr] + apt-cache policy ubuntu-fan 18:43:33 ERROR| [stderr] + FAN_VERSION=0.12.13 18:43:33 ERROR| [stderr] + dirname ./ubuntu_fan_smoke_test.sh 18:43:33 ERROR| [stderr] + RUN_DIR=. 18:43:33 ERROR| [stderr] + http_proxy= 18:43:33 ERROR| [stderr] + https_proxy= 18:43:33 ERROR| [stderr] + nc -w 2 squid.internal 3128 18:43:33 ERROR| [stderr] + echo 18:43:33 ERROR| [stderr] + nc -w 2 91.189.89.216 3128 18:43:33 ERROR| [stderr] + echo 18:43:35 ERROR| [stderr] + nc -w 2 10.245.64.1 3128 18:43:35 ERROR| [stderr] + echo 18:43:37 ERROR| [stderr] + export http_proxy 18:43:37 ERROR| [stderr] + export https_proxy 18:43:37 ERROR| [stderr] + URI=http://index.docker.io/v1/repositories/library/ubuntu/images 18:43:37 ERROR| [stderr] + wget -q -O/dev/null http://index.docker.io/v1/repositories/library/ubuntu/images 18:43:37 DEBUG| [stdout] 18:43:37 ERROR| [stderr] + echo 18:43:37 ERROR| [stderr] + [ -n ] 18:43:37 ERROR| [stderr] + dpkg --compare-versions 0.12.13 lt 0.13 18:43:37 DEBUG| [stdout] Testing Fan Networking (pre-0.13.0 API) 18:43:37 ERROR| [stderr] + echo Testing Fan Networking (pre-0.13.0 API) 18:43:37 ERROR| [stderr] + ./smoke_test_old.sh 10.240.0.0/16 18:43:39 DEBUG| [stdout] docker pull ubuntu: PASSED 18:43:40 DEBUG| [stdout] enable disable fan test: PASSED 18:43:41 DEBUG| [stdout] fanctl show test: PASSED 18:43:41 DEBUG| [stdout] fanctl check bridge config test: PASSED 18:44:52 ERROR| [stderr] debconf: delaying package configuration, since apt-utils is not installed 18:46:04 DEBUG| [stdout] fanatic docker test(--dns=169.254.169.254): PASSED 18:46:04 ERROR| [stderr] + RC=0 18:46:04 ERROR| [stderr] + exit 0 18:46:04 INFO | 18:46:04 INFO | Testing Fan Networking (pre-0.13.0 API) 18:46:04 INFO | docker pull ubuntu: PASSED 18:46:04 INFO | enable disable fan test: PASSED 18:46:04 INFO | fanctl show test: PASSED 18:46:04 INFO | fanctl check bridge config test: PASSED 18:46:04 INFO | fanatic docker test(--dns=169.254.169.254): PASSED 18:46:04 INFO | GOOD ubuntu_fan_smoke_test.fan-smoke-test ubuntu_fan_smoke_test.fan-smoke-test timestamp=1587062764 localtime=Apr 16 18:46:04completed successfully 18:46:04 INFO | END GOOD ubuntu_fan_smoke_test.fan-smoke-test ubuntu_fan_smoke_test.fan-smoke-test timestamp=1587062764 localtime=Apr 16 18:46:04 18:46:04 DEBUG| Persistent state client._record_indent now set to 1 18:46:04 DEBUG| Persistent state client.unexpected_reboot deleted 18:46:04 INFO | END GOOD ---- ---- timestamp=1587062764 localtime=Apr 16 18:46:04 18:46:04 DEBUG| Persistent state client._record_indent now set to 0 18:46:04 INFO | Report successfully generated at /home/jenkins/autotest/client/results/default/job_report.html I may be able to add testing of LXD services in addition to that. This will fix this test failing in the clouds.