Should not wait 30 minutes for an instance to be provisioned

[Brian Murray]

I've recently seen patterns of s390x instances being requested and their provisioning, or connectivity to them, failing for unknown reasons which results in this traceback.

untu/autopkgtest-cloud/worker/worker[3812943]: ERROR: 0s autopkgtest [17:21:44]: starting date: 2023-07-25
                                                 0s autopkgtest [17:21:44]: git checkout: cabdbf6 Add timestamps to logfile
                                                 0s autopkgtest [17:21:44]: host juju-4d1272-prod-proposed-migration-14; command line: /home/ubuntu/autopkgtest/runner/autopkgtest --output-di>
                                               1800s No UUID given. Instance won't be deleted!
                                               1800s Unexpected error:
                                               1800s Traceback (most recent call last):
                                               1800s File "/home/ubuntu/autopkgtest/lib/VirtSubproc.py", line 740, in mainloop
                                               1800s command()
                                               1800s File "/home/ubuntu/autopkgtest/lib/VirtSubproc.py", line 669, in command
                                               1800s r = f(c, ce)
                                               1800s File "/home/ubuntu/autopkgtest/lib/VirtSubproc.py", line 258, in cmd_open
                                               1800s caller.hook_open()
                                               1800s File "/home/ubuntu/autopkgtest/virt/autopkgtest-virt-ssh", line 436, in hook_open
                                               1800s host_setup('open')
                                               1800s File "/home/ubuntu/autopkgtest/virt/autopkgtest-virt-ssh", line 263, in host_setup
                                               1800s execute_setup_script(command)
                                               1800s File "/home/ubuntu/autopkgtest/virt/autopkgtest-virt-ssh", line 202, in execute_setup_script
                                               1800s (status, out, err) = VirtSubproc.execute_timeout(
                                               1800s File "/home/ubuntu/autopkgtest/lib/VirtSubproc.py", line 144, in execute_timeout
                                               1800s (out, err) = sp.communicate(instr)
                                               1800s File "/usr/lib/python3.8/subprocess.py", line 1028, in communicate
                                               1800s stdout, stderr = self._communicate(input, endtime, timeout)
                                               1800s File "/usr/lib/python3.8/subprocess.py", line 1868, in _communicate
                                               1800s ready = selector.select(timeout)
                                               1800s File "/usr/lib/python3.8/selectors.py", line 415, in select
                                               1800s fd_event_list = self._selector.poll(timeout)
                                               1800s File "/home/ubuntu/autopkgtest/lib/VirtSubproc.py", line 64, in alarm_handler
                                               1800s raise Timeout()
                                               1800s VirtSubproc.Timeout
                                               1801s autopkgtest [17:51:45]: ERROR: testbed failure: cannot send to testbed: [Errno 32] Broken pipe

I wonder if there is a way for us to wait a shorter period of time for the instance to be provisioned without affecting any other timeouts.

[Paride Legovini (paride)]

That timeout is hardcoded in autopkgtest-virt-ssh:

        (status, out, err) = VirtSubproc.execute_timeout(
            None,
            1800,
            cmd,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE)

We could add a --timeout-setup-script flag to override it (to be submitted upstream). Should be easy to do.

[Tim Andersson (andersson123)]

Ooops, I missed that, will do now.

[Tim Andersson (andersson123)]

MP for autopkgtest-cloud:
https://code.launchpad.net/~andersson123/autopkgtest-cloud/+git/autopkgtest-cloud/+merge/447879

upstream MP:
https://salsa.debian.org/ci-team/autopkgtest/-/merge_requests/254