When machines reboot after deployment the tftp paths are wrong

[Jacopo Rota]

When machines reboot after deployment, sometimes the ftfp paths are wrong

Mar 13 09:16:01 maas-ha-1 maas-rackd[164441]: provisioningserver.rackdservices.tftp: [info] /grub/x86_64-efi/command.lst requested by 10.0.2.6
Mar 13 09:16:01 maas-ha-1 maas-rackd[164441]: provisioningserver.rackdservices.http: [info] /images/grub/x86_64-efi/command.lst requested by 127.0.0.1
Mar 13 09:16:01 maas-ha-1 maas-rackd[164441]: provisioningserver.rackdservices.tftp: [info] /grub/x86_64-efi/fs.lst requested by 10.0.2.6
Mar 13 09:16:01 maas-ha-1 maas-rackd[164441]: provisioningserver.rackdservices.http: [info] /images/grub/x86_64-efi/fs.lst requested by 127.0.0.1
Mar 13 09:16:01 maas-ha-1 maas-rackd[164441]: provisioningserver.rackdservices.tftp: [info] /grub/x86_64-efi/crypto.lst requested by 10.0.2.6
Mar 13 09:16:01 maas-ha-1 maas-rackd[164441]: provisioningserver.rackdservices.http: [info] /images/grub/x86_64-efi/crypto.lst requested by 127.0.0.1
Mar 13 09:16:01 maas-ha-1 maas-rackd[164441]: provisioningserver.rackdservices.tftp: [info] /grub/x86_64-efi/terminal.lst requested by 10.0.2.6
Mar 13 09:16:01 maas-ha-1 maas-rackd[164441]: provisioningserver.rackdservices.http: [info] /images/grub/x86_64-efi/terminal.lst requested by 127.0.0.1
Mar 13 09:16:01 maas-ha-1 maas-rackd[164441]: provisioningserver.rackdservices.tftp: [info] /grub/grub.cfg requested by 10.0.2.6
Mar 13 09:16:01 maas-ha-1 maas-rackd[164441]: provisioningserver.rackdservices.tftp: [info] /grub/grub.cfg-00:16:3e:4a:03:02 requested by 10.0.2.6
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: provisioningserver.rackdservices.tftp: [info] /grub/x86_64-efi/</html>.mod requested by 10.0.2.6
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: provisioningserver.rackdservices.http: [info] /images/grub/x86_64-efi/</html>.mod requested by 127.0.0.1
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: provisioningserver.rackdservices.tftp: [info] /grub/x86_64-efi/</body>.mod requested by 10.0.2.6
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: provisioningserver.rackdservices.http: [info] /images/grub/x86_64-efi/</body>.mod requested by 127.0.0.1
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: provisioningserver.rackdservices.tftp: [info] /grub/x86_64-efi/<hr><center>nginx/1.18.0 (Ubuntu)</center>.mod requested by 10.0.2.6
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: provisioningserver.rackdservices.tftp: [critical] TFTP back-end failed.
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: Traceback (most recent call last):
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: File "/snap/maas/34025/lib/python3.10/site-packages/provisioningserver/utils/twisted.py", line 60, in wrapper
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: return maybeDeferred(func, *args, **kwargs)
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: File "/snap/maas/34025/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 190, in maybeDeferred
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: result = f(*args, **kwargs)
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: File "/snap/maas/34025/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1905, in unwindGenerator
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: return _cancellableInlineCallbacks(gen)
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: File "/snap/maas/34025/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1815, in _cancellableInlineCallbacks
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: _inlineCallbacks(None, gen, status)
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: --- <exception caught here> ---
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: File "/snap/maas/34025/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 857, in _runCallbacks
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: current.result = callback( # type: ignore[misc]
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: File "/snap/maas/34025/lib/python3.10/site-packages/provisioningserver/rackdservices/tftp.py", line 322, in no_response_errback
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: failure.trap(BootConfigNoResponse)
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: File "/snap/maas/34025/usr/lib/python3/dist-packages/twisted/python/failure.py", line 451, in trap
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: self.raiseException()
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: File "/snap/maas/34025/usr/lib/python3/dist-packages/twisted/python/failure.py", line 475, in raiseException
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: raise self.value.with_traceback(self.tb)
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: File "/snap/maas/34025/lib/python3.10/site-packages/provisioningserver/rackdservices/tftp.py", line 334, in handle_boot_method
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: reader = yield self.get_cache_reader(file_name)
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: File "/snap/maas/34025/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1660, in _inlineCallbacks
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: result = current_context.run(gen.send, result)
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: File "/snap/maas/34025/lib/python3.10/site-packages/provisioningserver/rackdservices/tftp.py", line 316, in get_cache_reader
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: resp = yield self._cache_proxy.request(b"GET", url)
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: File "/snap/maas/34025/usr/lib/python3/dist-packages/twisted/web/client.py", line 1145, in request
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: uri = _ensureValidURI(uri.strip())
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: File "/snap/maas/34025/usr/lib/python3/dist-packages/twisted/web/_newclient.py", line 646, in _ensureValidURI
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: raise ValueError(f"Invalid URI {uri!r}")
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]: builtins.ValueError: Invalid URI b'http://localhost:5248/images/grub/x86_64-efi/<hr><center>nginx/1.18.0 (Ubuntu)</center>.mod'
Mar 13 09:16:03 maas-ha-1 maas-rackd[164441]:

here's the pcap https://drive.google.com/file/d/1omOGZMNfRBCv58RdOr8DaQP4GceBsZyE/view?usp=sharing

I confirm that in 3.4

/grub/x86_64-efi/command.lst
/grub/x86_64-efi/fs.lst
/grub/x86_64-efi/crypto.lst
/grub/x86_64-efi/terminal.lst

were returning empty page in my case (and I guess in some cases they are populated), but in 3.5 the rack returns a 404 html page, so the rack parses the page and craft all the new paths like /grub/x86_64-efi/</html>.mod accordingly