maas 3.5 is returning bootx64.efi file with size 0

[Jacopo Rota]

In maas 3.5 the rack is returning zero data for the bootx64.efi file.

I can see the machine requested that file and the machine crashed (see screenshot)

Apr 05 10:32:55 maas-deb-enlist rackd[10611]: provisioningserver.rackdservices.tftp: [info] bootx64.efi requested by 10.0.2.184
Apr 05 10:32:55 maas-deb-enlist rackd[10611]: provisioningserver.rackdservices.http: [info] /images/bootx64.efi requested by 127.0.0.1

[Jacopo Rota (r00ta)]

Also snap seems to be affected

[Jacopo Rota (r00ta)]

Apparently it's completely random. I got a run where the region was reporting that the images were in sync and I kept asking for the bootloader.

For a couple of minutes I got responses with 200 and no payload, and then I finally got a response with the bootloader

[Jacopo Rota (r00ta)]

The image is available, but the agent is not fetching it apparently

ubuntu@maas-edge:/work/maas$ curl http://10.0.1.16:5248/bootx64.efi --output /tmp/test -v
* Trying 10.0.1.16:5248...
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
  0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 10.0.1.16 (10.0.1.16) port 5248 (#0)
> GET /bootx64.efi HTTP/1.1
> Host: 10.0.1.16:5248
> User-Agent: curl/7.81.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Server: nginx/1.18.0 (Ubuntu)
< Date: Fri, 05 Apr 2024 12:21:34 GMT
< Content-Length: 0
< Connection: keep-alive
<
  0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
* Connection #0 to host 10.0.1.16 left intact
ubuntu@maas-edge:/work/maas$ curl http://10.0.1.16:5240/MAAS/boot-resources/bootloaders/uefi/amd64/bootx64.efi --output /tmp/test -v
* Trying 10.0.1.16:5240...
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
  0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 10.0.1.16 (10.0.1.16) port 5240 (#0)
> GET /MAAS/boot-resources/bootloaders/uefi/amd64/bootx64.efi HTTP/1.1
> Host: 10.0.1.16:5240
> User-Agent: curl/7.81.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Server: nginx/1.18.0 (Ubuntu)
< Date: Fri, 05 Apr 2024 12:21:37 GMT
< Content-Type: application/octet-stream
< Content-Length: 960472
< Last-Modified: Wed, 04 Oct 2023 12:05:21 GMT
< Connection: keep-alive
< ETag: "651d5501-ea7d8"
< Accept-Ranges: bytes
<
{ [65269 bytes data]
100 937k 100 937k 0 0 777M 0 --:--:-- --:--:-- --:--:-- 915M
* Connection #0 to host 10.0.1.16 left intact
ubuntu@maas-edge:/work/maas$ curl http://10.0.1.16:5248/bootx64.efi --output /tmp/test -v
* Trying 10.0.1.16:5248...
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
  0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 10.0.1.16 (10.0.1.16) port 5248 (#0)
> GET /bootx64.efi HTTP/1.1
> Host: 10.0.1.16:5248
> User-Agent: curl/7.81.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Server: nginx/1.18.0 (Ubuntu)
< Date: Fri, 05 Apr 2024 12:21:44 GMT
< Content-Length: 0
< Connection: keep-alive
<
  0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
* Connection #0 to host 10.0.1.16 left intact
ubuntu@maas-edge:/work/maas$