MAAS 3.5 is deleting a boot resource immediately after it was downloaded

[Jacopo Rota]

MAAS 3.5 is deleting a boot resource immediately after it was downloaded.

Link to SolQA run https://solutions.qa.canonical.com/testruns/6c68a38c-1ecf-4827-a61e-59225dfaf25d

To recap my investigation:
1) we have first sync-boot-resource:streams started at 2024-05-02 UTC 02:45:16.89 and completed at 2024-05-02 UTC 02:51:25.11
2) then another one is started at 2024-05-02 UTC 02:51:49.12 and finishes at 2024-05-02 UTC 02:53:09.08
3) we start deleting the images
  - 2024-05-02 UTC 02:53:09.31 -> 2024-05-02 UTC 02:53:10.49
  - 2024-05-02 UTC 02:53:10.66 -> 2024-05-02 UTC 02:53:12.03
  - 2024-05-02 UTC 02:53:12.15 -> 2024-05-02 UTC 02:53:13.14
  - and so on

So it means we are in a kind of for loop deleting the images. My guess is this comes from maasserver.bootresources.BootResourceStore.resource_set_cleaner which is called in the finalize function after the images are download and synched.

[Jacopo Rota (r00ta)]

Without additional logs it's impossible to know what exactly happened. I'm building a ppa and schedule some runs in order to collect the logs to properly triage the bug