We investigated the issue, and it appears to be related to timing. If the copy operation occurs between step A/A2 (publishing) and step B (domination) of the publisher, the domination step deletes the package, but the new package is not published until the next run.
Since the publisher’s run time has been steadily increasing, this issue has become more evident, as it can take up to ~4 hours before the next run starts, during which the archive is in an unexpected state.
* From the logs (libstreamvbyte):
- Publisher run started before 4:52:00
- Copy operation occurred around 04:57:59
- Domination for resolute/proposed started at 04:59:36
* For the glibc incident:
- Publishing starts at 11:21:0
- Copy between 11:35:17 and 11:36:05
- Domination for questing/proposed starts at 11:38:11
We investigated the issue, and it appears to be related to timing. If the copy operation occurs between step A/A2 (publishing) and step B (domination) of the publisher, the domination step deletes the package, but the new package is not published until the next run.
Since the publisher’s run time has been steadily increasing, this issue has become more evident, as it can take up to ~4 hours before the next run starts, during which the archive is in an unexpected state.
* From the logs (libstreamvbyte):
- Publisher run started before 4:52:00
- Copy operation occurred around 04:57:59
- Domination for resolute/proposed started at 04:59:36
* For the glibc incident:
- Publishing starts at 11:21:0
- Copy between 11:35:17 and 11:36:05
- Domination for questing/proposed starts at 11:38:11