Contents files for releases are sometimes empty

Bug #1871920 reported by Dan Watkins on 2020-04-09
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Launchpad itself
Critical
Colin Watson

Bug Description

Both http://archive.ubuntu.com/ubuntu/dists/focal/Contents-amd64.gz and http://archive.ubuntu.com/ubuntu/dists/focal/Contents-i386.gz are currently empty (breaking apt-file, which is how I noticed).

< cjwatson> Odd_Bloke: That must be an LP bug - can you file it with us?

Related branches

Dan Watkins (oddbloke) wrote :

We saw something similar for eoan at one point: https://bugs.launchpad.net/ubuntu/+source/apt-file/+bug/1848768

Colin Watson (cjwatson) wrote :

From #launchpad-ops on 2019-10-29:

14:09 * cjwatson peers at the Contents thing
14:09 <cjwatson> We must have got very unlucky with hitting whatever this bug is on the last run before eoan released, but it doesn't seem to be a totally isolated incident. The same thing happened to focal on 2019-10-26
16:32 <infinity> cjwatson: Are we racing somewhere we shouldn't be with Contents? I didn't look into it.
16:32 <infinity> But I guess speeding up the publisher would make that sort of thing more frequent.
16:33 <cjwatson> infinity: I think so, yes. Timestamps don't quite line up perfectly for that but maybe there's some buffering getting in the way
16:34 <cjwatson> The cp -a of ubuntu-overrides at the start of generate-contents-files was while the publisher was generating overrides, and it doesn't do the proper "write to .new, move into place" thing for those
16:34 <infinity> cjwatson: I forget how it all works, but in essence, we run a-f in Contents mode out of band, and suck the results in when publishing, yeah? Could it be that we just need an interim step of "copy the final results from a-f to the interim location the publisher will look at".
16:35 <infinity> Jinx, ish.
16:35 <cjwatson> We have such an interim step, it's just not quite stitched together exactly correctly
16:35 <infinity> Oops?
16:35 <cjwatson> Yes
16:36 <cjwatson> I'm guessing slightly but it looks about right

However, Tom fixed the thing that we thought was the cause there in https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/374930; so we're going to need to investigate further.

tags: added: lp-soyuz soyuz-publish
Changed in launchpad:
status: New → Triaged
importance: Undecided → Critical
summary: - Contents files for focal are currently empty
+ Contents files for focal are sometimes empty

The Contents files aren't currently empty, so this remains an intermittent problem.

Colin Watson (cjwatson) wrote :

This happened again on 2021-02-11 (possibly 2021-02-10).

Brian Murray (brian-murray) wrote :

Looking at some production Error Tracker retracer log files I found a couple of failures as early as 2021-02-10 18:03:59 and 2021-02-10 14:31:48.

Brian Murray (brian-murray) wrote :

My cronjob runs every 4 hours so this happened between 20:53 and 00:52.

Brian Murray (brian-murray) wrote :

Date: Thu, 11 Mar 2021 08:53:03 -0800 (PST)
From: Cron Daemon <email address hidden>
To: <email address hidden>
Subject: Cron <bdmurray@bizarro> /home/bdmurray/bin/contents-checker.py

http://archive.ubuntu.com/ubuntu/dists/hirsute-proposed/Contents-amd64.gz is empty!
http://archive.ubuntu.com/ubuntu/dists/hirsute/Contents-amd64.gz is empty!

Brian Murray (brian-murray) wrote :

This resolved itself at most 4 hours after this check:

Date: Thu, 11 Mar 2021 20:52:52 -0800 (PST)
From: Cron Daemon <email address hidden>
To: <email address hidden>
Subject: Cron <bdmurray@bizarro> /home/bdmurray/bin/contents-checker.py

http://archive.ubuntu.com/ubuntu/dists/hirsute-proposed/Contents-amd64.gz is empty!

summary: - Contents files for focal are sometimes empty
+ Contents files for releases are sometimes empty
Colin Watson (cjwatson) on 2021-03-19
Changed in launchpad:
assignee: nobody → Colin Watson (cjwatson)
status: Triaged → In Progress
Colin Watson (cjwatson) on 2021-03-25
Changed in launchpad:
status: In Progress → Fix Released
Brian Murray (brian-murray) wrote :

Actually they are probably supposed to be empty for hirsute-security and hirsute-updates but its weird that I just got notified about them as that implies they previously were not empty.

Colin Watson (cjwatson) wrote :

All those files have modification times of 2020-10-24, so if your script detected that they changed then I think it must be buggy.

Brian Murray (brian-murray) wrote :

It is the script and it had not taken into account the development release being frozen.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers