named_get's do not support custom batches via slice
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Launchpad itself |
Triaged
|
High
|
Unassigned |
Bug Description
While working on the launchpadlib based changelog extract tool I run into the issue that I need to query for all published source packages to populate the initial changelogs (this is needed only once, from that point on its just a call to getPublishedSources with created_
I read on the launchpadlib api page that slicing is the solution for big queries like this, but that does not seem to work for me. I attach a small test program that demonstrates what I have in mind, the full source can be found at "lp:~mvo/+junk/lp-changelogs-crawler". The script works nicely if I limit the search to certain packages, but it does no longer work I ask it for all source packages.
Any hints or workarounds are welcome.
Note: after viewing oopsid=OOPS-1424S588 it is obvious that the default batching is being executed (50), but in this case it is timing out, hence trying to include the custom batch via a slice.
affects: | soyuz → launchpad-foundations |
Changed in launchpad-foundations: | |
status: | New → Triaged |
importance: | Undecided → High |
The output of the script for me is:
small query /api.staging. launchpad. net/beta/ ubuntu/ +archive/ primary/ +sourcepub/ 831491 /api.staging. launchpad. net/beta/ ubuntu/ +archive/ primary/ +sourcepub/ 590326 /api.staging. launchpad. net/beta/ ubuntu/ +archive/ primary/ +sourcepub/ 443952 /api.staging. launchpad. net/beta/ ubuntu/ +archive/ primary/ +sourcepub/ 380205 /api.staging. launchpad. net/beta/ ubuntu/ +archive/ primary/ +sourcepub/ 282482 /api.staging. launchpad. net/beta/ ubuntu/ +archive/ primary/ +sourcepub/ 226566 /api.staging. launchpad. net/beta/ ubuntu/ +archive/ primary/ +sourcepub/ 58979 -fail.py" , line 65, in <module> changelogs( ) -fail.py" , line 57, in get_changelogs getPublishedSou rces(status= "Published" )[0:10] python2. 6/dist- packages/ lazr/restfulcli ent/resource. py", line 460, in __call__ extra_headers) python2. 6/dist- packages/ lazr/restfulcli ent/_browser. py", line 204, in _request ent.errors. HTTPError: HTTP Error 503: Service Unavailable
https:/
https:/
https:/
https:/
https:/
https:/
https:/
big query
Traceback (most recent call last):
File "/tmp/launchpad
c.get_
File "/tmp/launchpad
changed = archive.
File "/usr/lib/
url, in_representation, http_method, extra_headers=
File "/usr/lib/
raise HTTPError(response, content)
lazr.restfulcli
I *assume* 503 is this case means (server) timeout.