Comment 8 for bug 541637

On 24 March 2010 22:21, Stuart Bishop <email address hidden> wrote:
> On Tue, Mar 23, 2010 at 4:10 AM, Martin Pool <email address hidden> wrote:
>> I think I would like the client to say "get me the first 50 bugs in
>> order by bug number" then "get me the next 50 with id >
>> previous_result[-1].id" and so on up.  That should be stable: I might
>> miss new things inserted into ranges I've already seen but that's
>> entirely reasonable.
>
> This is a good approach when results are ordered by a unique,
> unchangeable key. It is also more efficient at the database end (as
> 'WHERE id > x ORDER BY id LIMIT y' is faster than 'ORDER BY id OFFSET z
> LIMIT y' when you get to higher offsets).
>
> I wonder if we could special case things when results are ordered by id?

I almost think there is no point supporting other orders if in the
very plausible case of someone else editing bugs while you're
iterating you may lose many results.

However, it's also plausible that someone would want to know about the
50 hottest bugs in Ubuntu and we shouldn't make them retrieve all the
bugs and do a client-side sort.

Perhaps it should be special cased when ordered by id, id should be
made the default (if it's not already) and there should be a caution
about this in the api user documentation.

> Materializing the resultset on the server and reusing it for multiple
> client requests can be problematic, for example materializing several
> million rows of results but only ever retrieving the first 50 is very
> inefficient.

Right, letting users allocate arbitrary server-side storage seems poor.

Perhaps we should just document that people can avoid this by asking
for a larger batch. It doesn't seem unreasonable to me that you could
get a few hundred bugs in a single download; it may be much faster
too.

--
Martin <http://launchpad.net/~mbp/>