Snaps search not always same size

[Toto Bille]

When searching for a snap with page and size, the size is not always the same. There are a few different cases:

CURRENT BEHAVIOR

Search for web:

PAGE 1
curl -s -H "X-Ubuntu-Architecture: $ARCH" -H "X-Ubuntu-Frameworks: $FRAMEWORKS" -H 'Accept: application/hal+json' 'https://search.apps.ubuntu.com/api/v1/snaps/search?q=web&page=1&size=10'

-> size(result['_embedded']['clickindex:package']) = 10 OK

PAGE 2
curl -s -H "X-Ubuntu-Architecture: $ARCH" -H "X-Ubuntu-Frameworks: $FRAMEWORKS" -H 'Accept: application/hal+json' 'https://search.apps.ubuntu.com/api/v1/snaps/search?q=web&page=2&size=10'

-> size(result['_embedded']['clickindex:package']) = 10 OK

PAGE N
... OK

web is OK

Now search for core:

PAGE 1
curl -s -H "X-Ubuntu-Architecture: $ARCH" -H "X-Ubuntu-Frameworks: $FRAMEWORKS" -H 'Accept: application/hal+json' 'https://search.apps.ubuntu.com/api/v1/snaps/search?q=core&page=1&size=10'

-> size(result['_embedded']['clickindex:package']) = 10 OK

PAGE 2
curl -s -H "X-Ubuntu-Architecture: $ARCH" -H "X-Ubuntu-Frameworks: $FRAMEWORKS" -H 'Accept: application/hal+json' 'https://search.apps.ubuntu.com/api/v1/snaps/search?q=core&page=2&size=10'

-> size(result['_embedded']['clickindex:package']) = 8 NOK

PAGE 3
curl -s -H "X-Ubuntu-Architecture: $ARCH" -H "X-Ubuntu-Frameworks: $FRAMEWORKS" -H 'Accept: application/hal+json' 'https://search.apps.ubuntu.com/api/v1/snaps/search?q=core&page=3&size=10'

-> size(result['_embedded']['clickindex:package']) = 6 NOK

PAGE N
... NOK

core is not OK

EXPECTED BEHAVIOR

All page, except the last page, should have 10 (in this case) snaps in their search

[Kit Randel (blr)]

Hi Thomas, this behaviour is by design, as with the current implementation we are unable to guarantee fixed page sizes due to privacy filtering.

That being said, I think there is a change we can make to improve this significantly, resulting in the behaviour you would expect consistently for low page sizes. I'll let you know when we have a proposal for this work.

[Celso Providelo (cprov)]

Search batch-sizes are now constant (as requested) since the last store release:

{{{
$ echo "TOTAL: $(curl -s 'https://api.snapcraft.io/api/v1/snaps/search?q=core' | jq '._embedded["clickindex:package"] | length')" && for i in `seq 1 3`; do echo "PAG $i: $(curl -s "https://api.snapcraft.io/api/v1/snaps/search?q=core&size=10&page=$i" | jq '._embedded["clickindex:package"] | length')"; done
TOTAL: 27
PAG 1: 10
PAG 2: 10
PAG 3: 7
}}}

There are still use-cases that can't have guaranteed batch sizes as documented in https://docs.google.com/document/d/1kWV5PVVfYwU2qq4JrFkF_uI--2A-fCb2_MPYjhvGyKI/edit