Timeout with lots of floating ips

[Ben O'Hara]

We've a project with lots of floating ips allocated (~500)

When selecting the floating ip tab in the dashboard the list remains empty and after 20-30 seconds and error is show 'cant retrieve floating ips'

Horizon error logs this..

[Tue Sep 25 02:53:45.433507 2018] [:error] [pid 62287] ERROR openstack_dashboard.api.rest.utils HTTP exception with no status/code
[Tue Sep 25 02:53:45.433603 2018] [:error] [pid 62287] Traceback (most recent call last):
[Tue Sep 25 02:53:45.433626 2018] [:error] [pid 62287] File "/usr/share/openstack-dashboard/openstack_dashboard/api/rest/utils.py", line 127, in _wrapped
[Tue Sep 25 02:53:45.433644 2018] [:error] [pid 62287] data = function(self, request, *args, **kw)
[Tue Sep 25 02:53:45.433662 2018] [:error] [pid 62287] File "/usr/lib/python2.7/site-packages/designatedashboard/api/rest/passthrough.py", line 100, in get
[Tue Sep 25 02:53:45.433679 2018] [:error] [pid 62287] return passthrough_get(path, request).json()
[Tue Sep 25 02:53:45.433697 2018] [:error] [pid 62287] File "/usr/lib/python2.7/site-packages/designatedashboard/api/rest/passthrough.py", line 58, in _passthrough_request
[Tue Sep 25 02:53:45.433714 2018] [:error] [pid 62287] response.raise_for_status()
[Tue Sep 25 02:53:45.433731 2018] [:error] [pid 62287] File "/usr/lib/python2.7/site-packages/requests/models.py", line 928, in raise_for_status
[Tue Sep 25 02:53:45.433781 2018] [:error] [pid 62287] raise HTTPError(http_error_msg, response=self)
[Tue Sep 25 02:53:45.433837 2018] [:error] [pid 62287] HTTPError: 504 Server Error: GATEWAY TIMEOUT for url: https://XXXXXXXXX:9001/v2/reverse/floatingips

Other accounts work fine, and all hosts can talk to the designate api on 9001 so appears related to the number of floating ips allocated to a project.