[rfe] rate-limit metadata API

[Guillaume Espanel]

At the moment, there is no limit on how many requests the metadata-agent
will handle. Some users may sometimes run scripts in their instances that
try to query the metadata endpoint at high rate (for example a bugged k8s
cloud controller manager), causing an increased load on some or all the
component above the metadata-agent.

We'd like to add some rate-limiting around the metadata-agent level to give
some protection to the other components. We were thinking to implement
that through a change to the metadata's haproxy's configuration using
stick-tables. In that case, the rate-limited queries would get a 429
without even being passed to the metadata-agent itself.

[Lajos Katona (lajos-katona)]

We discussed this RFE during our drivers meeting, see the logs:
https://meetings.opendev.org/meetings/neutron_drivers/2022/neutron_drivers.2022-09-09-14.00.log.html#l-51

The agreement was to approve this RFE, and ask for a spec where the details can be discussed. thanks for proposing.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-specs/+/856831

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-specs/+/856831
Committed: https://opendev.org/openstack/neutron-specs/commit/9462d10a7b1c65c877bec5002315fafcc49ac28e
Submitter: "Zuul (22348)"
Branch: master

commit 9462d10a7b1c65c877bec5002315fafcc49ac28e
Author: Guillaume Espanel <email address hidden>
Date: Fri Sep 9 17:53:23 2022 +0200

    Add spec for metadata-rate-limit

    Related-Bug: #1989199
    Change-Id: I31fe80a2fa4c495acfab5886a304866c7da0d04b

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/858879

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Slawek Kaplonski <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/858879
Reason: This review is > 4 weeks without comment, and failed Zuul jobs the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[Miguel Lavalle (minsel)]

During the Neutron Drivers meeting on April 21st, 2023 (https://meetings.opendev.org/meetings/neutron_drivers/2023/neutron_drivers.2023-04-21-14.00.log.html#l-176), it was agreed that the implementation of this RFE will support rate limiting for IPv4 and IPv6, but not at the same time. The user will have to choose whether rate limiting will be carried out in the deployment on IPv4 or IPv6. This is due to a limitation in the open source version of HAProxy, that only provides a maximum of 3 sticky counters. To support rate limiting for IPV4 and IPv6 simultaneously, 4 sticky counters would be needed.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/858879
Committed: https://opendev.org/openstack/neutron/commit/5f4a41326d7b1da03e1929b4fd8bcdaf69da19ab
Submitter: "Zuul (22348)"
Branch: master

commit 5f4a41326d7b1da03e1929b4fd8bcdaf69da19ab
Author: Guillaume Espanel <email address hidden>
Date: Thu Sep 22 11:27:04 2022 +0200

    Add rate-limiting to metadata agents

    Requests handled by the metadata-agents can now be rate-limited by
    source-ip. This is done to protect the OpenStack control plane against
    VMs querying the metadata endpoint in an overly enthusiastic way.

    Co-authored-by: Miguel Lavalle <email address hidden>

    Related-Bug: #1989199
    Change-Id: I748ccfa8b50496dcbcbe41fd22f84249a4d46b11

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron/+/892754

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Miguel Lavalle <email address hidden>" on branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron/+/892754

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/zed
Review: https://review.opendev.org/c/openstack/neutron/+/892868

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Miguel Lavalle <email address hidden>" on branch: stable/zed
Review: https://review.opendev.org/c/openstack/neutron/+/892868

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Miguel Lavalle <email address hidden>" on branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron/+/892754

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron/+/892754
Reason: stable/wallaby branch of openstack/neutron is about to be deleted. To be able to do that, all open patches need to be abandoned. Please cherry pick the patch to unmaintained/wallaby if you want to further work on this patch.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/neutron/+/927301

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Chris Buggy <email address hidden>" on branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/neutron/+/927301
Reason: This feature should not be back ported