[RFE]Support get device_ids from floatingips

[zhaobo]

[problem]
Cannot get the device_id when query floatingip directly, users/admin need to query the specific Floating IP address to find the FloatingIP resources, then find the associated port_id, show port, finally get the device_id. Provide a way to get the device_id from floatingip could be very useful for users/external tools to query the instance by Floating IP address in high performance.

[Use cases]
In cloud env, the cloud admin find there is some abnormal traffics from source IP(Floating IP) in the external FW/Router/Network health check tools. Cloud admin need a ability to find the abnormal vnic and the VM owner to notify that the VM may be under virus attack. So the admin need to stop VM or take other rescue way to process it.

Proposal
========
* Support list/get floatingip show the related ports' device_id

[YAMAMOTO Takashi (yamamoto)]

does the use case need some work on nova side as well?

[Miguel Lavalle (minsel)]

Zhaobo,

It seems that two different things are being conflated in this RFE:

1) Show the in the floating IP the device_id of the associated port
2) Adding regex matching support in DB layer using "LIKE" SQL phrase

Regarding the second point I have two questions:

1) What would be regex matched? The floating ip address? Or the device_id associated to the floating IP?

2) Would the regex matching be used by Nova the same way this will be: https://review.openstack.org/#/c/521683/?

[hongbin (hongbin034)]

I can work on #1 if it is not taken. Please let me know.

[hongbin (hongbin034)]

For #2, I have the same question as @Miguel.

[zhaobo (zhaobo6)]

Hi Miguel,

This is very similar with current filter VM by IP. It is still prefix match.

For 1#, Nova want to query VMs through FloatingIP address by "172.*"/"172.16.*". So we need to get the device_id of associated internal port. The device_id should be the VM id.

For 2# Yeah, they are same. The requirement is the same with that one.

@hongbin, welcome. :)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/533327

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/534882

[Miguel Lavalle (minsel)]

Zhaobo,

Thanks for your response. In regards to #1, would elaborate as to how Nova will use the device_id in querying instances by floating IP?

[Matt Riedemann (mriedem)]

Nova has deprecated all of it's floating IP related REST APIs because they are proxies to the networking service (neutron) and nova-network is deprecated. So this doesn't sound like something we'd add to nova unless I'm not understanding the request.

Is the request that a new filter parameter is added to the "GET /servers" compute API such that you can filter server instances by a floating IP substring, similar to the "ip" filter parameter for GET /servers which does a substring match (LIKE in SQL) for the fixed IPs on the associated ports?

[Matt Riedemann (mriedem)]

Rather than add more proxy filter parameters to the compute API like the "ip" filter in GET /servers, it seems like it would make more sense to add something like https://review.openstack.org/#/c/521683/ where you can filter ports with a floating IP address substring.

[Hongbin Lu (hongbin.lu)]

Zhaobo,

Would I ask for clarification of the following:

  "cloud admin find there is some abnormal traffics from source IP(Floating IP)"

It sounds like the use case is that the cloud admin needs to identify the VM with an exact floating IP (i.e. nova list --ip 172.20.50.10). Then, it seems this can be done in an efficient way. What we need to done in nova is:

1. Query neutron API for the floating IP 172.20.50.10
2. Track down to the nova instance based on the returned floating IP properties

What confused me is why we would like a regex match of floating IP? What are the use cases that requires something like: nova list --ip 172.20.50.* ?

[zhaobo (zhaobo6)]

Sorry for this late return, I will make it clear with zhenyu, you will receive the email from him later.

Thanks every body. Then I will reply the questions 1 by 1 and order by the date time.

@Miguel,
* In regards to #1, would elaborate as to how Nova will use the device_id in querying instances by floating IP?
Nova now already support query the instances with fixed ip address and support prefix match. Currently, nova side can not know whether the request ip address is fixed ip or floating ip, according to zhenyu said, he will add 1 more try block there to try to call floatingip API to list instances by floating ip address and support prefix match, it sounds very simple.

Neutron side, neutron not support return device_id during list floatingip, nova need the returned device_ids from neutron to get the instances, just like filter port by ip address, nova side acctually use the device_ids to query instances.

@Matt,
https://review.openstack.org/#/c/521683/ Right, floatingip API may based on this for supporting this feature described by this RFE. So we don't want to extend nova API, as nova will just accept a IP address or a prefix match string. Kevin will add floatingip part in Nova, just another try block for Neutron floatingip API. :)

@Hongbin,
Actually, this requirement is from operational scenarios. Cloud admin found that some suspicious traffics from some known floatingips or a cidr(16.1.1.1, 16.1.1.8-253), they want to find these targets, and take some measures(shuting down/delete). Look that ips, that's why we need prefix match. And another reason is that fixed ip prefix match is supported already, we can based on it. Nova part owned by zhenyu, we just support him in neutron side. That means expose the device_id during list floatingip. That's all.

[Miguel Lavalle (minsel)]

To summarize:

What is needed from Neutron is:

1) The ability to to GET /floatingips providing in the filters s substring of the floating ip
2) That the device_id of the associated fixed ip address port be returned in the response to GET /floatingips

Correct?

[zhaobo (zhaobo6)]

Thanks, Miguel. That's correct.

[zhaobo (zhaobo6)]

So I think we still need to dicsuss and pursue the direction from Neutron driver core members in weekly driver meetings.
I will take part in the worklist then. :)

[Hongbin Lu (hongbin.lu)]

As pointed by by Miguel, this bug proposes two features: (1) support filtering of floating IP address substring and (2) add device_id to the floating IP response.

I have created a new bug report for #1 (https://bugs.launchpad.net/neutron/+bug/1754123) so that this bug can focus on #2.

[zhaobo (zhaobo6)]

Yeah, thanks Hongbin.

But I think it's better to just focus on the expose the device_id part in the next step. We could suspend the second part. :)

[Akihiro Motoki (amotoki)]

In my understanding, the motivation of this RFE bug is now to retrieve floating IP information and associated port information in one API call. This sounds a reasonable request.

IMHO instead of just adding device_id it is better to return port info as (for example) "port" or "port_detail" attribute of floating IP resource. Note that I don't think we need to include port attributes from extensions and we can start this from the basic port attributes.

[Miguel Lavalle (minsel)]

This RFE was approved by the Neutron Drivers meeting.

[Hongbin Lu (hongbin.lu)]

Note: during the neutron driver meeting, we settled on comment #18 suggested by @Akihiro

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/534882
Committed: https://git.openstack.org/cgit/openstack/neutron-lib/commit/?id=24a0877d1d234830a36794388bb342d3d91b9230
Submitter: Zuul
Branch: master

commit 24a0877d1d234830a36794388bb342d3d91b9230
Author: Hongbin Lu <email address hidden>
Date: Wed Jan 17 18:55:00 2018 +0000

    Add API extension fip-port-details

    This extension adds 'port_details' attribute to the floating IP resource.
    Neutron patch: https://review.openstack.org/#/c/533327/

    Change-Id: I568d07a778609cfd2a0ff72057be3f47a9aa8a0a
    Partial-Bug: #1723026

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/533327
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=c760d4f26f4b4753c80269437d2cd0b8f63dbc7c
Submitter: Zuul
Branch: master

commit c760d4f26f4b4753c80269437d2cd0b8f63dbc7c
Author: Hongbin Lu <email address hidden>
Date: Fri Jan 12 23:52:26 2018 +0000

    Add port_details to Floating IP

    If the floating IP is associated with a port, this attribute
    will contain detailed information of the associated port. This allows
    users to retrieve the floating IPs with information of its associated
    port (if any) in one API call.

    Other related patches:
    * neutron-lib: https://review.openstack.org/#/c/534882/
    * osc: https://review.openstack.org/#/c/533809/
    * sdk: https://review.openstack.org/#/c/533811/
    * tempest-plugin: https://review.openstack.org/#/c/561710/

    APIImpact the API reference needs to be updated

    Change-Id: I31e940d2986278d2fbee6fdfea4ff15f7c07ebaa
    Partial-Bug: #1723026

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/576000

[OpenStack Infra (hudson-openstack)]

Change abandoned by Maksim Malchuk (<email address hidden>) on branch: stable/queens
Review: https://review.openstack.org/576000

[Slawek Kaplonski (slaweq)]

According to comment #16 this RFE is only about "adding device_id to the floating IP response" and this is already done so I will mark it as fixed.