[Logging] firewall_group log resource and security_group log resource could not co-exist correctly

[LongKB]

I would like to report a bug that relates to co-existence between security_group log resource and firewall_group log resource in stable/rocky [1]. Please follow a given procedure to reproduce this bug.

Environment
-----------
- Devstack stable/rocky
- Install devstack with local.conf: http://paste.openstack.org/show/727916/
- Make sure that 'log' is added into '[agent] extensions' in '/etc/neutron/plugins/ml2/ml2_conf.ini'
- Topology: Set up topolocy with the following script http://paste.openstack.org/show/728095/

Testcase
--------
- Create firewall_group log resource:
  openstack network log create --resource-type firewall_group fwg_log
 +-----------------+--------------------------------------+
 | Field | Value |
 +-----------------+--------------------------------------+
 | Description | |
 | Enabled | True |
 | Event | ALL |
 | ID | ebe7a495-027e-4982-bd64-fe269617dd6d |
 | Name | fwg_log |
 | Project | 61c7600120ac44178c8064250d971b76 |
 | Resource | None |
 | Target | None |
 | Type | firewall_group |
 | created_at | 2018-08-15T07:55:37Z |
 | revision_number | 0 |
 | tenant_id | 61c7600120ac44178c8064250d971b76 |
 | updated_at | 2018-08-15T07:55:37Z |
 +-----------------+--------------------------------------+
- Ping from VM0 to router0 -> Cannot ping
- Check ovs flow with: sudo ovs-ofctl dump-flows br-int
  Results: http://paste.openstack.org/show/728098/
- Check log in /var/log/syslog with: tailf /var/log/syslog | grep -e ACCEPT
  Results: http://paste.openstack.org/show/728097/
  This log came from security_group log, but log_resource_ids=[u'ebe7a495-027e-4982-bd64-fe269617dd6d'] that include the ID of fwg_log

Each of log message contains a list of log objects that capture itself in log_resource_ids. This log message come from security_group logging, but it contains the ID of firewall_group log resource. Please note that, I only created firewall_group log with ID is 'ebe7a495-027e-4982-bd64-fe269617dd6d', and there is no security_group at this moment => Bug

References:
[1] https://docs.openstack.org/neutron/latest/admin/config-logging.html#service-workflow-for-operator

[Pawel Suder (pasuder)]

Hello Kim Bao Long,

I would like to raise following questions:

- what kind of message should be logged?
- could you extend that part of bug description "This log came from security_group log, but log_resource_ids=[u'ebe7a495-027e-4982-bd64-fe269617dd6d'] that include the ID of fwg_log", please?

Thank you,
Paweł

[LongKB (longkb.fvl)]

Hi Paweł,
Thank you for your question. Below is my answer:

> what kind of message should be logged?

The log message should follow: http://paste.openstack.org/show/728144/
firewall_group log should contain "port" field, and security_group should contains "vm_port" field. The reason is firewall_group log is working in router port(L3), and security_group log is working in vm_port(L2).

> could you extend that part of bug description "This log came from security_group log, but > log_resource_ids=[u'ebe7a495-027e-4982-bd64-fe269617dd6d'] that include the ID of fwg_log", please?

Each of log message contains a list of log objects that capture itself in log_resource_ids. This log message come from security_group logging, but it contains the ID of firewall_group log resource. Please note that, I only created firewall_group log with ID is 'ebe7a495-027e-4982-bd64-fe269617dd6d', and there is no security_group at this moment.

Thank you,
LongKB

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/591978
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=310bfa326fb9c016d02f9a505ae309ae0e15e7d4
Submitter: Zuul
Branch: master

commit 310bfa326fb9c016d02f9a505ae309ae0e15e7d4
Author: Kim Bao Long <email address hidden>
Date: Wed Aug 15 15:52:28 2018 +0700

    Fix incorrect log resources querying

    This patch aims to fix a co-existence problem between security_group
    and firewall_group log resources due to incorrect log querying from
    database.

    Change-Id: Ic60ad436e0fbb23cdae0e63eaeb73130ebf02089
    Closes-Bug: #1787119

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.openstack.org/598572

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/598572
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=3e68398335795d422d58a6e2110f2025907c5892
Submitter: Zuul
Branch: stable/rocky

commit 3e68398335795d422d58a6e2110f2025907c5892
Author: Kim Bao Long <email address hidden>
Date: Wed Aug 15 15:52:28 2018 +0700

    Fix incorrect log resources querying

    This patch aims to fix a co-existence problem between security_group
    and firewall_group log resources due to incorrect log querying from
    database.

    Change-Id: Ic60ad436e0fbb23cdae0e63eaeb73130ebf02089
    Closes-Bug: #1787119
    (Cherry-picked from commit 310bfa326fb9c016d02f9a505ae309ae0e15e7d4)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 13.0.1 release.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/604843

[OpenStack Infra (hudson-openstack)]

Change abandoned by Rodolfo Alonso Hernandez (<email address hidden>) on branch: stable/queens
Review: https://review.openstack.org/604843

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 14.0.0.0b1 development milestone.