Logging - No SG-log data found at /var/log/syslog

[Yushiro FURUKAWA]

When I created log-resource with security_group, log data didn't show at /var/log/syslog at all.

[Environment]
$ lsb_release -a; uname -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.4 LTS
Release: 16.04
Codename: xenial
Linux kolla 4.4.0-130-generic #156-Ubuntu SMP Thu Jun 14 08:53:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

devstack all-in-one

[Configuration]

/etc/neutron/neutron.conf
service_plugins = neutron.services.l3_router.l3_router_plugin.L3RouterPlugin,log

/etc/neutron/plugins/ml2/ml2_conf.ini
[securitygroup]
firewall_driver = openvswitch
[agent]
extensions = log

[Operation]
$ openstack server create --image cirros-0.3.5-x86_64-disk --flavor c1 --network private vm1
$ openstack network log create --resource-type security_group --resource <sg-id> --enable --event ALL sg-log

[ovs flow log]
I compared following conditions with'$ovs-ofctl dump-flows br-int':
http://paste.openstack.org/compare/726273/726272/

    1. Before creating log-resource
    2. After created log-resource

[Cao Xuan Hoang (hoangcx)]

I can confirm this bug and it affects to both stable/queens and current master

[Nguyen Phuong An (annp)]

From my understanding, we can get DROP log packets but we couldn't get ACCEPT log packet. Because the patch https://review.openstack.org/#/c/550421 removed conntrack information. Therefore no accept packet matched with accept log rule [1]. Here is my fix https://review.openstack.org/#/c/587681/.

[1] https://github.com/openstack/neutron/blob/master/neutron/services/logapi/drivers/openvswitch/ovs_firewall_log.py#L332

[Nguyen Phuong An (annp)]

@yushiro: Please note that, if you're using jounarl system, log-data will output to journal instead of syslog.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/587681
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=ced78395a7952d0e616055892645fd2a6165833f
Submitter: Zuul
Branch: master

commit ced78395a7952d0e616055892645fd2a6165833f
Author: Nguyen Phuong An <email address hidden>
Date: Wed Aug 1 10:55:55 2018 +0700

    Fix no ACCEPT event can get for security group logging

    Currently, we cannot get ACCEPT packet log because there are some
    changed related to ovs firewall code since ovs firewall logging has
    been merged.

    Regarding to performance perspective, we only log first accepted packet.
    So we only need to forward first accepted packet of each connection
    session to table 91 and table 92.

    So this patch fixes these issues.

    Closes-Bug: #1782576
    Change-Id: Ib6ced838a7ec6d5c459a8475318556001c31bdf0

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.openstack.org/591542

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/591547

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/591542
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=8aec1ecb09ec43aa97ba64a76839301d826447db
Submitter: Zuul
Branch: stable/rocky

commit 8aec1ecb09ec43aa97ba64a76839301d826447db
Author: Nguyen Phuong An <email address hidden>
Date: Wed Aug 1 10:55:55 2018 +0700

    Fix no ACCEPT event can get for security group logging

    Currently, we cannot get ACCEPT packet log because there are some
    changed related to ovs firewall code since ovs firewall logging has
    been merged.

    Regarding to performance perspective, we only log first accepted packet.
    So we only need to forward first accepted packet of each connection
    session to table 91 and table 92.

    So this patch fixes these issues.

    Closes-Bug: #1782576
    Change-Id: Ib6ced838a7ec6d5c459a8475318556001c31bdf0
    (cherry picked from commit ced78395a7952d0e616055892645fd2a6165833f)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 13.0.0.0rc2 release candidate.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/591547
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=c18e12800a82a1b53f4a407263f5c9c360e2febe
Submitter: Zuul
Branch: stable/queens

commit c18e12800a82a1b53f4a407263f5c9c360e2febe
Author: Nguyen Phuong An <email address hidden>
Date: Wed Aug 1 10:55:55 2018 +0700

    Fix no ACCEPT event can get for security group logging

    Currently, we cannot get ACCEPT packet log because there are some
    changed related to ovs firewall code since ovs firewall logging has
    been merged.

    Regarding to performance perspective, we only log first accepted packet.
    So we only need to forward first accepted packet of each connection
    session to table 91 and table 92.

    So this patch fixes these issues.

    Closes-Bug: #1782576
    Change-Id: Ib6ced838a7ec6d5c459a8475318556001c31bdf0
    (cherry picked from commit ced78395a7952d0e616055892645fd2a6165833f)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/591890
Committed: https://git.openstack.org/cgit/openstack/neutron-fwaas/commit/?id=93c71ce98af19f96282acabb58c446428bda2578
Submitter: Zuul
Branch: master

commit 93c71ce98af19f96282acabb58c446428bda2578
Author: Nguyen Phuong An <email address hidden>
Date: Wed Aug 15 10:05:51 2018 +0700

    Should forward only first accepted packet to table 91 and 92

    Regarding to performance perspective, we should only log first
    accepted packet. Therefore we need to forward only first accepted
    packet of each connection session to table 91 and table 92.
    This is also effort to sync up with ovsfw in neutron-side [1].

    [1] https://review.openstack.org/#/c/591547/

    Related-Bug: #1782576
    Change-Id: Iac01088bf2c76e3f28000389596f5a1a85478d9a

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/rocky
Review: https://review.openstack.org/599880

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/queens
Review: https://review.openstack.org/599903

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/599903
Committed: https://git.openstack.org/cgit/openstack/neutron-fwaas/commit/?id=87695e5652295dc3cfb147e9a8f34a333f649c94
Submitter: Zuul
Branch: stable/queens

commit 87695e5652295dc3cfb147e9a8f34a333f649c94
Author: Nguyen Phuong An <email address hidden>
Date: Wed Aug 15 10:05:51 2018 +0700

    Should forward only first accepted packet to table 91 and 92

    Regarding to performance perspective, we should only log first
    accepted packet. Therefore we need to forward only first accepted
    packet of each connection session to table 91 and table 92.
    This is also effort to sync up with ovsfw in neutron-side [1].

    [1] https://review.openstack.org/#/c/591547/

    Related-Bug: #1782576
    Change-Id: Iac01088bf2c76e3f28000389596f5a1a85478d9a
    (cherry picked from commit 93c71ce98af19f96282acabb58c446428bda2578)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/599880
Committed: https://git.openstack.org/cgit/openstack/neutron-fwaas/commit/?id=1b1c91c19ca294240b7566bc534f9801b25850cd
Submitter: Zuul
Branch: stable/rocky

commit 1b1c91c19ca294240b7566bc534f9801b25850cd
Author: Nguyen Phuong An <email address hidden>
Date: Wed Aug 15 10:05:51 2018 +0700

    Should forward only first accepted packet to table 91 and 92

    Regarding to performance perspective, we should only log first
    accepted packet. Therefore we need to forward only first accepted
    packet of each connection session to table 91 and table 92.
    This is also effort to sync up with ovsfw in neutron-side [1].

    [1] https://review.openstack.org/#/c/591547/

    Related-Bug: #1782576
    Change-Id: Iac01088bf2c76e3f28000389596f5a1a85478d9a
    (cherry picked from commit 93c71ce98af19f96282acabb58c446428bda2578)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 12.0.4 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 14.0.0.0b1 development milestone.