fluentd rewrite rules are using deprecated syntax

[Piotr Parczewski]

Since version 2.0.0 of fluent rewrite filter, 'rewriteruleXX' is no longer valid syntax in 01-rewrite.conf. Therefore, after deployment fluentd container is in restarting state.

[Piotr Parczewski (pparczewski)]

Reference: https://github.com/fluent/fluent-plugin-rewrite-tag-filter/commit/248ed8e97def0a60121199c0d0e2fa8785f0d076

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/518490

[OpenStack Infra (hudson-openstack)]

Change abandoned by Chason Chan (<email address hidden>) on branch: master
Review: https://review.openstack.org/518490
Reason: Abandon this one because of the other sample way to fix the bug, see https://review.openstack.org/#/c/518820/3

[Chason Chan (chen-xing)]

This bug can be fixed in Kolla project, see https://review.openstack.org/518820

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/518490
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=419a2fc9fd206447f37fbcf62da8e15ab96f8530
Submitter: Zuul
Branch: master

commit 419a2fc9fd206447f37fbcf62da8e15ab96f8530
Author: chenxing <email address hidden>
Date: Wed Nov 8 08:39:35 2017 +0000

    Fix the deprecated syntax in 01-rewrite.conf

    The current syntax of Ubuntu fluentd rewrite rules are not supported
    anymore, reference to this commit[1].
    According to the build of this patch[2], The Centos has no such isuue.
    Only Ubuntu needs to be upgrade to use the <rule> section.

    * Centos use 01-rewrite-0.12.conf.j2
    * Ubuntu use 01-rewrite-0.14.conf.j2

    backport: pike

    [1] https://github.com/fluent/fluent-plugin-rewrite-tag-filter/commit/248ed8e97def0a60121199c0d0e2fa8785f0d076
    [2] https://review.openstack.org/#/c/517907

    Depends-on: I47985113fe732569d640a262ca832c6edc8c2bb6
    Change-Id: I0b131c2f65652cf3c61b33d8162097b047603923
    Closes-Bug: #1730664

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/521433

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/521337
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=7ba2d7a383775bb8224c070bccfc4ee4edade08f
Submitter: Zuul
Branch: stable/ocata

commit 7ba2d7a383775bb8224c070bccfc4ee4edade08f
Author: chenxing <email address hidden>
Date: Wed Nov 8 08:39:35 2017 +0000

    Fix the deprecated syntax in 01-rewrite.conf

    Before pike, kolla use upstream fluentd repo. and it is now bump to
    v0.14. So need update the rewite syntax.

    Change-Id: I0b131c2f65652cf3c61b33d8162097b047603923
    Closes-Bug: #1730664
    (cherry picked from commit 419a2fc9fd206447f37fbcf62da8e15ab96f8530)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/521336
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=9c76488694c8f7e3251a8ee45c928d0a7960f469
Submitter: Zuul
Branch: stable/ocata

commit 9c76488694c8f7e3251a8ee45c928d0a7960f469
Author: Jeffrey Zhang <email address hidden>
Date: Sun Nov 19 11:34:02 2017 +0800

    Pin fluent-plugin-grok-parser to 2.1.4

    We are using upstream fluentd, which has bump to 0.14.0. Need use high
    version of plugin.

    Change-Id: I8bef4e153621be9f2983f05a045e7ce0849483a1
    Closes-Bug: #1730664

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/521045
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=ed669e22c830d7044c99891faf8ce9ae199f63b7
Submitter: Zuul
Branch: stable/pike

commit ed669e22c830d7044c99891faf8ce9ae199f63b7
Author: chenxing <email address hidden>
Date: Mon Nov 20 03:04:11 2017 +0000

    Fix the Ubuntu Fluentd image

    For ubuntu, we actually use fluentd 0.14.22 and ruby 2.1.10 now.
    Thus, we should use the specific gem version for the plugins:

     - fluent-plugin-rewrite-tag-filter:2.0.0
    According to the README.md of this plugin[1]
    It shows:
    ----------------------------------------------------------
    | fluent-plugin-rewrite-tag-filter | Fluentd | Ruby |
    |----------------------------------|------------|--------|
    | >= 2.0.0 | >= v0.14.2 | >= 2.1 |
    | < 2.0.0 | >= v0.12.0 | >= 1.9 |
    ----------------------------------------------------------

     - fluent-plugin-grok-parser:2.1.4
    When using the fluent-plugin-rewrite-tag-filter 2.0.0 to fix this bug, another
    bug[1] which is the same as this one[2] was triggered.
    According to https://github.com/fluent/fluent-plugin-grok-parser, it shows:
    ---------------------------------------------------
    | fluent-plugin-grok-parser | fluentd | Ruby |
    |---------------------------|------------|--------|
    | >= 1.0.0 | >= v0.14.0 | >= 2.1 |
    | < 1.0.0 | >= v0.12.0 | >= 1.9 |
    ---------------------------------------------------

    We actually use fluentd 0.14.22 now, but the version of fluent-plugin-grok-parser
    is still pinned in 0.3.1. So we need to upgrade the fluent-plugin-grok-parser to
    2.1.4.

    [1] https://github.com/fluent/fluent-plugin-rewrite-tag-filter
    [2] http://paste.openstack.org/show/626145/
    [3] https://github.com/fluent/fluent-plugin-grok-parser/issues/32

    Change-Id: I47985113fe732569d640a262ca832c6edc8c2bb6
    Partial-Bug: #1730664

[Serge Radinovich (srad015)]

This does not appear to have fixed the issue:

https://git.openstack.org/cgit/openstack/kolla/commit/?id=ed669e22c830d7044c99891faf8ce9ae199f63b7

I built stable/pike kolla and built images with -t source -b ubuntu.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/521433
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=444b0e2b5527951d6e3d95e11e0e375283e7bb7a
Submitter: Zuul
Branch: stable/pike

commit 444b0e2b5527951d6e3d95e11e0e375283e7bb7a
Author: chenxing <email address hidden>
Date: Mon Nov 20 06:33:36 2017 +0000

    Fix the deprecated syntax in 01-rewrite.conf

    The current syntax of Ubuntu fluentd rewrite rules are not supported
    anymore, reference to this commit[1].
    According to the build of this patch[2], The Centos has no such isuue.
    Only Ubuntu needs to be upgrade to use the <rule> section.

    * Centos use 01-rewrite-0.12.conf.j2
    * Ubuntu use 01-rewrite-0.14.conf.j2

    [1] https://github.com/fluent/fluent-plugin-rewrite-tag-filter/commit/248ed8e97def0a60121199c0d0e2fa8785f0d076
    [2] https://review.openstack.org/#/c/517907

    Depends-on: ed669e22c830d7044c99891faf8ce9ae199f63b7
    Change-Id: I0b131c2f65652cf3c61b33d8162097b047603923
    Closes-Bug: #1730664

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla-ansible 6.0.0.0b2 development milestone.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla 4.0.3 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla-ansible 4.0.3 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla-ansible 5.0.1 release.