Iptables rule wrong if created a rule with protocol 4

[Phat]

I followed this document to create security group rule for requirement "Allow ingress Protocol IPIP or 4" and I used "ipip" value
https://docs.openstack.org/api-ref/network/v2/index.html#create-security-group-rule
btw I expected my iptables rule on nova compute is "-A INPUT -s 172.16.2.0/24 -p 4 -j ACCEPT" but the runtime rule in kernel was "-A INPUT -s 172.16.2.0/24 -p ipip -j ACCEPT" and proto number was 94 not 4

// the rules output for one port
Chain neutron-linuxbri-idf95737e-7 (1 references)
 pkts bytes target prot opt in out source destination
 115K 219M RETURN all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED /* Direct packets associated with a known session to the RETURN chain. */
    0 0 RETURN udp -- * * 0.0.0.0/0 172.16.2.165 udp spt:67 dpt:68
    0 0 RETURN udp -- * * 0.0.0.0/0 255.255.255.255 udp spt:67 dpt:68
    0 0 RETURN 94 -- * * 172.16.2.0/24 0.0.0.0/0
    0 0 RETURN udp -- * * 172.16.2.0/24 0.0.0.0/0 udp dpt:8472
    0 0 RETURN tcp -- * * 172.16.2.0/24 0.0.0.0/0 tcp dpt:4240
    0 0 RETURN udp -- * * 172.16.2.0/24 0.0.0.0/0 udp multiport dports 30000:32767
    1 90 RETURN tcp -- * * 172.16.2.0/24 0.0.0.0/0 tcp dpt:10250
    0 0 RETURN tcp -- * * 172.16.2.0/24 0.0.0.0/0 tcp dpt:4245
  512 30720 RETURN tcp -- * * 172.16.2.0/24 0.0.0.0/0 tcp multiport dports 30000:32767
    0 0 RETURN icmp -- * * 172.16.2.0/24 0.0.0.0/0
    0 0 RETURN udp -- * * 172.16.2.0/24 0.0.0.0/0 udp dpt:4789
    0 0 RETURN tcp -- * * 172.16.2.0/24 0.0.0.0/0 tcp dpt:4244
    2 142 RETURN tcp -- * * 172.16.2.0/24 0.0.0.0/0 tcp dpt:179
    0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 state INVALID /* Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack. */
   14 2122 neutron-linuxbri-sg-fallback all -- * * 0.0.0.0/0 0.0.0.0/0 /* Send unmatched traffic to the fallback chain. */

[Brian Haley (brian-haley)]

So this is one of those confusing edge cases and I think Linux is conspiring against us. Let me explain.

1) neutron-lib does correctly define the protocol name 'ipip' as 4.

2) The linux kernel uses the same in in.h:

IPPROTO_IPIP = 4
IPPROTO_BEETPH = 94 (?)

3) iptables maps 'ipip' to 94 and 'ipencap' to 4.

# for num in {0..255}; do iptables -A INPUT -p $num; done
# iptables-save | grep -E 'ipip|ipencap'
-A INPUT -p ipencap
-A INPUT -p ipip

4) /etc/protocols does the same as iptables:

grep -E 'ipencap|ipip' /etc/protocols
ipencap 4 IP-ENCAP # IP encapsulated in IP (officially ``IP'')
ipip 94 IPIP # IP-within-IP Encapsulation Protocol

5) getprotoby{name|number} does what /etc/protocols does:

$ getprotobyname ipip
struct protoent: (0x7fbbbcca9c60)
  p_name ipip
  p_aliases IPIP
  p_proto 94

$ getprotobynumber 4
struct protoent: (0x7fc51ad86be0)
  p_name ipencap
  p_aliases IP-ENCAP
  p_proto 4

Neutron actually builds a mapping based on the getprotoby* calls, so in the iptables case it winds-up doing the wrong thing (it's actually more complicated than that, but...)

I'm assuming the value in the DB is actually 4, correct? We just maybe need to do a little trickery when dealing with iptables-save based on #3 above.

As a workaround please use 4, I think that will do the correct thing, right?

[Phat (letonphat1988)]

Hi Brian Haley,
Thank you for your explain, it's very detail. so I've applied workaround use "4" bypass this issue before. BTW in the DB is actually is a string "ipip" not "4" that why in iptables, i saw the wrong rule
    `-A neutron-linuxbri-i23bbded3-2 -s 172.16.2.0/24 -p ipip -j RETURN`
The iptables translated the ipip which set from neutron to 94. I'm assuming the api accepted and stored this "ipip" value.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-lib/+/909942

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/909943

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/909943
Committed: https://opendev.org/openstack/neutron/commit/793dfb04d087c96470dc4913eb4f17aeff777534
Submitter: "Zuul (22348)"
Branch: master

commit 793dfb04d087c96470dc4913eb4f17aeff777534
Author: Brian Haley <email address hidden>
Date: Thu Feb 22 18:33:14 2024 -0500

    Fix iptables mapping of 'ipip' protocol

    Map 'ipip' to use the string 'ipencap' so the
    IptablesFirewallDriver class in neutron works correctly.
    Once neutron-lib is bumped this can be removed.

    Add tests for IP protocol 'ipip', '4' and '94' to make
    sure the IptablesFirewallDriver class in neutron treats
    them correctly.

    Long description below.

    This is one of those confusing edge cases and I think
    Linux is conspiring against us. Let me explain.

    1) neutron-lib does correctly define the protocol name 'ipip' as 4.

    2) The linux kernel uses the same in in.h:

     IPPROTO_IPIP = 4
     IPPROTO_BEETPH = 94 (?)

    3) iptables maps 'ipip' to 94 and 'ipencap' to 4.

     # for num in {0..255}; do iptables -A INPUT -p $num; done
     # iptables-save | grep -E 'ipip|ipencap'
     -A INPUT -p ipencap
     -A INPUT -p ipip

    4) /etc/protocols does the same as iptables:

     grep -E 'ipencap|ipip' /etc/protocols
     ipencap 4 IP-ENCAP # IP encapsulated in IP (officially ``IP'')
     ipip 94 IPIP # IP-within-IP Encapsulation Protocol

    5) getprotoby{name|number} does what /etc/protocols does:

     $ getprotobyname ipip
     struct protoent: (0x7fbbbcca9c60)
       p_name ipip
       p_aliases IPIP
       p_proto 94

     $ getprotobynumber 4
     struct protoent: (0x7fc51ad86be0)
       p_name ipencap
       p_aliases IP-ENCAP
       p_proto 4

    Neutron actually builds a mapping based on the getprotoby*
    calls, so in the iptables case it winds-up doing the wrong
    thing.

    Partial-bug: #2054324
    Change-Id: Icc84b54be07d39059723d6c233c03aa130102423

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/2023.2
Review: https://review.opendev.org/c/openstack/neutron/+/910911

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/neutron/+/910912

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/zed
Review: https://review.opendev.org/c/openstack/neutron/+/910913

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/911154

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/911154
Committed: https://opendev.org/openstack/neutron/commit/cd1d191e335dca707ac9324fd81e642cb453a6cf
Submitter: "Zuul (22348)"
Branch: master

commit cd1d191e335dca707ac9324fd81e642cb453a6cf
Author: Brian Haley <email address hidden>
Date: Tue Mar 5 11:59:05 2024 -0500

    Use the system-dependent string for IP protocol 4

    iptables-save uses a system-dependent value, usually that
    found in /etc/protocols, when 'ipip' is given as the
    security group protocol. The intent is to always use the
    string value for IP protocol '4', as iptables-save has no
    '-n' flag to print values numerically.

    This updates a previous change (793dfb04d) that hard-coded
    that string to 'ipencap', which broke CentOS/Fedora, which
    uses 'ipv4'.

    For this reason we cannot hard-code anything in neutron-lib,
    this needs to be added dynamically, so this one-line change
    needs to stay here, and effectively closes the bug.

    Closes-bug: #2054324
    Change-Id: Ic40b539c9ef5cfa4cbbd6575e19e653342e8342b

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Brian Haley <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron-lib/+/909942

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/2023.2
Review: https://review.opendev.org/c/openstack/neutron/+/911604

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/neutron/+/911605

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Brian Haley <email address hidden>" on branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/neutron/+/910912
Reason: Need to do this one again

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Brian Haley <email address hidden>" on branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/neutron/+/911605

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/zed
Review: https://review.opendev.org/c/openstack/neutron/+/911606

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 24.0.0.0rc1 release candidate.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/910911
Committed: https://opendev.org/openstack/neutron/commit/44c113651b493544b25cd432f1265be5d0897dca
Submitter: "Zuul (22348)"
Branch: stable/2023.2

commit 44c113651b493544b25cd432f1265be5d0897dca
Author: Brian Haley <email address hidden>
Date: Thu Feb 22 18:33:14 2024 -0500

    Fix iptables mapping of 'ipip' protocol

    Map 'ipip' to use the string 'ipencap' so the
    IptablesFirewallDriver class in neutron works correctly.
    Once neutron-lib is bumped this can be removed.

    Add tests for IP protocol 'ipip', '4' and '94' to make
    sure the IptablesFirewallDriver class in neutron treats
    them correctly.

    Long description below.

    This is one of those confusing edge cases and I think
    Linux is conspiring against us. Let me explain.

    1) neutron-lib does correctly define the protocol name 'ipip' as 4.

    2) The linux kernel uses the same in in.h:

     IPPROTO_IPIP = 4
     IPPROTO_BEETPH = 94 (?)

    3) iptables maps 'ipip' to 94 and 'ipencap' to 4.

     # for num in {0..255}; do iptables -A INPUT -p $num; done
     # iptables-save | grep -E 'ipip|ipencap'
     -A INPUT -p ipencap
     -A INPUT -p ipip

    4) /etc/protocols does the same as iptables:

     grep -E 'ipencap|ipip' /etc/protocols
     ipencap 4 IP-ENCAP # IP encapsulated in IP (officially ``IP'')
     ipip 94 IPIP # IP-within-IP Encapsulation Protocol

    5) getprotoby{name|number} does what /etc/protocols does:

     $ getprotobyname ipip
     struct protoent: (0x7fbbbcca9c60)
       p_name ipip
       p_aliases IPIP
       p_proto 94

     $ getprotobynumber 4
     struct protoent: (0x7fc51ad86be0)
       p_name ipencap
       p_aliases IP-ENCAP
       p_proto 4

    Neutron actually builds a mapping based on the getprotoby*
    calls, so in the iptables case it winds-up doing the wrong
    thing.

    Partial-bug: #2054324
    Change-Id: Icc84b54be07d39059723d6c233c03aa130102423
    (cherry picked from commit 793dfb04d087c96470dc4913eb4f17aeff777534)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/911604
Committed: https://opendev.org/openstack/neutron/commit/6925391e96cde7c4fe27f782f19a64857f52f7c5
Submitter: "Zuul (22348)"
Branch: stable/2023.2

commit 6925391e96cde7c4fe27f782f19a64857f52f7c5
Author: Brian Haley <email address hidden>
Date: Tue Mar 5 11:59:05 2024 -0500

    Use the system-dependent string for IP protocol 4

    iptables-save uses a system-dependent value, usually that
    found in /etc/protocols, when 'ipip' is given as the
    security group protocol. The intent is to always use the
    string value for IP protocol '4', as iptables-save has no
    '-n' flag to print values numerically.

    This updates a previous change (793dfb04d) that hard-coded
    that string to 'ipencap', which broke CentOS/Fedora, which
    uses 'ipv4'.

    For this reason we cannot hard-code anything in neutron-lib,
    this needs to be added dynamically, so this one-line change
    needs to stay here, and effectively closes the bug.

    Closes-bug: #2054324
    Change-Id: Ic40b539c9ef5cfa4cbbd6575e19e653342e8342b
    (cherry picked from commit cd1d191e335dca707ac9324fd81e642cb453a6cf)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/910913
Committed: https://opendev.org/openstack/neutron/commit/7c7b020abe15fb7086dcca9095cddb964f03eab6
Submitter: "Zuul (22348)"
Branch: stable/zed

commit 7c7b020abe15fb7086dcca9095cddb964f03eab6
Author: Brian Haley <email address hidden>
Date: Thu Feb 22 18:33:14 2024 -0500

    Fix iptables mapping of 'ipip' protocol

    Map 'ipip' to use the string 'ipencap' so the
    IptablesFirewallDriver class in neutron works correctly.
    Once neutron-lib is bumped this can be removed.

    Add tests for IP protocol 'ipip', '4' and '94' to make
    sure the IptablesFirewallDriver class in neutron treats
    them correctly.

    Long description below.

    This is one of those confusing edge cases and I think
    Linux is conspiring against us. Let me explain.

    1) neutron-lib does correctly define the protocol name 'ipip' as 4.

    2) The linux kernel uses the same in in.h:

     IPPROTO_IPIP = 4
     IPPROTO_BEETPH = 94 (?)

    3) iptables maps 'ipip' to 94 and 'ipencap' to 4.

     # for num in {0..255}; do iptables -A INPUT -p $num; done
     # iptables-save | grep -E 'ipip|ipencap'
     -A INPUT -p ipencap
     -A INPUT -p ipip

    4) /etc/protocols does the same as iptables:

     grep -E 'ipencap|ipip' /etc/protocols
     ipencap 4 IP-ENCAP # IP encapsulated in IP (officially ``IP'')
     ipip 94 IPIP # IP-within-IP Encapsulation Protocol

    5) getprotoby{name|number} does what /etc/protocols does:

     $ getprotobyname ipip
     struct protoent: (0x7fbbbcca9c60)
       p_name ipip
       p_aliases IPIP
       p_proto 94

     $ getprotobynumber 4
     struct protoent: (0x7fc51ad86be0)
       p_name ipencap
       p_aliases IP-ENCAP
       p_proto 4

    Neutron actually builds a mapping based on the getprotoby*
    calls, so in the iptables case it winds-up doing the wrong
    thing.

    Partial-bug: #2054324
    Change-Id: Icc84b54be07d39059723d6c233c03aa130102423
    (cherry picked from commit 793dfb04d087c96470dc4913eb4f17aeff777534)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/911606
Committed: https://opendev.org/openstack/neutron/commit/386ebde547419a3cd2ba9e2228da000cf43bfa17
Submitter: "Zuul (22348)"
Branch: stable/zed

commit 386ebde547419a3cd2ba9e2228da000cf43bfa17
Author: Brian Haley <email address hidden>
Date: Tue Mar 5 11:59:05 2024 -0500

    Use the system-dependent string for IP protocol 4

    iptables-save uses a system-dependent value, usually that
    found in /etc/protocols, when 'ipip' is given as the
    security group protocol. The intent is to always use the
    string value for IP protocol '4', as iptables-save has no
    '-n' flag to print values numerically.

    This updates a previous change (793dfb04d) that hard-coded
    that string to 'ipencap', which broke CentOS/Fedora, which
    uses 'ipv4'.

    For this reason we cannot hard-code anything in neutron-lib,
    this needs to be added dynamically, so this one-line change
    needs to stay here, and effectively closes the bug.

    Closes-bug: #2054324
    Change-Id: Ic40b539c9ef5cfa4cbbd6575e19e653342e8342b
    (cherry picked from commit cd1d191e335dca707ac9324fd81e642cb453a6cf)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/910912
Committed: https://opendev.org/openstack/neutron/commit/d17b1a6abbd0d4e68ce305c54f17e90e547f1ba8
Submitter: "Zuul (22348)"
Branch: stable/2023.1

commit d17b1a6abbd0d4e68ce305c54f17e90e547f1ba8
Author: Brian Haley <email address hidden>
Date: Thu Feb 22 18:33:14 2024 -0500

    Fix iptables mapping of 'ipip' protocol

    Map 'ipip' to use the string 'ipencap' so the
    IptablesFirewallDriver class in neutron works correctly.
    Once neutron-lib is bumped this can be removed.

    Add tests for IP protocol 'ipip', '4' and '94' to make
    sure the IptablesFirewallDriver class in neutron treats
    them correctly.

    Long description below.

    This is one of those confusing edge cases and I think
    Linux is conspiring against us. Let me explain.

    1) neutron-lib does correctly define the protocol name 'ipip' as 4.

    2) The linux kernel uses the same in in.h:

     IPPROTO_IPIP = 4
     IPPROTO_BEETPH = 94 (?)

    3) iptables maps 'ipip' to 94 and 'ipencap' to 4.

     # for num in {0..255}; do iptables -A INPUT -p $num; done
     # iptables-save | grep -E 'ipip|ipencap'
     -A INPUT -p ipencap
     -A INPUT -p ipip

    4) /etc/protocols does the same as iptables:

     grep -E 'ipencap|ipip' /etc/protocols
     ipencap 4 IP-ENCAP # IP encapsulated in IP (officially ``IP'')
     ipip 94 IPIP # IP-within-IP Encapsulation Protocol

    5) getprotoby{name|number} does what /etc/protocols does:

     $ getprotobyname ipip
     struct protoent: (0x7fbbbcca9c60)
       p_name ipip
       p_aliases IPIP
       p_proto 94

     $ getprotobynumber 4
     struct protoent: (0x7fc51ad86be0)
       p_name ipencap
       p_aliases IP-ENCAP
       p_proto 4

    Neutron actually builds a mapping based on the getprotoby*
    calls, so in the iptables case it winds-up doing the wrong
    thing.

    Partial-bug: #2054324
    Change-Id: Icc84b54be07d39059723d6c233c03aa130102423
    (cherry picked from commit 793dfb04d087c96470dc4913eb4f17aeff777534)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 21.2.1 release.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/911605
Committed: https://opendev.org/openstack/neutron/commit/10af328885ac0f85f161c454dd53463cee57bb23
Submitter: "Zuul (22348)"
Branch: stable/2023.1

commit 10af328885ac0f85f161c454dd53463cee57bb23
Author: Brian Haley <email address hidden>
Date: Tue Mar 5 11:59:05 2024 -0500

    Use the system-dependent string for IP protocol 4

    iptables-save uses a system-dependent value, usually that
    found in /etc/protocols, when 'ipip' is given as the
    security group protocol. The intent is to always use the
    string value for IP protocol '4', as iptables-save has no
    '-n' flag to print values numerically.

    This updates a previous change (793dfb04d) that hard-coded
    that string to 'ipencap', which broke CentOS/Fedora, which
    uses 'ipv4'.

    For this reason we cannot hard-code anything in neutron-lib,
    this needs to be added dynamically, so this one-line change
    needs to stay here, and effectively closes the bug.

    Closes-bug: #2054324
    Change-Id: Ic40b539c9ef5cfa4cbbd6575e19e653342e8342b
    (cherry picked from commit cd1d191e335dca707ac9324fd81e642cb453a6cf)