Activity log for bug #1938670

Date Who What changed Old value New value Message
2021-08-02 05:37:08 Jake Yip bug added bug
2021-08-02 15:58:16 Jeremy Stanley bug task added ossa
2021-08-02 15:58:24 Jeremy Stanley ossa: status New Incomplete
2021-08-02 15:59:31 Jeremy Stanley description We are running an OpenStack cloud with linux bridge. We have found that, in certain conditions, ARP spoofing protection is not working as intended. This allows a user do bad things like spoof gratuitous ARP to DoS another user's virtual machine. More details below. In an environment using linux bridge, neutron-linuxbridge-agent uses ebtables to prevent ARP spoofing. A list of typical ebtables rules for a VM looks like this: :neutronMAC-tapdb545a8c-8f DROP :neutronARP-tapdb545a8c-8f DROP -A PREROUTING -i tapdb545a8c-8f -j neutronMAC-tapdb545a8c-8f -A PREROUTING -p ARP -i tapdb545a8c-8f -j neutronARP-tapdb545a8c-8f -A neutronMAC-tapdb545a8c-8f -i tapdb545a8c-8f --among-src fa:16:3e:84:cd:b4 -j RETURN -A neutronARP-tapdb545a8c-8f -p ARP --arp-ip-src 192.0.2.5 -j ACCEPT The neutronARP-xxx chain, however, has a problem during the creation of it. The source for that [1] looks like this: ebtables(['-N', vif_chain, '-P', 'DROP']) ebtables(['-F', vif_chain]) This creates a chain with default policy of DROP, and FLUSHes any existing rules. However, we have found that in certain OS, the FLUSH reverts the default policy back to RETURN. E.g. root@jake-focal:~# eatables -t nat -N newchain -P DROP root@jake-focal:~# ebtables-save | grep newchain :newchain DROP root@jake-focal:~# ebtables -t nat -F newchain root@jake-focal:~# ebtables-save | grep newchain :newchain RETURN root@jake-focal:~# ebtables --version ebtables 1.8.4 (nf_tables) The OSes that exhibit this issue seems to be OSes that uses ebtables-nft - Ubuntu Focal, CentOS Stream. Ubuntu Bionic is fine. E.g. root@jake-bionic:~# ebtables -t nat -N newchain -P DROP root@jake-bionic:~# ebtables-save | grep newchain :newchain DROP root@jake-bionic:~# ebtables -t nat -F newchain root@jake-bionic:~# ebtables-save | grep newchain :newchain DROP root@jake-bionic:~# ebtables --version ebtables v2.0.10-4 (December 2011) I have a patch for this, but as this is a security issue I am refraining from posting it up to OpenStack's Gerrit. Also, this might have been fixed in master, but it still affects Ussuri and Victoria. Please advise on what I should do next? [1] https://opendev.org/openstack/neutron/src/branch/stable/ussuri/neutron/plugins/ml2/drivers/linuxbridge/agent/arp_protect.py#L135-L139 This issue is being treated as a potential security risk under embargo. Please do not make any public mention of embargoed (private) security vulnerabilities before their coordinated publication by the OpenStack Vulnerability Management Team in the form of an official OpenStack Security Advisory. This includes discussion of the bug or associated fixes in public forums such as mailing lists, code review systems and bug trackers. Please also avoid private disclosure to other individuals not already approved for access to this information, and provide this same reminder to those who are made aware of the issue prior to publication. All discussion should remain confined to this private bug report, and any proposed fixes should be added to the bug as attachments. This embargo shall not extend past 2021-10-31 and will be made public by or on that date even if no fix is identified. We are running an OpenStack cloud with linux bridge. We have found that, in certain conditions, ARP spoofing protection is not working as intended. This allows a user do bad things like spoof gratuitous ARP to DoS another user's virtual machine. More details below. In an environment using linux bridge, neutron-linuxbridge-agent uses ebtables to prevent ARP spoofing. A list of typical ebtables rules for a VM looks like this:  :neutronMAC-tapdb545a8c-8f DROP  :neutronARP-tapdb545a8c-8f DROP  -A PREROUTING -i tapdb545a8c-8f -j neutronMAC-tapdb545a8c-8f  -A PREROUTING -p ARP -i tapdb545a8c-8f -j neutronARP-tapdb545a8c-8f  -A neutronMAC-tapdb545a8c-8f -i tapdb545a8c-8f --among-src fa:16:3e:84:cd:b4 -j RETURN  -A neutronARP-tapdb545a8c-8f -p ARP --arp-ip-src 192.0.2.5 -j ACCEPT The neutronARP-xxx chain, however, has a problem during the creation of it. The source for that [1] looks like this:  ebtables(['-N', vif_chain, '-P', 'DROP'])  ebtables(['-F', vif_chain]) This creates a chain with default policy of DROP, and FLUSHes any existing rules. However, we have found that in certain OS, the FLUSH reverts the default policy back to RETURN. E.g.  root@jake-focal:~# eatables -t nat -N newchain -P DROP  root@jake-focal:~# ebtables-save | grep newchain  :newchain DROP  root@jake-focal:~# ebtables -t nat -F newchain  root@jake-focal:~# ebtables-save | grep newchain  :newchain RETURN  root@jake-focal:~# ebtables --version  ebtables 1.8.4 (nf_tables) The OSes that exhibit this issue seems to be OSes that uses ebtables-nft - Ubuntu Focal, CentOS Stream. Ubuntu Bionic is fine. E.g.  root@jake-bionic:~# ebtables -t nat -N newchain -P DROP  root@jake-bionic:~# ebtables-save | grep newchain  :newchain DROP  root@jake-bionic:~# ebtables -t nat -F newchain  root@jake-bionic:~# ebtables-save | grep newchain  :newchain DROP  root@jake-bionic:~# ebtables --version  ebtables v2.0.10-4 (December 2011) I have a patch for this, but as this is a security issue I am refraining from posting it up to OpenStack's Gerrit. Also, this might have been fixed in master, but it still affects Ussuri and Victoria. Please advise on what I should do next? [1] https://opendev.org/openstack/neutron/src/branch/stable/ussuri/neutron/plugins/ml2/drivers/linuxbridge/agent/arp_protect.py#L135-L139
2021-08-02 15:59:59 Jeremy Stanley bug added subscriber Neutron Core Security reviewers
2021-08-03 05:30:14 Jake Yip bug added subscriber Justin Mammarella
2021-08-10 09:23:15 Rodolfo Alonso neutron: assignee Rodolfo Alonso (rodolfo-alonso-hernandez)
2021-08-10 09:23:20 Rodolfo Alonso neutron: status New Confirmed
2021-08-10 14:30:22 Jeremy Stanley description This issue is being treated as a potential security risk under embargo. Please do not make any public mention of embargoed (private) security vulnerabilities before their coordinated publication by the OpenStack Vulnerability Management Team in the form of an official OpenStack Security Advisory. This includes discussion of the bug or associated fixes in public forums such as mailing lists, code review systems and bug trackers. Please also avoid private disclosure to other individuals not already approved for access to this information, and provide this same reminder to those who are made aware of the issue prior to publication. All discussion should remain confined to this private bug report, and any proposed fixes should be added to the bug as attachments. This embargo shall not extend past 2021-10-31 and will be made public by or on that date even if no fix is identified. We are running an OpenStack cloud with linux bridge. We have found that, in certain conditions, ARP spoofing protection is not working as intended. This allows a user do bad things like spoof gratuitous ARP to DoS another user's virtual machine. More details below. In an environment using linux bridge, neutron-linuxbridge-agent uses ebtables to prevent ARP spoofing. A list of typical ebtables rules for a VM looks like this:  :neutronMAC-tapdb545a8c-8f DROP  :neutronARP-tapdb545a8c-8f DROP  -A PREROUTING -i tapdb545a8c-8f -j neutronMAC-tapdb545a8c-8f  -A PREROUTING -p ARP -i tapdb545a8c-8f -j neutronARP-tapdb545a8c-8f  -A neutronMAC-tapdb545a8c-8f -i tapdb545a8c-8f --among-src fa:16:3e:84:cd:b4 -j RETURN  -A neutronARP-tapdb545a8c-8f -p ARP --arp-ip-src 192.0.2.5 -j ACCEPT The neutronARP-xxx chain, however, has a problem during the creation of it. The source for that [1] looks like this:  ebtables(['-N', vif_chain, '-P', 'DROP'])  ebtables(['-F', vif_chain]) This creates a chain with default policy of DROP, and FLUSHes any existing rules. However, we have found that in certain OS, the FLUSH reverts the default policy back to RETURN. E.g.  root@jake-focal:~# eatables -t nat -N newchain -P DROP  root@jake-focal:~# ebtables-save | grep newchain  :newchain DROP  root@jake-focal:~# ebtables -t nat -F newchain  root@jake-focal:~# ebtables-save | grep newchain  :newchain RETURN  root@jake-focal:~# ebtables --version  ebtables 1.8.4 (nf_tables) The OSes that exhibit this issue seems to be OSes that uses ebtables-nft - Ubuntu Focal, CentOS Stream. Ubuntu Bionic is fine. E.g.  root@jake-bionic:~# ebtables -t nat -N newchain -P DROP  root@jake-bionic:~# ebtables-save | grep newchain  :newchain DROP  root@jake-bionic:~# ebtables -t nat -F newchain  root@jake-bionic:~# ebtables-save | grep newchain  :newchain DROP  root@jake-bionic:~# ebtables --version  ebtables v2.0.10-4 (December 2011) I have a patch for this, but as this is a security issue I am refraining from posting it up to OpenStack's Gerrit. Also, this might have been fixed in master, but it still affects Ussuri and Victoria. Please advise on what I should do next? [1] https://opendev.org/openstack/neutron/src/branch/stable/ussuri/neutron/plugins/ml2/drivers/linuxbridge/agent/arp_protect.py#L135-L139 We are running an OpenStack cloud with linux bridge. We have found that, in certain conditions, ARP spoofing protection is not working as intended. This allows a user do bad things like spoof gratuitous ARP to DoS another user's virtual machine. More details below. In an environment using linux bridge, neutron-linuxbridge-agent uses ebtables to prevent ARP spoofing. A list of typical ebtables rules for a VM looks like this:  :neutronMAC-tapdb545a8c-8f DROP  :neutronARP-tapdb545a8c-8f DROP  -A PREROUTING -i tapdb545a8c-8f -j neutronMAC-tapdb545a8c-8f  -A PREROUTING -p ARP -i tapdb545a8c-8f -j neutronARP-tapdb545a8c-8f  -A neutronMAC-tapdb545a8c-8f -i tapdb545a8c-8f --among-src fa:16:3e:84:cd:b4 -j RETURN  -A neutronARP-tapdb545a8c-8f -p ARP --arp-ip-src 192.0.2.5 -j ACCEPT The neutronARP-xxx chain, however, has a problem during the creation of it. The source for that [1] looks like this:  ebtables(['-N', vif_chain, '-P', 'DROP'])  ebtables(['-F', vif_chain]) This creates a chain with default policy of DROP, and FLUSHes any existing rules. However, we have found that in certain OS, the FLUSH reverts the default policy back to RETURN. E.g.  root@jake-focal:~# eatables -t nat -N newchain -P DROP  root@jake-focal:~# ebtables-save | grep newchain  :newchain DROP  root@jake-focal:~# ebtables -t nat -F newchain  root@jake-focal:~# ebtables-save | grep newchain  :newchain RETURN  root@jake-focal:~# ebtables --version  ebtables 1.8.4 (nf_tables) The OSes that exhibit this issue seems to be OSes that uses ebtables-nft - Ubuntu Focal, CentOS Stream. Ubuntu Bionic is fine. E.g.  root@jake-bionic:~# ebtables -t nat -N newchain -P DROP  root@jake-bionic:~# ebtables-save | grep newchain  :newchain DROP  root@jake-bionic:~# ebtables -t nat -F newchain  root@jake-bionic:~# ebtables-save | grep newchain  :newchain DROP  root@jake-bionic:~# ebtables --version  ebtables v2.0.10-4 (December 2011) I have a patch for this, but as this is a security issue I am refraining from posting it up to OpenStack's Gerrit. Also, this might have been fixed in master, but it still affects Ussuri and Victoria. Please advise on what I should do next? [1] https://opendev.org/openstack/neutron/src/branch/stable/ussuri/neutron/plugins/ml2/drivers/linuxbridge/agent/arp_protect.py#L135-L139
2021-08-10 14:30:29 Jeremy Stanley information type Private Security Public Security
2021-08-10 15:16:17 Jeremy Stanley ossa: status Incomplete Confirmed
2021-08-10 15:16:29 Jeremy Stanley ossa: importance Undecided High
2021-08-10 15:16:37 Jeremy Stanley ossa: assignee Jeremy Stanley (fungi)
2021-08-10 16:42:19 OpenStack Infra ossa: status Confirmed In Progress
2021-08-12 13:19:32 Jeremy Stanley summary linuxbridge: ebtables-nft allows ARP spoofing Linuxbridge ARP filter bypass on Netfilter platforms
2021-08-12 15:18:01 Jeremy Stanley summary Linuxbridge ARP filter bypass on Netfilter platforms Linuxbridge ARP filter bypass on Netfilter platforms (CVE-2021-38598)
2021-08-16 12:12:56 OpenStack Infra tags in-stable-victoria
2021-08-16 12:17:19 OpenStack Infra tags in-stable-victoria in-stable-ussuri in-stable-victoria
2021-08-16 12:17:36 OpenStack Infra tags in-stable-ussuri in-stable-victoria in-stable-train in-stable-ussuri in-stable-victoria
2021-08-16 19:37:59 Jeremy Stanley cve linked 2021-38598
2021-08-17 14:14:24 OpenStack Infra ossa: status In Progress Fix Released
2021-08-17 14:32:19 Jeremy Stanley summary Linuxbridge ARP filter bypass on Netfilter platforms (CVE-2021-38598) [OSSA-2021-004] Linuxbridge ARP filter bypass on Netfilter platforms (CVE-2021-38598)
2022-04-05 14:39:29 Slawek Kaplonski neutron: status Confirmed Fix Released