Checksum drop of metadata traffic on isolated networks with DPDK

Bug #1832021 reported by David Ames
22
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack neutron-openvswitch charm
Undecided
Erlon R. Cruz
Ubuntu Cloud Archive
Medium
Erlon R. Cruz
Queens
Medium
Erlon R. Cruz
Rocky
Medium
Erlon R. Cruz
Stein
Medium
Erlon R. Cruz
neutron
Medium
Alexander Vlasov
neutron (Ubuntu)
Medium
Unassigned
Bionic
Medium
Unassigned
Focal
Medium
Unassigned

Bug Description

[Impact]

When an isolated network using provider networks for tenants (meaning without virtual routers: DVR or network node), metadata access occurs in the qdhcp ip netns rather than the qrouter netns.

The following options are set in the dhcp_agent.ini file:
force_metadata = True
enable_isolated_metadata = True

VMs on the provider tenant network are unable to access metadata as packets are dropped due to checksum.

[Test Plan]

1. Create an OpenStack deployment with DPDK options enabled and 'enable-local-dhcp-and-metadata: true' in neutron-openvswitch. A sample, simple 3 node bundle can be found here[1].

2. Create an external flat network and subnet:

openstack network show dpdk_net || \
  openstack network create --provider-network-type flat \
                           --provider-physical-network physnet1 dpdk_net \
                           --external

openstack subnet show dpdk_net || \
    openstack subnet create --allocation-pool start=10.230.58.100,end=10.230.58.200 \
                            --subnet-range 10.230.56.0/21 --dhcp --gateway 10.230.56.1 \
                            --dns-nameserver 10.230.56.2 \
                            --ip-version 4 --network dpdk_net dpdk_subnet

3. Create an instance attached to that network. The instance must have a flavor that uses huge pages.

openstack flavor create --ram 8192 --disk 50 --vcpus 4 m1.dpdk
openstack flavor set m1.dpdk --property hw:mem_page_size=large

openstack server create --wait --image xenial --flavor m1.dpdk --key-name testkey --network dpdk_net i1

4. Log into the instance host and check the instance console. The instance will hang into the boot and show the following message:

2020-11-20 09:43:26,790 - openstack.py[DEBUG]: Failed reading optional path http://169.254.169.254/openstack/2015-10-15/user_data due to: HTTPConnectionPool(host='169.254.169.254', port=80): Read timed out. (read timeout=10.0)

5. Apply the fix in all computes, restart the DHCP agents in all computes and create the instance again.

6. No errors should be shown and the instance quickly boots.

[Where problems could occur]

* This change is only touched if datapath_type and ovs_use_veth. Those settings are mostly used for DPDK environments. The core of the fix is
to toggle off checksum offload done by the DHCP namespace interfaces.
This will have the drawback of adding some overhead on the packet processing for DHCP traffic but given DHCP does not demand too much data, this should be a minor proble.

* Future changes on the syntax of the ethtool command could cause regressions

[Other Info]

 * None

BIONIC VERIFICATION DONE

1 - Followed through the process above and confirmed that after installing the package the problems are solved.
2 - Testing output can be seen here: https://paste.ubuntu.com/p/mM22nBsSG2/

ROCKY VERIFICATION DONE

1 - Followed through the process above and confirmed that after installing the package the problems are solved.
2 - Testing output can be seen here: https://paste.ubuntu.com/p/qKcdWMxnsJ/

[1] https://gist.github.com/sombrafam/e0741138773e444960eb4aeace6e3e79

Jeff Hillman (jhillman)
tags: added: cpe-onsite
Revision history for this message
Brian Haley (brian-haley) wrote :

David - please see the link that was the reason I reverted this change, https://lore.kernel.org/patchwork/patch/824819/ - that is basically saying this rule has no effect for TCP, it was only meant for UDP, and was finally changed to log a warning in the kernel.

There is probably something else going on here causing issues, possibly outside of neutron.

Revision history for this message
David Ames (thedac) wrote :

Brian,

Thanks for getting back to me. It seems this is a duplicate of LP Bug #1722584 [0]. And the explanation for my running into it is that we have not yet pushed your reversion into our Ubuntu packaging.

Marking this bug a duplicate of LP Bug #1722584

[0] https://bugs.launchpad.net/cloud-archive/+bug/1722584

James Page (james-page)
Changed in neutron:
status: New → Incomplete
Revision history for this message
David Ames (thedac) wrote :
Download full text (3.2 KiB)

We de-duplicated this bug as we have narrowed the focus. This is DPDK specific.

When using isolated provider networks AND DPDK metadata is dropped due to incorrect TCP checksum. Specifically when the provider interface is a DPDK interface.

It is temporarily mitigated by adding the iptables rule:
iptables -t mangle -A OUTPUT -o ns-+ -p tcp --sport 80 -j CHECKSUM --checksum-fill
However this is not sustainable as any restart of openvswitch will clear this setting.

Here is an example of a tcpdump in the qdhcp netns:

    172.20.0.23.50060 > 169.254.169.254.80: Flags [S], cksum 0xb3c1 (correct), seq 3293184694, win 29200, options [mss 1460,sackOK,TS val 78090881 ecr 0,nop,wscale 7], length 0
23:36:43.932728 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60)

    169.254.169.254.80 > 172.20.0.23.50060: Flags [S.], cksum 0x0057 (incorrect -> 0x2b02), seq 2915867972, ack 3293184695, win 28960, options [mss 1460,sackOK,TS val 4009971593 ecr 78090881,nop,wscale 7], length 0

Continuing with re-transmissions of the same. Note the incorrect cksum.

With the mangle rule in place:

    172.20.0.4.46706 > 169.254.169.254.80: Flags [S], cksum 0xbf25 (correct), seq 4115688639, win 29200, options [mss 1460,sackOK,TS val 2390126443 ecr 0,nop,wscale
7], length 0
23:40:01.510745 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60)
    169.254.169.254.80 > 172.20.0.4.46706: Flags [S.], cksum 0xe3f3 (correct), seq 1829822633, ack 4115688640, win 28960, options [mss 1460,sackOK,TS val 812998113 e
cr 2390126443,nop,wscale 7], length 0
23:40:01.510919 IP (tos 0x0, ttl 64, id 38572, offset 0, flags [DF], proto TCP (6), length 52)
    172.20.0.4.46706 > 169.254.169.254.80: Flags [.], cksum 0x82fb (correct), seq 1, ack 1, win 229, options [nop,nop,TS val 2390126443 ecr 812998113], length 0
23:40:01.510974 IP (tos 0x0, ttl 64, id 38573, offset 0, flags [DF], proto TCP (6), length 229)
    172.20.0.4.46706 > 169.254.169.254.80: Flags [P.], cksum 0x5e40 (correct), seq 1:178, ack 1, win 229, options [nop,nop,TS val 2390126443 ecr 812998113], length 1
77: HTTP, length: 177
        GET /openstack HTTP/1.1
        Host: 169.254.169.254
        User-Agent: Cloud-Init/19.1-1-gbaa47854-0ubuntu1~18.04.1
        Accept-Encoding: gzip, deflate
        Accept: */*
        Connection: keep-alive

23:40:01.553043 IP (tos 0x0, ttl 64, id 62471, offset 0, flags [DF], proto TCP (6), length 52)
    169.254.169.254.80 > 172.20.0.4.46706: Flags [.], cksum 0x8219 (correct), seq 1, ack 178, win 235, options [nop,nop,TS val 812998156 ecr 2390126443], length 0
23:40:02.036984 IP (tos 0x0, ttl 64, id 62472, offset 0, flags [DF], proto TCP (6), length 252)
    169.254.169.254.80 > 172.20.0.4.46706: Flags [P.], cksum 0xd303 (correct), seq 1:201, ack 178, win 235, options [nop,nop,TS val 812998640 ecr 2390126443], length
 200: HTTP, length: 200
        HTTP/1.1 200 OK
        Content-Type: text/plain; charset=UTF-8
        Content-Length: 83
        Date: Thu, 13 Jun 2019 23:40:02 GMT

        2012-08-10
        2013-04-04
        2013-10-17
        2015-10-15
        2016-06-30
        2016-10-06
        2017-02-22
        latest[!http]

We can provi...

Read more...

David Ames (thedac)
Changed in neutron:
status: Incomplete → New
David Ames (thedac)
summary: - Checksum drop of metadata traffic on isolated provider networks
+ Checksum drop of metadata traffic on isolated provider networks with
+ DPDK
Revision history for this message
David Ames (thedac) wrote : Re: Checksum drop of metadata traffic on isolated provider networks with DPDK

Further testing shows the provider network is irrelevant. With DPDK and an isolated network (qdhcp only no qrouter) either GRE or provider, any traffic initiated by the qdhcp netns, including response traffic, gets an incorrect TCP checksum.

This packet gets put on the "wire" and it is the VM that drops the packet due to an invalid TCP checksum.

In a DPDK isolated network environment from the qdhcp netns you can see this in action with an arbitrary netcat call:

nc -vz $VM_IP 73 (Any TCP port)

tcpdump on the VM side and you can see

22:06:31.424716 IP (tos 0x0, ttl 64, id 14532, offset 0, flags [DF], proto TCP (6), length 60)
    172.20.0.2.39784 > 172.20.0.6.73: Flags [S], cksum 0x585f (incorrect -> 0x0437), seq 114680395, win 26880, options [mss 8960,sackOK,TS val 1660321155 ecr 0,nop,wscale 7], length 0
22:06:39.616633 IP (tos 0x0, ttl 64, id 14533, offset 0, flags [DF], proto TCP (6), length 60)
    172.20.0.2.39784 > 172.20.0.6.73: Flags [S], cksum 0x585f (incorrect -> 0xe436), seq 114680395, win 26880, options [mss 8960,sackOK,TS val 1660329347 ecr 0,nop,wscale 7], length 0
22:06:55.744502 IP (tos 0x0, ttl 64, id 14534, offset 0, flags [DF], proto TCP (6), length 60)
    172.20.0.2.39784 > 172.20.0.6.73: Flags [S], cksum 0x585f (incorrect -> 0xa536), seq 114680395, win 26880, options [mss 8960,sackOK,TS val 1660345475 ecr 0,nop,wscale 7], length 0

So the VM sees the response traffic from the qdhcp netns but drops it because the TCP checksum is invalid.

When we turn on DVR and create a virtual router (unused) the qrouter netns does not have this problem. I have not root caused why but there are a number of other iptables settings in the qrouter netns that are not in the qdhcp that may be required.

So what we are looking for is differences in the setup of the qdhcp netns from the qrouter netns.

summary: - Checksum drop of metadata traffic on isolated provider networks with
- DPDK
+ Checksum drop of metadata traffic on isolated networks with DPDK
Revision history for this message
David Ames (thedac) wrote :

Also, note the qrouter nets works with or without the neutron-l3-agent-POSTROUTING checksum fill rule. This is important as this checksum fill was removed from the neutron code.

Miguel Lavalle (minsel)
Changed in neutron:
importance: Undecided → Medium
Revision history for this message
David Ames (thedac) wrote :

A bit more research. I tried to find differences in iptables rules between qdhcp and qrouter netns-es.

I went so far as to restore iptables-save of qrouter in the qdhcp with no change.

Any TCP connection (or response) in the qdhcp netns generates an invalid TCP checksum.

Revision history for this message
Liam Young (gnuoy) wrote :

The issue appears to be that ovs_use_veth=True was set to true in dhcp_agent.ini. This causes the qdhcp namespace to be connected to the bridge via a veth pair. This appears to leave checksum offloading enabled on the device in the qdhcp namespace. Manually turning off tx-checksum-ip-generic appears to fix the issue but for a permanent fix switch ovs_use_veth to False.

Revision history for this message
Brian Haley (brian-haley) wrote :

Liam - thanks for the information. Since the default value for ovs_use_veth is False in the neutron tree, was it the installer tools that changed the value?

Revision history for this message
Liam Young (gnuoy) wrote :

Hi Brian, yes it was, I'll be proposing a change to fix that today.

Revision history for this message
Slawek Kaplonski (slaweq) wrote :

Maybe we should add some warning about it to our docs too?

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/666277

Revision history for this message
Nivedita Singhvi (niveditasinghvi) wrote :

Liam, can you clarify what change you're proposing and where?

I'm assuming setting ovs_use_veth=false in dhcp_agent.ini by default?

(and not disabling checksumming somehow).

Revision history for this message
Liam Young (gnuoy) wrote :

Hi niveditasinghvi. Yes, I'm proposing ovs_use_veth be set to False rather than manually disabling checksumming. The changes are here: https://review.opendev.org/#/q/topic:bug/1832021+(status:open+OR+status:merged). We are having a few issue with CI but I hope to have them landed asap.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (master)

Reviewed: https://review.opendev.org/666277
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=051b58f566dd382f22df937d2e90c06e9ca2faad
Submitter: Zuul
Branch: master

commit 051b58f566dd382f22df937d2e90c06e9ca2faad
Author: Slawek Kaplonski <email address hidden>
Date: Wed Jun 19 14:27:18 2019 +0200

    Update DPDK docs with note about using veth pairs

    In case when ovs-dpdk is used together with ``ovs_use_veth`` config
    option set to True, it cause invalid checksum on all packets send from
    qdhcp namespace.
    This commit adds short info about this limitation to ovs-dpdk config
    guide.

    Change-Id: I6237abab3d9e625440e95e75f5091d09a1ec44f0
    Related-Bug: #1832021

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-neutron-openvswitch (master)

Reviewed: https://review.opendev.org/666293
Committed: https://git.openstack.org/cgit/openstack/charm-neutron-openvswitch/commit/?id=7578326c592a81e6cc6737f360fc2b15ed175cf0
Submitter: Zuul
Branch: master

commit 7578326c592a81e6cc6737f360fc2b15ed175cf0
Author: Liam Young <email address hidden>
Date: Wed Jun 19 13:29:18 2019 +0000

    Stop using veth pairs to connect qdhcp ns

    veth pairs are currently being used to connect the qdhcp namespace
    to the underlying bridge. This behaviour appears to only be needed
    for old kernels with limited namespaces support (pre trusty).

    Change-Id: If1f669de09e2499e74e88e2b72203047e7f9f957
    Closes-Bug: #1832021

Changed in charm-neutron-openvswitch:
status: New → Fix Committed
Revision history for this message
Andre Ruiz (andre-ruiz) wrote :

Just a late note: the upgrade was applied to the customer in question and it indeed fixed the problem.

charm neutron-openvswitch-next-359 -> neutron-openvswitch-next-367

James Page (james-page)
Changed in charm-neutron-openvswitch:
milestone: none → 19.07
David Ames (thedac)
Changed in charm-neutron-openvswitch:
status: Fix Committed → Fix Released
Changed in neutron:
assignee: nobody → Alexander Vlasov (avlasov)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.opendev.org/713719
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=11838a2bc50caa40e776bce211f5f2d1d16a14af
Submitter: Zuul
Branch: master

commit 11838a2bc50caa40e776bce211f5f2d1d16a14af
Author: Alexander Vlasov <email address hidden>
Date: Wed Mar 18 13:35:20 2020 -0500

    Workaround for TCP checksum issue with ovs-dpdk and veth pair

    The need for this change stems from following issues:
    1) When ovs_use_veth = False with ovs-dpdk issue with ovs
    was observed - after vswitch restart interface is not comming up.
    Meaning ovs-dpdk uses ovs internal ports and it is not able to bring
    them up on restart.
    2) When ovs_use_veth = True and ovs-dpkd is used, packets sent with
    incorrect checksum due to the fact that ovs-dpdk does not do checksum
    calculations for veth interface.

    This commit allows to use second option and resolve checksum issue by
    disabling checksum offload.

    Closes-Bug: #1832021
    Related-Bug: #1831935

    Change-Id: Iecce8d2c6c2c46718cc1020c6e8f914cd4560e4b

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/738483

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/738485

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/train)

Reviewed: https://review.opendev.org/738485
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=85f95e0bd9a80b952c6591119372d24b79ac24d2
Submitter: Zuul
Branch: stable/train

commit 85f95e0bd9a80b952c6591119372d24b79ac24d2
Author: Alexander Vlasov <email address hidden>
Date: Wed Mar 18 13:35:20 2020 -0500

    Workaround for TCP checksum issue with ovs-dpdk and veth pair

    The need for this change stems from following issues:
    1) When ovs_use_veth = False with ovs-dpdk issue with ovs
    was observed - after vswitch restart interface is not comming up.
    Meaning ovs-dpdk uses ovs internal ports and it is not able to bring
    them up on restart.
    2) When ovs_use_veth = True and ovs-dpkd is used, packets sent with
    incorrect checksum due to the fact that ovs-dpdk does not do checksum
    calculations for veth interface.

    This commit allows to use second option and resolve checksum issue by
    disabling checksum offload.

    Closes-Bug: #1832021
    Related-Bug: #1831935

    Change-Id: Iecce8d2c6c2c46718cc1020c6e8f914cd4560e4b
    (cherry picked from commit 11838a2bc50caa40e776bce211f5f2d1d16a14af)

tags: added: in-stable-train
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/ussuri)

Reviewed: https://review.opendev.org/738483
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=e79afe115cb4db588fdd6bbd7500775173fd88c5
Submitter: Zuul
Branch: stable/ussuri

commit e79afe115cb4db588fdd6bbd7500775173fd88c5
Author: Alexander Vlasov <email address hidden>
Date: Wed Mar 18 13:35:20 2020 -0500

    Workaround for TCP checksum issue with ovs-dpdk and veth pair

    The need for this change stems from following issues:
    1) When ovs_use_veth = False with ovs-dpdk issue with ovs
    was observed - after vswitch restart interface is not comming up.
    Meaning ovs-dpdk uses ovs internal ports and it is not able to bring
    them up on restart.
    2) When ovs_use_veth = True and ovs-dpkd is used, packets sent with
    incorrect checksum due to the fact that ovs-dpdk does not do checksum
    calculations for veth interface.

    This commit allows to use second option and resolve checksum issue by
    disabling checksum offload.

    Closes-Bug: #1832021
    Related-Bug: #1831935

    Change-Id: Iecce8d2c6c2c46718cc1020c6e8f914cd4560e4b
    (cherry picked from commit 11838a2bc50caa40e776bce211f5f2d1d16a14af)

tags: added: in-stable-ussuri
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/stein)

Reviewed: https://review.opendev.org/713717
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=81dfa48466a594da78d93494e805b9ba97f94bc6
Submitter: Zuul
Branch: stable/stein

commit 81dfa48466a594da78d93494e805b9ba97f94bc6
Author: Alexander Vlasov <email address hidden>
Date: Wed Mar 18 13:35:20 2020 -0500

    Workaround for TCP checksum issue with ovs-dpdk and veth pair

    The need for this change stems from following issues:
    1) When ovs_use_veth = False with ovs-dpdk issue with ovs
    was observed - after vswitch restart interface is not comming up.
    Meaning ovs-dpdk uses ovs internal ports and it is not able to bring
    them up on restart.
    2) When ovs_use_veth = True and ovs-dpkd is used, packets sent with
    incorrect checksum due to the fact that ovs-dpdk does not do checksum
    calculations for veth interface.

    This commit allows to use second option and resolve checksum issue by
    disabling checksum offload.

    Closes-Bug: #1832021
    Related-Bug: #1831935

    Change-Id: Iecce8d2c6c2c46718cc1020c6e8f914cd4560e4b
    (cherry picked from commit 11838a2bc50caa40e776bce211f5f2d1d16a14af)

tags: added: in-stable-stein
Revision history for this message
Rahmat Agung Wibowo (riupie) wrote :

Hi, I have same issue on OVN, is there any bug fix for this on Openstack Ussuri

description: updated
Changed in charm-neutron-openvswitch:
assignee: nobody → Erlon R. Cruz (sombrafam)
Changed in cloud-archive:
assignee: nobody → Erlon R. Cruz (sombrafam)
Changed in cloud-archive:
importance: Undecided → Medium
Revision history for this message
Erlon R. Cruz (sombrafam) wrote :

Attached debdiff ready for sru cloud review.

Revision history for this message
Erlon R. Cruz (sombrafam) wrote :

I think I don't have the permissions to properly set the revisions if someone can set it, please.
This should be added to:
  - UCA: queens
  - Distro: Bionic (queens)

tags: added: sts-sru-needed
Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote :

The attachment "neutron-dpdk-fix.debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

tags: added: patch
Mathew Hodson (mhodson)
Changed in neutron (Ubuntu):
importance: Undecided → Medium
Changed in neutron (Ubuntu Bionic):
importance: Undecided → Medium
Revision history for this message
Mathew Hodson (mhodson) wrote :

Fixed in Ubuntu Focal.

---
neutron (2:16.1.0-0ubuntu1) focal; urgency=medium

  * New stable point release for OpenStack Ussuri (LP: #1892139).
  * d/control: Align (Build-)Depends with upstream.

 -- Chris MacNaughton <email address hidden> Thu, 27 Aug 2020 05:31:05 +0000

Changed in neutron (Ubuntu Focal):
importance: Undecided → Medium
status: New → Fix Released
Revision history for this message
Mathew Hodson (mhodson) wrote :

Fixed in neutron 17.0.0 and later.

Changed in neutron (Ubuntu):
status: New → Fix Released
Revision history for this message
Erlon R. Cruz (sombrafam) wrote :

Attached debdiff for Bionic Rocky

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to charm-neutron-openvswitch (master)
Revision history for this message
Erlon R. Cruz (sombrafam) wrote :
Revision history for this message
Corey Bryant (corey.bryant) wrote :

Thanks Erlon, I've uploaded to rocky-staging and the bionic unapproved queue. The SRU team will need to review the bionic version, so I have subscribed them.

Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello David, or anyone else affected,

Accepted neutron into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/neutron/2:12.1.1-0ubuntu8 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in neutron (Ubuntu Bionic):
status: New → Fix Committed
tags: added: verification-needed verification-needed-bionic
Revision history for this message
Corey Bryant (corey.bryant) wrote :

Hello David, or anyone else affected,

Accepted neutron into queens-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:queens-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-queens-needed to verification-queens-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-queens-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Revision history for this message
Corey Bryant (corey.bryant) wrote :

Hello David, or anyone else affected,

Accepted neutron into rocky-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:rocky-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-rocky-needed to verification-rocky-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-rocky-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-rocky-needed
Changed in cloud-archive:
status: New → Invalid
description: updated
tags: added: verification-bionic-done
removed: verification-needed-bionic
description: updated
Mathew Hodson (mhodson)
tags: added: verification-done-bionic
removed: verification-bionic-done verification-needed
description: updated
tags: added: verification-done-rocky
removed: verification-rocky-needed
Revision history for this message
Chris Halse Rogers (raof) wrote : Update Released

The verification of the Stable Release Update for neutron has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package neutron - 2:12.1.1-0ubuntu8

---------------
neutron (2:12.1.1-0ubuntu8) bionic; urgency=medium

  * Backport fix for TCP checksum issue (LP: #1832021)
    - d/p/0001-Workaround-for-TCP-checksum-issue-with-ovs-dpdk-and-.patch

 -- erlon <email address hidden> Mon, 26 Apr 2021 14:01:49 -0300

Changed in neutron (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Corey Bryant (corey.bryant) wrote :

The verification of the Stable Release Update for neutron has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Corey Bryant (corey.bryant) wrote :

This bug was fixed in the package neutron - 2:13.0.7-0ubuntu1~cloud6
---------------

 neutron (2:13.0.7-0ubuntu1~cloud6) bionic-rocky; urgency=medium
 .
   * Backport fix for TCP checksum issue (LP: #1832021)
     - d/p/0001-Workaround-for-TCP-checksum-issue-with-ovs-dpdk-and-.patch

tags: added: verification-needed-queens
Revision history for this message
Erlon R. Cruz (sombrafam) wrote :

I tried to test this on xenial-queens, but the charm does not work and seems to be broken. See that the queens backport fails only on xenial. On bionic-queens, it worked and passed the tests. So, we should not add this package to the xenial-queens cloud archive.

tags: added: verification-queens-failed
removed: verification-needed-queens
Revision history for this message
Corey Bryant (corey.bryant) wrote :

@erlon, I can revert this, however if the bug in the charm can be hit with or without this SRU code, I think we should open a bug against the charm. Would you be able to look into that?

Revision history for this message
Corey Bryant (corey.bryant) wrote :

Reverting for Queens cloud archive only.

Revision history for this message
Erlon R. Cruz (sombrafam) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-neutron-openvswitch (master)

Reviewed: https://review.opendev.org/c/openstack/charm-neutron-openvswitch/+/790354
Committed: https://opendev.org/openstack/charm-neutron-openvswitch/commit/1f6e29fbe2c6cabbdb082f5d8899520809f9b7ae
Submitter: "Zuul (22348)"
Branch: master

commit 1f6e29fbe2c6cabbdb082f5d8899520809f9b7ae
Author: Erlon R. Cruz <email address hidden>
Date: Sun May 9 10:38:02 2021 -0300

    Adds Neutron OVS agent config to DHCP daemon

    The neutron-dhcp-agent might rely on options defined in
    openvswitch_agent.ini. By default this config file is not passed to
    neutron-dhcp-agent daemon, and therefore those options are not
    loaded and fall back for the default values and by that inhibits
    the desired behavior.

    Charm helpers' PR: https://github.com/juju/charm-helpers/pull/615

    Depends-on: I39024855c3e42ee135b6ad5e7618a770219b6994

    Closes-bug: #1832021
    Change-Id: I134c8077ee52ccdb4e383109ecbea27ed1633fb8

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers