When enabling DPDK, the charm fails to install showing the error: AttributeError: 'OVSDPDKDeviceContext' object has no attribute 'pmd_cpu_mask'

Bug #2007996 reported by Marcelo Subtil Marcal
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
charm-ovn-chassis
Fix Released
Undecided
Unassigned

Bug Description

When deploying DPDK enabled ovn-chassis, the charm stays in install error state.

The unit log shows:

```
2023-02-21 16:23:59 INFO unit.ovn-chassis/5.juju-log server.go:316 Making dir /var/lib/charm/ovn-chassis root:root 555
2023-02-21 16:23:59 INFO unit.ovn-chassis/5.juju-log server.go:316 Making dir /etc/policy-rc.d root:root 555
2023-02-21 16:23:59 INFO unit.ovn-chassis/5.juju-log server.go:316 Invoking reactive handler: reactive/layer_openstack.py:15:default_install
2023-02-21 16:24:00 ERROR unit.ovn-chassis/5.juju-log server.go:316 Hook error:
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-ovn-chassis-5/.venv/lib/python3.10/site-packages/charms/reactive/__init__.py", line 74, in main
    bus.dispatch(restricted=restricted_mode)
  File "/var/lib/juju/agents/unit-ovn-chassis-5/.venv/lib/python3.10/site-packages/charms/reactive/bus.py", line 390, in dispatch
    _invoke(other_handlers)
  File "/var/lib/juju/agents/unit-ovn-chassis-5/.venv/lib/python3.10/site-packages/charms/reactive/bus.py", line 359, in _invoke
    handler.invoke()
  File "/var/lib/juju/agents/unit-ovn-chassis-5/.venv/lib/python3.10/site-packages/charms/reactive/bus.py", line 181, in invoke
    self._action(*args)
  File "/var/lib/juju/agents/unit-ovn-chassis-5/charm/reactive/layer_openstack.py", line 28, in default_install
    instance.install()
  File "/var/lib/juju/agents/unit-ovn-chassis-5/charm/lib/charms/ovn_charm.py", line 198, in install
    super().install()
  File "/var/lib/juju/agents/unit-ovn-chassis-5/charm/lib/charms/ovn_charm.py", line 675, in install
    self.configure_sources()
  File "/var/lib/juju/agents/unit-ovn-chassis-5/charm/lib/charms/ovn_charm.py", line 669, in configure_sources
    self.configure_ovn_source()
  File "/var/lib/juju/agents/unit-ovn-chassis-5/charm/lib/charms/ovn_charm.py", line 633, in configure_ovn_source
    if self.options.ovn_source:
  File "/var/lib/juju/agents/unit-ovn-chassis-5/.venv/lib/python3.10/site-packages/charms_openstack/charm/core.py", line 492, in options
    self.__options = os_adapters.make_default_options(
  File "/var/lib/juju/agents/unit-ovn-chassis-5/.venv/lib/python3.10/site-packages/charms_openstack/adapters.py", line 633, in make_default_options
    return make_default_configuration_adapter_class(
  File "/var/lib/juju/agents/unit-ovn-chassis-5/charm/lib/charms/ovn_charm.py", line 228, in __init__
    self._ovs_dpdk_cpu_overlap_check()
  File "/var/lib/juju/agents/unit-ovn-chassis-5/charm/lib/charms/ovn_charm.py", line 371, in _ovs_dpdk_cpu_overlap_check
    if not (int(dpdk_context.pmd_cpu_mask(), 16) &
AttributeError: 'OVSDPDKDeviceContext' object has no attribute 'pmd_cpu_mask'
```

== Environment
Version: yoga-jammy
Hardware: Huawei 2288H V5 servers

$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.15.0-60-generic root=UUID=1a1b273b-a56b-456a-909a-cc786e523f80 ro console=tty0 console=ttyS0,115200n8 transparent_hugepage=never hugepagesz=1G hugepages=350 default_hugepagesz=1G iommu=pt intel_iommu=on

$ grep HugePages_ /proc/meminfo
HugePages_Total: 350
HugePages_Free: 350
HugePages_Rsvd: 0
HugePages_Surp: 0

$ lspci | grep Ethernet
1a:00.0 Ethernet controller: Intel Corporation Ethernet Connection X722 for 10GBASE-T (rev 09)
1a:00.1 Ethernet controller: Intel Corporation Ethernet Connection X722 for 10GBASE-T (rev 09)
1a:00.2 Ethernet controller: Intel Corporation Ethernet Connection X722 for 1GbE (rev 09)
1a:00.3 Ethernet controller: Intel Corporation Ethernet Connection X722 for 1GbE (rev 09)
3b:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02)
3b:00.1 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02)
3d:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02)
3d:00.1 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02)
86:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02)
86:00.1 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02)
88:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02)
88:00.1 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02)
af:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
af:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
af:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
af:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
b1:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
b1:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
b1:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
b1:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)

$ sudo dmesg | grep -e IOMMU
[ 1.372317] DMAR: IOMMU enabled
[ 7.243845] DMAR-IR: IOAPIC id 12 under DRHD base 0xc5ffc000 IOMMU 6
[ 7.250274] DMAR-IR: IOAPIC id 11 under DRHD base 0xb87fc000 IOMMU 5
[ 7.256703] DMAR-IR: IOAPIC id 10 under DRHD base 0xaaffc000 IOMMU 4
[ 7.263132] DMAR-IR: IOAPIC id 18 under DRHD base 0xfbffc000 IOMMU 3
[ 7.269560] DMAR-IR: IOAPIC id 17 under DRHD base 0xee7fc000 IOMMU 2
[ 7.275991] DMAR-IR: IOAPIC id 16 under DRHD base 0xe0ffc000 IOMMU 1
[ 7.282419] DMAR-IR: IOAPIC id 15 under DRHD base 0xd37fc000 IOMMU 0
[ 7.288847] DMAR-IR: IOAPIC id 8 under DRHD base 0x9d7fc000 IOMMU 7
[ 7.295189] DMAR-IR: IOAPIC id 9 under DRHD base 0x9d7fc000 IOMMU 7

Revision history for this message
Frode Nordahl (fnordahl) wrote :

And which track/channel/revision of the charm is this?

Changed in charm-ovn-chassis:
status: New → Incomplete
Revision history for this message
Marcelo Subtil Marcal (msmarcal) wrote :

Hi Frode,

Channel: 22.03/stable
Rev: 118

Revision history for this message
Frode Nordahl (fnordahl) wrote :
Changed in charm-ovn-chassis:
status: Incomplete → Confirmed
Revision history for this message
Bayani Carbone (bcarbone) wrote (last edit ):

Subscribed field-high
Using charm version 22.09 as a workaround.

Frode Nordahl (fnordahl)
Changed in charm-ovn-chassis:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-ovn-chassis (stable/22.03)

Reviewed: https://review.opendev.org/c/x/charm-ovn-chassis/+/888999
Committed: https://opendev.org/x/charm-ovn-chassis/commit/0aa27a4c43da7e0cb20d5e47b2ba291b794ccd41
Submitter: "Zuul (22348)"
Branch: stable/22.03

commit 0aa27a4c43da7e0cb20d5e47b2ba291b794ccd41
Author: Frode Nordahl <email address hidden>
Date: Fri Apr 22 17:02:19 2022 +0200

    Add DPDK low level gate tests

    Func-Test-Pr: https://github.com/openstack-charmers/zaza-openstack-tests/pull/1083
    Closes-Bug: #2007996
    (cherry picked from commit 0d42eb2b0c4a1030c18165b559bb4e422a0a9a28)
    (cherry picked from commit 121d2571dd8b91355bd2ae19a178e0ea46179657)
    Change-Id: I10685e32f431fc9b51444608135e8810df9cf7dd

Frode Nordahl (fnordahl)
Changed in charm-ovn-chassis:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.