Task "Enable tuned profile" fail Cannot load profile(s) ''cpu-partitioning'': Assertion ''isolated_cores are set'' failed

Bug #1879458 reported by Noam Angel
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
High
Noam Angel
Ussuri
Fix Committed
Undecided
Noam Angel
Victoria
Fix Released
High
Noam Angel

Bug Description

Description
===========

Task "Enable tuned profile" fail Cannot load profile(s) ''cpu-partitioning'': Assertion ''isolated_cores are set'' failed

Steps to reproduce
==================
Deploy with cpu-partitioning and isolated_cpu

Expected result
===============
Pass deployment and cpu-partitioning configured

Actual result
=============
 cmd:
# tuned-adm profile cpu-partitioning
'Cannot load profile(s) ''cpu-partitioning'': Assertion ''isolated_cores are set'' failed.'

reason:
TASK [tuned : Configure isolated cores for profile cpu-partitioning] ***********
Monday 18 May 2020 22:36:19 +0300 (0:00:00.152) 0:00:57.833 ************
skipping: [overcloud-computeovsdpdk-0]
skipping: [overcloud-computeovsdpdk-1]

blame:
https://review.opendev.org/#/c/714610/

Logs & Configs
==============

TASK [tuned : Fail if tuned profile conf is absent but isolated cores is provided] ***
Monday 18 May 2020 22:36:19 +0300 (0:00:00.402) 0:00:57.680 ************
skipping: [overcloud-computeovsdpdk-0]
skipping: [overcloud-computeovsdpdk-1]

TASK [tuned : Configure isolated cores for profile cpu-partitioning] ***********
Monday 18 May 2020 22:36:19 +0300 (0:00:00.152) 0:00:57.833 ************
skipping: [overcloud-computeovsdpdk-0]
skipping: [overcloud-computeovsdpdk-1]

TASK [tuned : Enable tuned profile] ********************************************
Monday 18 May 2020 22:36:19 +0300 (0:00:00.149) 0:00:57.983 ************
fatal: [overcloud-computeovsdpdk-1]: FAILED! => changed=true
  cmd:
  - tuned-adm
  - profile
  - cpu-partitioning
  delta: '0:00:00.259931'
  end: '2020-05-18 19:36:20.507243'
  msg: non-zero return code
  rc: 1
  start: '2020-05-18 19:36:20.247312'
  stderr: 'Cannot load profile(s) ''cpu-partitioning'': Assertion ''isolated_cores are set'' failed.'
  stderr_lines: <omitted>
  stdout: ''
  stdout_lines: <omitted>
fatal: [overcloud-computeovsdpdk-0]: FAILED! => changed=true
  cmd:
  - tuned-adm
  - profile
  - cpu-partitioning
  delta: '0:00:00.329479'
  end: '2020-05-18 19:36:20.552545'
  msg: non-zero return code
  rc: 1
  start: '2020-05-18 19:36:20.223066'
  stderr: 'Cannot load profile(s) ''cpu-partitioning'': Assertion ''isolated_cores are set'' failed.'
  stderr_lines: <omitted>
  stdout: ''
  stdout_lines: <omitted>

NO MORE HOSTS LEFT *************************************************************

PLAY RECAP *********************************************************************
overcloud-computeovsdpdk-0 : ok=37 changed=13 unreachable=0 failed=1 skipped=8 rescued=0 ignored=0
overcloud-computeovsdpdk-1 : ok=34 changed=13 unreachable=0 failed=1 skipped=8 rescued=0 ignored=0
overcloud-controller-0 : ok=41 changed=19 unreachable=0 failed=0 skipped=20 rescued=0 ignored=0
undercloud : ok=6 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Monday 18 May 2020 22:36:20 +0300 (0:00:00.678) 0:00:58.661 ************

Changed in tripleo:
assignee: nobody → Noam Angel (noama)
status: New → In Progress
Revision history for this message
Noam Angel (noama) wrote :

parameter_defaults:

  ############
  # OVS-DPDK #
  ############
  NeutronOVSFirewallDriver: openvswitch
  OvsDpdkDriverType: "mlx5_core"
  ComputeOvsDpdkParameters:
    ExtraKernelModules:
      nvme: {}
      nvme-rdma: {}
    NeutronBridgeMappings: ['tenant:br-mlnx']
    TunedProfileName: "cpu-partitioning"
    KernelArgs: "default_hugepagesz=1G hugepagesz=1G hugepages=12 iommu=pt intel_iommu=on"
    OvsDpdkSocketMemory: "1024,1024"
    OvsDpdkMemoryChannels: "4"
    OvsDpdkCoreList: "0,12,1,13"
    OvsPmdCoreList: "2,14,3,15,4,16,5,17"
    NovaVcpuPinSet: "6,18,7,19,8,20,9,21,10,22,11,23"
    IsolCpusList: "2,14,3,15,4,16,5,17,6,18,7,19,8,20,9,21,10,22,11,23"

Saravanan KR (skramaja)
Changed in tripleo:
importance: Undecided → High
milestone: none → victoria-1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-ansible (master)

Reviewed: https://review.opendev.org/729181
Committed: https://git.openstack.org/cgit/openstack/tripleo-ansible/commit/?id=858f49573824eb5a34326b4c25d15dd4b9b27576
Submitter: Zuul
Branch: master

commit 858f49573824eb5a34326b4c25d15dd4b9b27576
Author: Noam Angel <email address hidden>
Date: Tue May 19 08:56:40 2020 +0000

    Fix conditions tuned profile

    tuned_isolated_cores is not bool type, check "is defined" and length >0
    instead.

    Change-Id: Iaeec9a8cc116023b34b357f68614c986992054a0
    Closes-Bug: #1879458

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-ansible (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/732503

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-ansible (stable/ussuri)

Reviewed: https://review.opendev.org/732503
Committed: https://git.openstack.org/cgit/openstack/tripleo-ansible/commit/?id=de1995a2849e2838d8d96777b27e0ac2abd59662
Submitter: Zuul
Branch: stable/ussuri

commit de1995a2849e2838d8d96777b27e0ac2abd59662
Author: Noam Angel <email address hidden>
Date: Tue May 19 08:56:40 2020 +0000

    Fix conditions tuned profile

    tuned_isolated_cores is not bool type, check "is defined" and length >0
    instead.

    Change-Id: Iaeec9a8cc116023b34b357f68614c986992054a0
    Closes-Bug: #1879458
    (cherry picked from commit 858f49573824eb5a34326b4c25d15dd4b9b27576)

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.