nova_wait_for_compute_service.py Failing When Supplying Multiple Options In NovaPCIPassthrough

[Vadim Khitrin]

During deployment of Compute Nodes, when using 'NovaPCIPassthrough' with multiple options, for example:
  NovaPCIPassthrough:
    - devname: "eth10"
      trusted: "true"
      physical_network: "sriov-1"
    - devname: "eth11"
      trusted: "true"
      physical_network: "sriov-2"

TripleO will generate in nova.conf the following entries:
passthrough_whitelist={"devname":"eth10","physical_network":"sriov-1","trusted":"true"}
passthrough_whitelist={"devname":"eth11","physical_network":"sriov-2","trusted":"true"}

Which will cause nova_wait_for_compute_service.py to fail with the following exception:
configparser.DuplicateOptionError: While reading from '/etc/nova/nova.conf' [line 8669]: option 'passthrough_whitelist' in section 'pci' already exists
Log of script with error: http://paste.openstack.org/show/750597/

When omitting one of the entries or supplying both entries in one single list the script will pass, example:
passthrough_whitelist=[{"devname":"eth10","physical_network":"sriov-1","trusted":"true"},
                       {"devname":"eth12","physical_network":"sriov-2","trusted":"true"}]

Log of script passing with the change above: http://paste.openstack.org/show/750596/

Update:

Doesn't seem like an issue in TripleO, output of heat resource generated:
u'nova::compute::pci::passthrough': u'[{"devname": "eth10", "physical_network": "sriov-1", "trusted": "true"}, {"devname": "eth11", "physical_network": "sriov-2", "trusted": "true"}]'

A list containing the parameters is passed to puppet.

Will take a look in puppet-nova.

Perhaps meanwhile, it is worth to provide some kind of a disclaimer warning users that they might encounter the same scenario when deploying OpenStack Stein release.

[Vadim Khitrin (vkhitrin)]

Found a possible solution in puppet-nova,

Will attempt to deploy with multiple NovaPCIPassthrough options with puppet-nova fix and report there as well.

[Vadim Khitrin (vkhitrin)]

Opened a launchpad to puppet-nova: https://bugs.launchpad.net/puppet-nova/+bug/1828105

Proposed a patch that will make this bug obsolete:
https://review.opendev.org/#/c/657673/

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/657824

[Martin Schuppert (mschuppert)]

In python3 SafeConfigParser was renamed to ConfigParser and the default
for duplicate options default to true [1]. In case of nova it is valid to
have duplicate option lines, e.g. passthrough_whitelist which leads to
issues reading the nova.conf

The proposed patch from #3 will allow multiple same options in nova.conf for
python3.

[1] https://docs.python.org/3/library/configparser.html#configparser.ConfigParser

[Martin Schuppert (mschuppert)]

I meant like e.g. pci_alias can be specified multiple times in nova.conf

[Vadim Khitrin (vkhitrin)]

Martin I wasn't sure if I should tackle the issue from TripleO side or puupet-nova. Personally I thought that having multiple entries generated when there is no need is something that should be looked into, but you're right, there are some additional options that will be generated multiple times (like pci_alias) that would've required a change as well.

Since you're altering the scripts in TripleO I'll abandon the puppet-nova bug and patch.

Thanks for letting into this.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/657824
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=bbd2d944836d07aba396998f6590c87a978e59f0
Submitter: Zuul
Branch: master

commit bbd2d944836d07aba396998f6590c87a978e59f0
Author: Martin Schuppert <email address hidden>
Date: Wed May 8 15:40:55 2019 +0200

    Allow multiple same options in nova.conf

    In python3 SafeConfigParser was renamed to ConfigParser and the default
    for duplicate options default to true. In case of nova it is valid to
    have duplicate option lines, e.g. pci_alias can be specified more then
    once in nova.conf and results in an error like seen in
    https://bugs.launchpad.net/tripleo/+bug/1827775

    https://docs.python.org/3/library/configparser.html#configparser.ConfigParser

    Closes-Bug: #1827775

    Change-Id: I410af66d8dceb6dde84828c9bd1969aa623bf34c

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/658260

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/658260
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=93155b544b5b2738273e013cc86e812e38f0c163
Submitter: Zuul
Branch: stable/stein

commit 93155b544b5b2738273e013cc86e812e38f0c163
Author: Martin Schuppert <email address hidden>
Date: Wed May 8 15:40:55 2019 +0200

    Allow multiple same options in nova.conf

    In python3 SafeConfigParser was renamed to ConfigParser and the default
    for duplicate options default to true. In case of nova it is valid to
    have duplicate option lines, e.g. pci_alias can be specified more then
    once in nova.conf and results in an error like seen in
    https://bugs.launchpad.net/tripleo/+bug/1827775

    https://docs.python.org/3/library/configparser.html#configparser.ConfigParser

    Closes-Bug: #1827775

    Change-Id: I410af66d8dceb6dde84828c9bd1969aa623bf34c
    (cherry picked from commit bbd2d944836d07aba396998f6590c87a978e59f0)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 11.0.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 10.6.0 release.