Backup & Restore: Unable to unlock after restore - puppet error

[Senthil Mukundakumar]

Brief Description
-----------------

Active controller failed to unlock after restore, puppet manifest error

[sysadmin@controller-0 log(keystone_admin)]$ system host-unlock controller-0
Expecting number of interface sriov_numvfs=16. Please wait a few minutes for inventory update and retry host-unlock.

2020-06-25T18:40:56.962 Error: 2020-06-25 18:40:55 +0000 Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Comparison of: Undef Value > Integer, is not possible. Caused by 'Only Strings, Numbers, Timespans, Timestamps, and Versions are comparable'. at /usr/share/puppet/modules/platform/manifests/network.pp:187:16 on node controller-0
2020-06-25T18:40:56.964 /usr/share/ruby/vendor_ruby/puppet/pops/evaluator/compare_operator.rb:111:in `cmp_Object'
2020-06-25T18:40:56.967 /usr/share/ruby/vendor_ruby/puppet/pops/visitor.rb:48:in `block in visit_this'
2020-06-25T18:40:56.970 /usr/share/ruby/vendor_ruby/puppet/pops/visitor.rb:42:in `each'
2020-06-25T18:40:56.974 /usr/share/ruby/vendor_ruby/puppet/pops/visitor.rb:42:in `visit_this'
2020-06-25T18:40:56.976 /usr/share/ruby/vendor_ruby/puppet/pops/visitor.rb:71:in `visit_this_1'
2020-06-25T18:40:56.979 /usr/share/ruby/vendor_ruby/puppet/pops/evaluator/compare_operator.rb:33:in `compare'
2020-06-25T18:40:56.982 /usr/share/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:527:in `eval_ComparisonExpression'
2020-06-25T18:40:56.985 /usr/share/ruby/vendor_ruby/puppet/pops/visitor.rb:69:in `visit_this_1'
2020-06-25T18:40:56.988 /usr/share/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:82:in `evaluate'
2020-06-25T18:40:56.992 /usr/share/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:695:in `eval_ParenthesizedExpression'
2020-06-25T18:40:56.995 /usr/share/ruby/vendor_ruby/puppet/pops/visitor.rb:69:in `visit_this_1'
2020-06-25T18:40:57.002 /usr/share/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:82:in `evaluate'
2020-06-25T18:40:57.006 /usr/share/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:1004:in `block in eval_IfExpression'
2020-06-25T18:40:57.009 /usr/share/ruby/vendor_ruby/puppet/parser/scope.rb:944:in `with_guarded_scope'
2020-06-25T18:40:57.012 /usr/share/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:1003:in `eval_IfExpression'
2020-06-25T18:40:57.014 /usr/share/ruby/vendor_ruby/puppet/pops/visitor.rb:69:in `visit_this_1'
2020-06-25T18:40:57.017 /usr/share/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:82:in `evaluate'
2020-06-25T18:40:57.019 /usr/share/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:638:in `block in eval_BlockExpression'
2020-06-25T18:40:57.022 /usr/share/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:638:in `each'
2020-06-25T18:40:57.024 /usr/share/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:638:in `reduce'
2020-06-25T18:40:57.027 /usr/share/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:638:in `eval_BlockExpression'
2020-06-25T18:40:57.029 /usr/share/ruby/vendor_ruby/puppet/pops/visitor.rb:69:in `visit_this_1'
2020-06-25T18:40:57.033 /usr/share/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:82:in `evaluate'
2020-06-25T18:40:57.036 /usr/share/ruby/vendor_ruby/puppet/pops/parser/evaluating_parser.rb:63:in `evaluate'
2020-06-25T18:40:57.038 /usr/share/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:32:in `evaluate'
2020-06-25T18:40:57.042 /usr/share/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:72:in `block (2 levels) in evaluate'
2020-06-25T18:40:57.044 /usr/share/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:71:in `catch'
2020-06-25T18:40:57.047 /usr/share/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:71:in `block in evaluate'
2020-06-25T18:40:57.049 /usr/share/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:70:in `catch'
2020-06-25T18:40:57.052 /usr/share/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:70:in `evaluate'
2020-06-25T18:40:57.055 /usr/share/ruby/vendor_ruby/puppet/parser/ast.rb:31:in `safeevaluate'
2020-06-25T18:40:57.057 /usr/share/ruby/vendor_ruby/puppet/resource/type.rb:184:in `evaluate_code'
2020-06-25T18:40:57.060 /usr/share/ruby/vendor_ruby/puppet/parser/resource.rb:81:in `block in evaluate'
2020-06-25T18:40:57.062
2020-06-25T18:40:57.105 /usr/share/ruby/vendor_ruby/puppet/parser/compiler.rb:610:in `loop'
2020-06-25T18:40:57.107 /usr/share/ruby/vendor_ruby/puppet/parser/compiler.rb:610:in `evaluate_generators'
2020-06-25T18:40:57.110 /usr/share/ruby/vendor_ruby/puppet/parser/compiler.rb:187:in `block (2 levels) in compile'

Severity
--------
Critical - Restore operation fails

Steps to Reproduce
------------------
1. Backup the file using ansible backup cli.
ansible-playbook /usr/share/ansible/stx-ansible/playbooks/backup.yml -e "ansible_become_pass=Li69nux* admin_password=Li69nux*"
2. FTP backup file
3. Reinstall the AIO-SX
4. Restore the backup file using below cli.
ansible-playbook /usr/share/ansible/stx-ansible/playbooks/restore_platform.yml -e "backup_filename=localhost_platform_backup_2020_01_21_18_30_21.tgz admin_passwori69nux* ansible_become_pass=Li69nux* initial_backup_dir=/home/sysadmin"
5. system host-unlock controller-0

Expected Behavior
------------------
Controller should be unlocked and become available

Actual Behavior
----------------
Controller failed to unlock

Reproducibility
---------------
(1/1)

System Configuration
--------------------
AIO-SX SM3

Branch/Pull Time/Commit
-----------------------
BUILD_ID="2020-06-24_22-16-59"

Timestamp/Logs
--------------
2020-06-25T18:40:56.962
https://files.starlingx.kube.cengn.ca/launchpad/1885173

LAST PASS
---------
2020-05-20_21-00-00

Test Activity
-------------
Regression

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/738108

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/738108
Committed: https://git.openstack.org/cgit/starlingx/stx-puppet/commit/?id=c30274e7adb8b80eceef7a3e96ffa378b5fca607
Submitter: Zuul
Branch: master

commit c30274e7adb8b80eceef7a3e96ffa378b5fca607
Author: Steven Webster <email address hidden>
Date: Thu Jun 25 15:19:02 2020 -0400

    Fix SR-IOV enable for VF type interfaces

    The puppet config for type VF interfaces deliberately has the
    number of virtual functions set to None, as they borrow from
    their parent SR-IOV interface. That is, it's only necessary to
    set the number of VFs on the parent interface.

    A previous commit changed the check on the num_vfs parameter to
    ensure it is > 1. We need to also check that this parameter is
    defined.

    Closes-Bug: #1885173
    Change-Id: I671af0bc16aed3a38d2ebe84b0f6b7195180e97f
    Signed-off-by: Steven Webster <email address hidden>

[Senthil Mukundakumar (smukunda)]

Verified in SM3 using 2020-06-26_01-17-51

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: f/centos8
Review: https://review.opendev.org/762919