Brief Description
-----------------
When doing an optimized restore after an optimized upgrade LDAP errors are encountered during puppet AIO manifest.
Possible cause: During an upgrade the original CentOS ldap config files are left on system in /etc/openldap. This confuses the restore playbook causing the wrong files to be restored for LDAP.
If /etc/openldap is determined to be the issue, this is also present during legacy restore. Making this an uncaught previous issue.
Severity
-----------------
Critical
Steps to Reproduce
-----------------
Perform optimized upgrade from 21.12 to 22.12.
Perform optimized backup and restore
Expected Behavior
-----------------
Restore is successful.
Actual Behavior
-----------------
Restore is unsuccessful.
Reproducibility
-----------------
100%
System Configuration
-----------------
AIO-SX
Load info (eg: 2022-03-10_20-00-07)
-----------------
master
Last Pass
-----------------
Never
Timestamp/Logs
-----------------
2023-08-10T17:34:58.455 Info: 2023-08-10 17:34:58 +0000 /Stage[main]/Platform::Ldap::Server::Local/Service[openldap]: Unscheduling refresh on Service[openldap] [2566/788030]
2023-08-10T17:34:58.460 Debug: 2023-08-10 17:34:58 +0000 Exec[slapd-convert-config](provider=posix): Executing check '/usr/bin/test -e /etc/ldap/slapd.conf'
2023-08-10T17:34:58.464 Debug: 2023-08-10 17:34:58 +0000 Executing: '/usr/bin/test -e /etc/ldap/slapd.conf'
2023-08-10T17:34:58.469 Debug: 2023-08-10 17:34:58 +0000 /Stage[main]/Platform::Ldap::Server::Local/Exec[slapd-convert-config]: '/usr/sbin/slaptest -f /etc/ldap/slapd.conf -F /etc/ldap/schema/' won't be executed because of failed check 'onlyif'
2023-08-10T17:34:58.474 Debug: 2023-08-10 17:34:58 +0000 Exec[restart-openldap](provider=posix): Executing check '/usr/bin/test -e /etc/ldap/slapd.conf'
2023-08-10T17:34:58.478 Debug: 2023-08-10 17:34:58 +0000 Executing: '/usr/bin/test -e /etc/ldap/slapd.conf'
2023-08-10T17:34:58.483 Debug: 2023-08-10 17:34:58 +0000 /Stage[main]/Platform::Ldap::Server::Local/Exec[restart-openldap]: '/usr/bin/systemctl restart slapd.service' won't be executed because of failed check 'onlyif'
2023-08-10T17:34:58.488 Debug: 2023-08-10 17:34:58 +0000 Exec[configure-ldaps](provider=posix): Executing check 'test -e /etc/ldap/certs/openldap-cert.crt'
2023-08-10T17:34:58.493 Debug: 2023-08-10 17:34:58 +0000 Executing: 'test -e /etc/ldap/certs/openldap-cert.crt'
2023-08-10T17:34:58.497 Debug: 2023-08-10 17:34:58 +0000 Exec[configure-ldaps](provider=posix): Executing check 'test -e /etc/ldap/certs/openldap-cert.key'
2023-08-10T17:34:58.502 Debug: 2023-08-10 17:34:58 +0000 Executing: 'test -e /etc/ldap/certs/openldap-cert.key'
2023-08-10T17:34:58.507 Debug: 2023-08-10 17:34:58 +0000 Exec[configure-ldaps](provider=posix): Executing 'ldapmodify -D cn=config -w "t3UCJ+8EANw7_bu_" -xH ldap:/// -f /etc/ldap/certs.ldif'
2023-08-10T17:34:58.511 Debug: 2023-08-10 17:34:58 +0000 Executing: 'ldapmodify -D cn=config -w "t3UCJ+8EANw7_bu_" -xH ldap:/// -f /etc/ldap/certs.ldif'
2023-08-10T17:34:58.516 Notice: 2023-08-10 17:34:58 +0000 /Stage[main]/Platform::Ldap::Server::Local/Exec[configure-ldaps]/returns: ldap_bind: Invalid credentials (49)
2023-08-10T17:34:58.520 Error: 2023-08-10 17:34:58 +0000 'ldapmodify -D cn=config -w "t3UCJ+8EANw7_bu_" -xH ldap:/// -f /etc/ldap/certs.ldif' returned 49 instead of one of [0]
2023-08-10T17:34:58.524 /usr/lib/ruby/vendor_ruby/puppet/util/errors.rb:157:in `fail'
2023-08-10T17:34:58.529 /usr/lib/ruby/vendor_ruby/puppet/type/exec.rb:168:in `sync'
2023-08-10T17:34:58.534 /usr/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:239:in `sync'
2023-08-10T17:34:58.539 /usr/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:134:in `sync_if_needed'
2023-08-10T17:34:58.544 /usr/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:88:in `block in perform_changes'
2023-08-10T17:34:58.548 /usr/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:87:in `each'
2023-08-10T17:34:58.553 /usr/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:87:in `perform_changes'
2023-08-10T17:34:58.558 /usr/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:21:in `evaluate'
2023-08-10T17:34:58.562 /usr/lib/ruby/vendor_ruby/puppet/transaction.rb:259:in `apply'
2023-08-10T17:34:58.567 /usr/lib/ruby/vendor_ruby/puppet/transaction.rb:279:in `eval_resource'
2023-08-10T17:34:58.572 /usr/lib/ruby/vendor_ruby/puppet/transaction.rb:183:in `call'
2023-08-10T17:34:58.576 /usr/lib/ruby/vendor_ruby/puppet/transaction.rb:183:in `block (2 levels) in evaluate'
2023-08-10T17:34:58.581 /usr/lib/ruby/vendor_ruby/puppet/util.rb:539:in `block in thinmark'
2023-08-10T17:34:58.586 /usr/lib/ruby/2.7.0/benchmark.rb:308:in `realtime'
2023-08-10T17:34:58.591 /usr/lib/ruby/vendor_ruby/puppet/util.rb:538:in `thinmark'
2023-08-10T17:34:58.595 /usr/lib/ruby/vendor_ruby/puppet/transaction.rb:183:in `block in evaluate'
2023-08-10T17:34:58.600 /usr/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:121:in `traverse'
2023-08-10T17:34:58.605 /usr/lib/ruby/vendor_ruby/puppet/transaction.rb:173:in `evaluate'
2023-08-10T17:34:58.609 /usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:239:in `block (2 levels) in apply'
2023-08-10T17:34:58.615 /usr/lib/ruby/vendor_ruby/puppet/util.rb:539:in `block in thinmark'
2023-08-10T17:34:58.620 /usr/lib/ruby/2.7.0/benchmark.rb:308:in `realtime'
2023-08-10T17:34:58.624 /usr/lib/ruby/vendor_ruby/puppet/util.rb:538:in `thinmark'
2023-08-10T17:34:58.630 /usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:238:in `block in apply'
2023-08-10T17:34:58.635 /usr/lib/ruby/vendor_ruby/puppet/util/log.rb:161:in `with_destination'
2023-08-10T17:34:58.640 /usr/lib/ruby/vendor_ruby/puppet/transaction/report.rb:146:in `as_logging_destination'
2023-08-10T17:34:58.644 /usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:237:in `apply'
2023-08-10T17:34:58.648 /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:190:in `block (2 levels) in apply_catalog'
2023-08-10T17:34:58.652 /usr/lib/ruby/vendor_ruby/puppet/util.rb:539:in `block in thinmark'
2023-08-10T17:34:58.655 /usr/lib/ruby/2.7.0/benchmark.rb:308:in `realtime'
2023-08-10T17:34:58.659 /usr/lib/ruby/vendor_ruby/puppet/util.rb:538:in `thinmark'
2023-08-10T17:34:58.662 /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:189:in `block in apply_catalog'
2023-08-10T17:34:58.666 /usr/lib/ruby/vendor_ruby/puppet/util.rb:232:in `block in benchmark'
2023-08-10T17:34:58.669 /usr/lib/ruby/2.7.0/benchmark.rb:308:in `realtime'
2023-08-10T17:34:58.673 /usr/lib/ruby/vendor_ruby/puppet/util.rb:231:in `benchmark'
2023-08-10T17:34:58.676 /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:188:in `apply_catalog'
2023-08-10T17:34:58.680 /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:390:in `run_internal'
2023-08-10T17:34:58.685 /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:238:in `block in run'
2023-08-10T17:34:58.688 /usr/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override'
2023-08-10T17:34:58.692 /usr/lib/ruby/vendor_ruby/puppet.rb:263:in `override'
2023-08-10T17:34:58.697 /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:215:in `run'
2023-08-10T17:34:58.702 /usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:355:in `apply_catalog'
2023-08-10T17:34:58.707 /usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:280:in `block (2 levels) in main'
2023-08-10T17:34:58.719 /usr/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override'
2023-08-10T17:34:58.725 /usr/lib/ruby/vendor_ruby/puppet.rb:263:in `override'
2023-08-10T17:34:58.727 /usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:280:in `block in main'
2023-08-10T17:34:58.730 /usr/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override'
2023-08-10T17:34:58.733 /usr/lib/ruby/vendor_ruby/puppet.rb:263:in `override'
2023-08-10T17:34:58.736 /usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:233:in `main'
2023-08-10T17:34:58.739 /usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:174:in `run_command'
2023-08-10T17:34:58.742 /usr/lib/ruby/vendor_ruby/puppet/application.rb:375:in `block in run'
2023-08-10T17:34:58.746 /usr/lib/ruby/vendor_ruby/puppet/util.rb:710:in `exit_on_fail'
2023-08-10T17:34:58.750 /usr/lib/ruby/vendor_ruby/puppet/application.rb:375:in `run'
2023-08-10T17:34:58.753 /usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:139:in `run'
2023-08-10T17:34:58.756 /usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:77:in `execute'
2023-08-10T17:34:58.759 /usr/bin/puppet:11:in `<main>'
2023-08-10T17:34:58.762 Error: 2023-08-10 17:34:58 +0000 /Stage[main]/Platform::Ldap::Server::Local/Exec[configure-ldaps]/returns: change from 'notrun' to ['0'] failed: 'ldapmodify -D cn=config -w "t3UCJ+8EANw7_bu_" -xH ldap:/// -f /etc/ldap/certs.ldif' returned 49 instead of one of [0]
Alarms
-----------------
N/A
Test Activity
-----------------
Developer Testing
Workaround
-----------------
Exclude /etc/openldap during backup
Fix proposed to branch: master /review. opendev. org/c/starlingx /ansible- playbooks/ +/891616
Review: https:/