After upgrading controller-1 host fails to swact to controller-1 and then active controller goes back to controller-0

[Thiago Antonio Miranda]

Brief Description
-----------------
After controller-1 is upgraded controller-0 is unable to swact to controller-1 as active.

Severity
--------
Critical unable to continue upgrade

Steps to Reproduce
------------------
Install StarlingX DX
Reinstall controller-0
Follow the upgrade procedure

Expected Behavior
-----------------
1. Able to swact from controller-0 to controller-1 after controller-1 upgrade.

Actual Behavior
---------------
State what is the actual behavior

Reproducibility
---------------

System Configuration
--------------------
DC

Last Pass
---------

Timestamp/Logs
--------------
2023-10-25T19:31:17.343 controller-1 postgres[136113]: err [1-1] db=,user= FATAL: configuration file "/etc/postgresql/postgresql.conf" contains errors
2023-10-25T19:33:17.074 controller-1 postgres[155043]: err [1-1] db=,user= FATAL: configuration file "/etc/postgresql/postgresql.conf" contains errors
2023-10-25T19:35:17.650 controller-1 postgres[170357]: err [1-1] db=,user= FATAL: configuration file "/etc/postgresql/postgresql.conf" contains errors
2023-10-25T19:37:17.719 controller-1 postgres[184849]: err [1-1] db=,user= FATAL: configuration file "/etc/postgresql/postgresql.conf" contains errors
2023-10-25T19:54:22.629 controller-1 postgres[384716]: err [1-1] db=,user= FATAL: configuration file "/etc/postgresql/postgresql.conf" contains errors
2023-10-25T19:56:22.648 controller-1 postgres[403939]: err [1-1] db=,user= FATAL: configuration file "/etc/postgresql/postgresql.conf" contains errors
2023-10-25T19:58:23.145 controller-1 postgres[418688]: err [1-1] db=,user= FATAL: configuration file "/etc/postgresql/postgresql.conf" contains errors
2023-10-25T20:00:23.171 controller-1 postgres[433741]: err [1-1] db=,user= FATAL: configuration file "/etc/postgresql/postgresql.conf" contains errors
2023-10-25T20:27:01.502 controller-1 postgres[744467]: err [1-1] db=,user= FATAL: configuration file "/etc/postgresql/postgresql.conf" contains errors

sudo -u postgres /usr/lib/postgresql/13/bin/postgres -D /var/lib/postgresql/22.12 -c config_file=/etc/postgresql/postgresql.conf
2023-10-25 21:29:34.582 GMT [1468453] LOG: invalid value for parameter "log_timezone": "UCT"
2023-10-25 21:29:34.582 GMT [1468453] LOG: invalid value for parameter "TimeZone": "UCT"
db=,user= FATAL: configuration file "/etc/postgresql/postgresql.conf" contains errors

Test Activity
-------------
Regression

Workaround
----------

sudo vim /etc/postgresql/postgresql.conf

You system probably has these two parameters set as this:

log_timezone = 'UCT'

timezone = 'UCT'

You have to change to:

log_timezone = 'UTC'

timezone = 'UTC'

Then run:

sudo sm-restart service postgres

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/stx-puppet/+/899478
Committed: https://opendev.org/starlingx/stx-puppet/commit/5dd9aed8edf0b6c64ceb8f17168ddf1d05d3f64f
Submitter: "Zuul (22348)"
Branch: master

commit 5dd9aed8edf0b6c64ceb8f17168ddf1d05d3f64f
Author: Thiago Miranda <email address hidden>
Date: Fri Oct 27 10:39:17 2023 -0300

    Add the configuration of postgresql.conf parameters timezone and log_timezone through puppet

    This update adds the timezone and log_timezone to postgresql.conf trough
    puppet.

    Test Plan:

    PASS: Verify after a lock/unlock the postgresql.conf file should have params
    timezone and log_timezone equal to platform config timezone param (UTC)
    PASS: Verify after a bootstrap the postgresql.conf file should have params
    timezone and log_timezone equial to platform config timezone param (UTC)
    PASS: Verify after controller-1 is upgraded controller-0 is able to swact
    to controller-1 as active

    Closes-Bug: 2041696
    Change-Id: Id328420a33c6aa755f671998d75c72f3f8621603
    Signed-off-by: Thiago Miranda <email address hidden>