etcd stuck on restarting

Bug #1958176 reported by alpha23
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kolla-ansible
New
Undecided
Unassigned

Bug Description

train, centos/source
kolla-ansible 9.3.2
ansible 2.8.5

etcd container stuck on restarting. There are no logs in kolla_logs. The following is from the container logs:

+ sudo -E kolla_set_configs
INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json
INFO:__main__:Validating config file
INFO:__main__:Kolla config strategy set to: COPY_ALWAYS
INFO:__main__:Writing out command to execute
++ cat /run_command
+ CMD=etcd
+ ARGS=
+ sudo kolla_copy_cacerts
+ [[ ! -n '' ]]
+ . kolla_extend_start
++ [[ ! -d /var/log/kolla/etcd ]]
+++ stat -c %a /var/log/kolla/etcd
++ [[ 2755 != \7\5\5 ]]
++ chmod 755 /var/log/kolla/etcd
+++ stat -c %U /var/lib/etcd/
++ [[ etcd != \e\t\c\d ]]
+ echo 'Running command: '\''etcd'\'''
+ exec etcd
Running command: 'etcd'
2022-01-17 13:24:32.567889 I | pkg/flags: recognized and used environment variable ETCD_ADVERTISE_CLIENT_URLS=http://<CONTROLLER_IP>:2379
2022-01-17 13:24:32.568105 I | pkg/flags: recognized and used environment variable ETCD_DATA_DIR=/var/lib/etcd
2022-01-17 13:24:32.568148 I | pkg/flags: recognized and used environment variable ETCD_INITIAL_ADVERTISE_PEER_URLS=http://<CONTROLLER_IP>:2380
2022-01-17 13:24:32.568162 I | pkg/flags: recognized and used environment variable ETCD_INITIAL_CLUSTER=st1=http://<CONTROLLER_IP>:2380
2022-01-17 13:24:32.568208 I | pkg/flags: recognized and used environment variable ETCD_INITIAL_CLUSTER_STATE=new
2022-01-17 13:24:32.568222 I | pkg/flags: recognized and used environment variable ETCD_INITIAL_CLUSTER_TOKEN=h5PyOEojNQfYBAAklH6Wb1SG6w3AwPHVzrb446JG
2022-01-17 13:24:32.568236 I | pkg/flags: recognized and used environment variable ETCD_LISTEN_CLIENT_URLS=http://<CONTROLLER_IP>:2379
2022-01-17 13:24:32.568249 I | pkg/flags: recognized and used environment variable ETCD_LISTEN_PEER_URLS=http://<CONTROLLER_IP>:2380
2022-01-17 13:24:32.568277 I | pkg/flags: recognized and used environment variable ETCD_NAME=st1
2022-01-17 13:24:32.568315 W | pkg/flags: unrecognized environment variable ETCD_OUT_FILE=/var/log/kolla/etcd/etcd.log
2022-01-17 13:24:32.568381 I | etcdmain: etcd Version: 3.2.21
2022-01-17 13:24:32.568393 I | etcdmain: Git SHA: 3ac81f3
2022-01-17 13:24:32.568399 I | etcdmain: Go Version: go1.12.8
2022-01-17 13:24:32.568405 I | etcdmain: Go OS/Arch: linux/amd64
2022-01-17 13:24:32.568412 I | etcdmain: setting maximum number of CPUs to 24, total number of available CPUs is 24
2022-01-17 13:24:32.568588 N | etcdmain: the server is already initialized as member before, starting as etcd member...
2022-01-17 13:24:32.568937 I | embed: listening for peers on http://<CONTROLLER_IP>:2380
2022-01-17 13:24:32.569035 I | embed: listening for client requests on <CONTROLLER_IP>:2379
2022-01-17 13:24:32.595661 I | etcdserver: recovered store from snapshot at index 266003146
2022-01-17 13:24:32.614880 I | etcdserver: name = st1
2022-01-17 13:24:32.615011 I | etcdserver: data dir = /var/lib/etcd
2022-01-17 13:24:32.615064 I | etcdserver: member dir = /var/lib/etcd/member
2022-01-17 13:24:32.615107 I | etcdserver: heartbeat = 100ms
2022-01-17 13:24:32.615147 I | etcdserver: election = 1000ms
2022-01-17 13:24:32.615187 I | etcdserver: snapshot count = 100000
2022-01-17 13:24:32.615240 I | etcdserver: advertise client URLs = http://<CONTROLLER_IP>:2379
2022-01-17 13:24:33.926719 I | etcdserver: restarting member 992b265e4c5c0056 in cluster 27dcc4e71e160b at commit index 266095209
2022-01-17 13:24:33.939861 I | raft: 992b265e4c5c0056 became follower at term 215
2022-01-17 13:24:33.939943 I | raft: newRaft 992b265e4c5c0056 [peers: [992b265e4c5c0056], term: 215, commit: 266095209, applied: 266003146, lastindex: 266095209, lastterm: 215]
2022-01-17 13:24:33.940498 C | etcdserver/membership: cluster cannot be downgraded (current version: 3.2.21 is lower than determined cluster version: 3.3).

Tags: train
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.