etcd stuck on restarting
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
kolla-ansible |
New
|
Undecided
|
Unassigned |
Bug Description
train, centos/source
kolla-ansible 9.3.2
ansible 2.8.5
etcd container stuck on restarting. There are no logs in kolla_logs. The following is from the container logs:
+ sudo -E kolla_set_configs
INFO:__
INFO:__
INFO:__main__:Kolla config strategy set to: COPY_ALWAYS
INFO:__
++ cat /run_command
+ CMD=etcd
+ ARGS=
+ sudo kolla_copy_cacerts
+ [[ ! -n '' ]]
+ . kolla_extend_start
++ [[ ! -d /var/log/kolla/etcd ]]
+++ stat -c %a /var/log/kolla/etcd
++ [[ 2755 != \7\5\5 ]]
++ chmod 755 /var/log/kolla/etcd
+++ stat -c %U /var/lib/etcd/
++ [[ etcd != \e\t\c\d ]]
+ echo 'Running command: '\''etcd'\'''
+ exec etcd
Running command: 'etcd'
2022-01-17 13:24:32.567889 I | pkg/flags: recognized and used environment variable ETCD_ADVERTISE_
2022-01-17 13:24:32.568105 I | pkg/flags: recognized and used environment variable ETCD_DATA_
2022-01-17 13:24:32.568148 I | pkg/flags: recognized and used environment variable ETCD_INITIAL_
2022-01-17 13:24:32.568162 I | pkg/flags: recognized and used environment variable ETCD_INITIAL_
2022-01-17 13:24:32.568208 I | pkg/flags: recognized and used environment variable ETCD_INITIAL_
2022-01-17 13:24:32.568222 I | pkg/flags: recognized and used environment variable ETCD_INITIAL_
2022-01-17 13:24:32.568236 I | pkg/flags: recognized and used environment variable ETCD_LISTEN_
2022-01-17 13:24:32.568249 I | pkg/flags: recognized and used environment variable ETCD_LISTEN_
2022-01-17 13:24:32.568277 I | pkg/flags: recognized and used environment variable ETCD_NAME=st1
2022-01-17 13:24:32.568315 W | pkg/flags: unrecognized environment variable ETCD_OUT_
2022-01-17 13:24:32.568381 I | etcdmain: etcd Version: 3.2.21
2022-01-17 13:24:32.568393 I | etcdmain: Git SHA: 3ac81f3
2022-01-17 13:24:32.568399 I | etcdmain: Go Version: go1.12.8
2022-01-17 13:24:32.568405 I | etcdmain: Go OS/Arch: linux/amd64
2022-01-17 13:24:32.568412 I | etcdmain: setting maximum number of CPUs to 24, total number of available CPUs is 24
2022-01-17 13:24:32.568588 N | etcdmain: the server is already initialized as member before, starting as etcd member...
2022-01-17 13:24:32.568937 I | embed: listening for peers on http://<CONTROLLER_
2022-01-17 13:24:32.569035 I | embed: listening for client requests on <CONTROLLER_
2022-01-17 13:24:32.595661 I | etcdserver: recovered store from snapshot at index 266003146
2022-01-17 13:24:32.614880 I | etcdserver: name = st1
2022-01-17 13:24:32.615011 I | etcdserver: data dir = /var/lib/etcd
2022-01-17 13:24:32.615064 I | etcdserver: member dir = /var/lib/
2022-01-17 13:24:32.615107 I | etcdserver: heartbeat = 100ms
2022-01-17 13:24:32.615147 I | etcdserver: election = 1000ms
2022-01-17 13:24:32.615187 I | etcdserver: snapshot count = 100000
2022-01-17 13:24:32.615240 I | etcdserver: advertise client URLs = http://<CONTROLLER_
2022-01-17 13:24:33.926719 I | etcdserver: restarting member 992b265e4c5c0056 in cluster 27dcc4e71e160b at commit index 266095209
2022-01-17 13:24:33.939861 I | raft: 992b265e4c5c0056 became follower at term 215
2022-01-17 13:24:33.939943 I | raft: newRaft 992b265e4c5c0056 [peers: [992b265e4c5c0056], term: 215, commit: 266095209, applied: 266003146, lastindex: 266095209, lastterm: 215]
2022-01-17 13:24:33.940498 C | etcdserver/