Some logs and info (took time to re-run the whole thing)
In the deploy logs:
"Running container: keystone_bootstrap",
"$ podman ps -a --filter label=container_name=keystone --filter label=config_id=tripleo_step3 --format {{.Names}}", "keystone-vmn4c9po",
"keystone",
"$ podman exec --user=root keystone-vmn4c9po /usr/bin/bootstrap_host_exec keystone keystone-manage bootstrap --bootstrap-password vFViRs1DFjwJrmDz4I1ltcpgZ",
"cannot exec into container that is not running",
"Error running ['podman', 'exec', '--user=root', u'keystone-vmn4c9po', '/usr/bin/bootstrap_host_exec', 'keystone', 'keystone-manage', 'bootstrap', '--bootstrap-password', 'vFViRs1DFjwJrmDz4I1ltcpgZ']. [125]",
"stderr: cannot exec into container that is not running",
On controller-0, we can see those NEW containers:
d64303343f36 docker.io/tripleomaster/centos-binary-keystone:current-tripleo /bin/bash -c /usr... 5 minutes ago Up 5 minutes ago keystone_cron-8p9o9k0h
b674af359a06 docker.io/tripleomaster/centos-binary-mariadb:current-tripleo kolla_start 8 minutes ago Up About a minute ago mysql-kw4r3w5x
069b6ca20a83 docker.io/tripleomaster/centos-binary-haproxy:current-tripleo kolla_start 10 minutes ago Up 10 minutes ago haproxy-nse1lud1
c2f11cba5a32 docker.io/tripleomaster/centos-binary-keepalived:current-tripleo /usr/local/bin/ko... 10 minutes ago Up 10 minutes ago keepalived-qapg6ssa
While the older ones are still running, and apparently in good shape:
e59cdeacdce9 docker.io/tripleomaster/centos-binary-keystone:current-tripleo /bin/bash -c /usr... About an hour ago Up About an hour ago keystone_cron
25d9bb139ed4 docker.io/tripleomaster/centos-binary-mariadb:current-tripleo kolla_start About an hour ago Up About an hour ago mysql
96919bdb2ac2 docker.io/tripleomaster/centos-binary-haproxy:current-tripleo kolla_start About an hour ago Up About an hour ago haproxy
8cca1f0d47cc docker.io/tripleomaster/centos-binary-keepalived:current-tripleo /usr/local/bin/ko... About an hour ago Up About an hour ago keepalived
I'm wondering if I4386b155a4bdba430dc350914db7a6b6fdf92ac0[1] could do that kind of thing?
Having multiple mysqld processes hitting the very same DB creates this issue in service log:
2018-11-27 15:41:37 140499793242304 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
2018-11-27 15:41:37 140499793242304 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
Also, regarding the keystone container, if we `podman logs keystone-vmn4c9po` we can see this:
[Tue Nov 27 15:32:23.781274 2018] [alias:warn] [pid 9] AH00671: The Alias directive in /etc/httpd/conf.d/autoindex.conf at line 21 will probably never match because it overlaps an earlier Alias.
(98)Address already in use: AH00072: make_sock: could not bind to address 192.168.24.8:35357
no listening sockets available, shutting down
AH00015: Unable to open logs
Meaning the container is stopped, and we of course can't `exec` in a stopped container:
e371cac37e51 docker.io/tripleomaster/centos-binary-keystone:current-tripleo kolla_start 11 minutes ago Exited (1) 11 minutes ago keystone-vmn4c9po
Some logs and info (took time to re-run the whole thing)
In the deploy logs: bootstrap" , _name=keystone --filter label=config_ id=tripleo_ step3 --format {{.Names}}",
"keystone- vmn4c9po" , bootstrap_ host_exec keystone keystone-manage bootstrap --bootstrap- password vFViRs1DFjwJrmD z4I1ltcpgZ" , vmn4c9po' , '/usr/bin/ bootstrap_ host_exec' , 'keystone', 'keystone-manage', 'bootstrap', '--bootstrap- password' , 'vFViRs1DFjwJrm Dz4I1ltcpgZ' ]. [125]",
"Running container: keystone_
"$ podman ps -a --filter label=container
"keystone",
"$ podman exec --user=root keystone-vmn4c9po /usr/bin/
"cannot exec into container that is not running",
"Error running ['podman', 'exec', '--user=root', u'keystone-
"stderr: cannot exec into container that is not running",
On controller-0, we can see those NEW containers: io/tripleomaste r/centos- binary- keystone: current- tripleo /bin/bash -c /usr... 5 minutes ago Up 5 minutes ago keystone_ cron-8p9o9k0h io/tripleomaste r/centos- binary- mariadb: current- tripleo kolla_start 8 minutes ago Up About a minute ago mysql-kw4r3w5x io/tripleomaste r/centos- binary- haproxy: current- tripleo kolla_start 10 minutes ago Up 10 minutes ago haproxy-nse1lud1 io/tripleomaste r/centos- binary- keepalived: current- tripleo /usr/local/ bin/ko. .. 10 minutes ago Up 10 minutes ago keepalived-qapg6ssa
d64303343f36 docker.
b674af359a06 docker.
069b6ca20a83 docker.
c2f11cba5a32 docker.
While the older ones are still running, and apparently in good shape: io/tripleomaste r/centos- binary- keystone: current- tripleo /bin/bash -c /usr... About an hour ago Up About an hour ago keystone_cron io/tripleomaste r/centos- binary- mariadb: current- tripleo kolla_start About an hour ago Up About an hour ago mysql io/tripleomaste r/centos- binary- haproxy: current- tripleo kolla_start About an hour ago Up About an hour ago haproxy io/tripleomaste r/centos- binary- keepalived: current- tripleo /usr/local/ bin/ko. .. About an hour ago Up About an hour ago keepalived
e59cdeacdce9 docker.
25d9bb139ed4 docker.
96919bdb2ac2 docker.
8cca1f0d47cc docker.
I'm wondering if I4386b155a4bdba 430dc350914db7a 6b6fdf92ac0[ 1] could do that kind of thing?
Having multiple mysqld processes hitting the very same DB creates this issue in service log:
2018-11-27 15:41:37 140499793242304 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
2018-11-27 15:41:37 140499793242304 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
Also, regarding the keystone container, if we `podman logs keystone-vmn4c9po` we can see this: conf.d/ autoindex. conf at line 21 will probably never match because it overlaps an earlier Alias.
[Tue Nov 27 15:32:23.781274 2018] [alias:warn] [pid 9] AH00671: The Alias directive in /etc/httpd/
(98)Address already in use: AH00072: make_sock: could not bind to address 192.168.24.8:35357
no listening sockets available, shutting down
AH00015: Unable to open logs
Meaning the container is stopped, and we of course can't `exec` in a stopped container: io/tripleomaste r/centos- binary- keystone: current- tripleo kolla_start 11 minutes ago Exited (1) 11 minutes ago keystone-vmn4c9po
e371cac37e51 docker.
[1] https:/ /review. openstack. org/#/c/ 613295/