Adding a new compute starts new service containers on controller
Bug #1805410 reported by
Cédric Jeanneret
This bug affects 1 person
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
tripleo |
Fix Released
|
High
|
Bogdan Dobrelya |
Bug Description
Hello,
Infra:
- 1 undercloud
- 1 controller
- 1 compute , then 2
- tripleo master
- podman as container engine
The initial deploy works as expected, the overcloud is deployed on the two selected machines.
But when I add the second compute, I have the following issue:
If I do no set the --skip-
Apparently, something doesn't properly detect the controller is already deployed, or something like that, and we end with new containers while the original ones are still here, running and happy to live.
Changed in tripleo: | |
status: | Triaged → Incomplete |
summary: |
- Adding a new compute starts a new service containers on controller + Adding a new compute starts new service containers on controller |
Changed in tripleo: | |
importance: | Medium → High |
Changed in tripleo: | |
milestone: | stein-3 → stein-2 |
Changed in tripleo: | |
assignee: | nobody → Bogdan Dobrelya (bogdando) |
To post a comment you must log in.
Some logs and info (took time to re-run the whole thing)
In the deploy logs: bootstrap" , _name=keystone --filter label=config_ id=tripleo_ step3 --format {{.Names}}",
"keystone- vmn4c9po" , bootstrap_ host_exec keystone keystone-manage bootstrap --bootstrap- password vFViRs1DFjwJrmD z4I1ltcpgZ" , vmn4c9po' , '/usr/bin/ bootstrap_ host_exec' , 'keystone', 'keystone-manage', 'bootstrap', '--bootstrap- password' , 'vFViRs1DFjwJrm Dz4I1ltcpgZ' ]. [125]",
"Running container: keystone_
"$ podman ps -a --filter label=container
"keystone",
"$ podman exec --user=root keystone-vmn4c9po /usr/bin/
"cannot exec into container that is not running",
"Error running ['podman', 'exec', '--user=root', u'keystone-
"stderr: cannot exec into container that is not running",
On controller-0, we can see those NEW containers: io/tripleomaste r/centos- binary- keystone: current- tripleo /bin/bash -c /usr... 5 minutes ago Up 5 minutes ago keystone_ cron-8p9o9k0h io/tripleomaste r/centos- binary- mariadb: current- tripleo kolla_start 8 minutes ago Up About a minute ago mysql-kw4r3w5x io/tripleomaste r/centos- binary- haproxy: current- tripleo kolla_start 10 minutes ago Up 10 minutes ago haproxy-nse1lud1 io/tripleomaste r/centos- binary- keepalived: current- tripleo /usr/local/ bin/ko. .. 10 minutes ago Up 10 minutes ago keepalived-qapg6ssa
d64303343f36 docker.
b674af359a06 docker.
069b6ca20a83 docker.
c2f11cba5a32 docker.
While the older ones are still running, and apparently in good shape: io/tripleomaste r/centos- binary- keystone: current- tripleo /bin/bash -c /usr... About an hour ago Up About an hour ago keystone_cron io/tripleomaste r/centos- binary- mariadb: current- tripleo kolla_start About an hour ago Up About an hour ago mysql io/tripleomaste r/centos- binary- haproxy: current- tripleo kolla_start About an hour ago Up About an hour ago haproxy io/tripleomaste r/centos- binary- keepalived: current- tripleo /usr/local/ bin/ko. .. About an hour ago Up About an hour ago keepalived
e59cdeacdce9 docker.
25d9bb139ed4 docker.
96919bdb2ac2 docker.
8cca1f0d47cc docker.
I'm wondering if I4386b155a4bdba 430dc350914db7a 6b6fdf92ac0[ 1] could do that kind of thing?
Having multiple mysqld processes hitting the very same DB creates this issue in service log:
2018-11-27 15:41:37 140499793242304 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
2018-11-27 15:41:37 140499793242304 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
Also, regarding the keystone container, if we `podman logs keystone-vmn4c9po` we can see this: conf.d/ autoindex. conf at line 21 will probably never match because it overlaps an earlier Alias.
[Tue Nov 27 15:32:23.781274 2018] [alias:warn] [pid 9] AH00671: The Alias directive in /etc/httpd/
(98)Address already in use: AH00072: make_sock: could not bind to address 192.168.24.8:35357
no listening...