First seen via https://bugzilla.redhat.com/show_bug.cgi?id=1842919
1) Undercloud is deployed successfully and has the following in /etc/hosts:
[stack@undercloud-0 ~]$ grep undercloud-0 /etc/hosts
192.168.24.1 undercloud-0.ctlplane.redhat.local undercloud-0.ctlplane
127.0.0.1 undercloud-0.redhat.local undercloud-0
192.168.24.1 undercloud-0.ctlplane.redhat.local undercloud-0.ctlplane
Let's leave out for a second that there are two identical lines.
2) User calls overcloud deploy
3) The parameter 'UndercloudHostsEntries' gets populated via https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_deploy.py#L100
4) UndercloudHostsEntries gets populated with the following pseudo-code:
getent hosts "$(hostname -s).ctlplane"
Now the output of that command is actually the following:
192.168.24.1 undercloud-0.ctlplane.redhat.local undercloud-0.ctlplane undercloud-0.ctlplane
192.168.24.1 undercloud-0.ctlplane.redhat.local undercloud-0.ctlplane undercloud-0.ctlplane
Notice how getent hosts adds another undercloud-0.ctlplane towards the ends.
5) The overcloud deploy uses UndercloudHostsEntries -> undercloud_hosts_entries -> ends up being used to populate /etc/hosts and we keep growing our lines.
As an additional test I manually added another undercloud-0.ctplane entry on all relevant lines in /etc/hosts like the following (so we went from one to two in /etc/hosts):
[stack@undercloud-0 ~]$ grep undercloud-0.ctlplane /etc/hosts
192.168.24.1 undercloud-0.ctlplane.redhat.local undercloud-0.ctlplane undercloud-0.ctlplane
192.168.24.1 undercloud-0.ctlplane.redhat.local undercloud-0.ctlplane undercloud-0.ctlplane
Now look at getent hosts:
[stack@undercloud-0 ~]$ getent hosts "$(hostname -s).ctlplane"
192.168.24.1 undercloud-0.ctlplane.redhat.local undercloud-0.ctlplane undercloud-0.ctlplane undercloud-0.ctlplane undercloud-0.ctlplane
192.168.24.1 undercloud-0.ctlplane.redhat.local undercloud-0.ctlplane undercloud-0.ctlplane undercloud-0.ctlplane undercloud-0.ctlplane
So two undercloud-0.ctplane entries in /etc/hosts end up being 4 in the output of getent hosts, which is making this thing grow quadratically and explains why we quickly end up hitting the json limit in heat.
Now the reason for getent hosts to keep adding host entries is due to the duplicate line in /etc/hosts.
Here is the proof:
[root@undercloud ~]# echo '1.2.3.4 pippo.localdomain pippo.ctlplane' >> /etc/hosts
[root@undercloud ~]# getent hosts pippo.ctlplane
# Normal
1.2.3.4 pippo.localdomain pippo.ctlplane
[root@undercloud ~]# echo '1.2.3.4 pippo.localdomain pippo.ctlplane' >> /etc/hosts
# Two lines automatically add the hosts
[root@undercloud ~]# getent hosts pippo.ctlplane
1.2.3.4 pippo.localdomain pippo.ctlplane pippo.ctlplane
1.2.3.4 pippo.localdomain pippo.ctlplane pippo.ctlplane
I think a few things need to happen to fix this properly. Namely we need to:
A) in 3) we should prune identical multiple lines
B) in 3) we should also prune identical entries
C) We should ideally also figure out who/what on earth adds 192.168.24.1 undercloud-0.ctlplane.redhat.local undercloud-0.ctlplane twice to /etc/hosts
Now even though C) is not ideal, we should still be more robust in the face of an /etc/hosts file that has duplicate entries.
Fix proposed to branch: master
Review: https:/