Comment 4 for bug 1918366

Revision history for this message
Michele Baldessari (michele) wrote :

So I have also been observing keystone timeouts on RDO on some of my patches. The timeouts are because some of the memcached patches that have been merged lately are broken in regards to opening up the firewall rules.

Let's take the first failure mentioned at the beginning:
https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-master/6b98df3/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz

We see that nova is configured to use memcached:
https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-master/6b98df3/logs/overcloud-controller-0/var/lib/config-data/puppet-generated/nova/etc/nova/nova.conf.txt.gz

memcache_servers=172.17.0.139:11211,172.17.0.15:11211,172.17.0.220:11211

The problem is there are no iptables rules for memcached:
https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-master/6b98df3/logs/overcloud-controller-0/var/log/extra/network.txt.gz

In there you will see no rules opening up 11211. I think the logic I7a38a01f498d350d065a7c312a6654832fe24e6a is broken in regards to firewall rules, hence the timeouts.

Note: I don't think my analysis applies in case of standalone as there I believe iptables rules do not really play a role.