Load balancer sits in PENDING_CREATE state much longer than necessary

[Stephen Balukoff]

This might be related to the REST client unreasonable timeout bug ( #1496634 ) and is almost certainly related to the session_persistence tempest test intermittent failure bug ( #1541675 ), but I wanted to log this information because it is not the first time I've seen it (and before you ask this was before shared pools or L7):

When creating a load balancer, especially if it is not the first load balancer being deployed, the amphora will boot and be accessible for about 5 minutes before the controller worker bothers to complete the deployment of the load balancer on the amphora.

Specifically, here are two consecutive log lines, the first of which appeared shortly after the load balancer create command was issued:

2016-02-21 23:31:12.854 5084 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [-] Could not connect to instance. Retrying.

And the second of which was when the load balancer create was completed and the load balancer finally went from the PENDING_CREATE to the ACTIVE state:

2016-02-21 23:36:19.842 5084 INFO octavia.controller.worker.tasks.database_tasks [-] Mark ACTIVE in DB for load balancer id: c429bde5-6fb8-4557-a522-5f9184c65fb2

Between the above log lines, I was able to log into the amphora using SSH, was able to ping out, and was observing the amphora_health database entry for the amphora getting updated (indicating that the daemons on the amphora are clearly working just fine).

We should be noticing that the amphora is up much sooner than we actually are. It's unacceptable to wait 5 minutes when the amphora is actually up and ready within about 30 seconds.

[Michael Johnson (johnsom)]

Possible duplicate to: https://bugs.launchpad.net/octavia/+bug/1496634

[Michael Johnson (johnsom)]

Fixed as part of https://review.openstack.org/#/c/283255/