Comment 12 for bug 1731032

Revision history for this message
Thomas Herve (therve) wrote :

I'm starting to narrow it down to an area. We go up to NetworkDeployment. Once it's signaled, 2 deployments are triggered from the stack: ControllerUpgradeInitDeployment, and UpdateDeployment (as both depend on NetworkDeployment). Then those are supposed to update The swift temp_url with the deployment data. What seems to happen is that they step on each other, and the second one to run override the first. We can see it in the swift logs:

Nov 16 19:44:10 centos-7-citycloud-lon1-0000925781 proxy-server[7670]: 192.168.24.1 192.168.24.1 16/Nov/2017/19/44/10 PUT /v1/AUTH_0b280f98fead4055bce9a4b8860998b5/ov-ntroller-leqby5yewtov-deployed-server-e6phgktmis65/ff613249-3912-453b-beb0-8710c4af0b22%3Ftemp_url_expires%3D2147483586%26temp_url_sig%3Dbeeb444483649059ba3ee9364d672101a0a9ff50 HTTP/1.0 201 - python-requests/2.14.2 - 36588 - - txad48021f81c040aeabb2a-005a0dea8a - 0.1672 - - 1510861450.004256964 1510861450.171437025 0
Nov 16 19:44:10 centos-7-citycloud-lon1-0000925781 proxy-server[7671]: 192.168.24.1 192.168.24.1 16/Nov/2017/19/44/10 PUT /v1/AUTH_0b280f98fead4055bce9a4b8860998b5/ov-ntroller-leqby5yewtov-deployed-server-e6phgktmis65/ff613249-3912-453b-beb0-8710c4af0b22%3Ftemp_url_expires%3D2147483586%26temp_url_sig%3Dbeeb444483649059ba3ee9364d672101a0a9ff50 HTTP/1.0 201 - python-requests/2.14.2 - 34811 - - txf4d5aa9a0b384d0a87ef5-005a0dea89 - 0.5486 - - 1510861449.966480970 1510861450.515069008 0

Size goes from 36588 to 34811 (the size of the JSON blob delivered to the instance).

https://github.com/openstack/heat/blob/master/heat/engine/service_software_config.py#L108 is where we retrieve the deployments. We're doing a metadata update to synchronize, but something wrong is happening.