OpenStack deployment failed due to glance db connectivity

[Sergey Murashov]

iso: {"build_id": "2014-08-25_00-01-17", "ostf_sha": "907f25f8fad39b177bf6a66fba9785afa7dd8008", "build_number": "476", "auth_required": true, "api": "1.0", "nailgun_sha": "44876ddae29823449e0cbc59428aafa466cbbbc2", "production": "docker", "fuelmain_sha": "5d44e4173f3b15b5991737269fbfae71275a63da", "astute_sha": "0c1b16721f7810b1c400f51937ecc6f058be8cff", "feature_groups": ["mirantis"], "release": "5.1", "fuellib_sha": "31debf95128afa44a9991491eaff169625cb49b1"}

Steps to reproduce:
1. Deploy OS(HA, Ubuntu, 3controllers, 1compute+ceph-osd, neutron gre, murano)

Actual result:
Deployment fails. in log of glance-registry for node-3:

OperationalError: (OperationalError) (2013, "Lost connection to MySQL server at 'reading authorization packet', system error: 110") None None

[Mike Scherbakov (mihgen)]

Did you try to run Network Verification to ensure you have L2 connectivity for all networks?

[Sergey Murashov (smurashov)]

yes, I did it, network verification passed

[Sergii Golovatiuk (sgolovatiuk)]

This is usually caused by an aborted connect. You can verify this by checking the status:
mysql> SHOW GLOBAL STATUS LIKE 'Aborted_connects';
If this counter keeps increasing as you get the lost connections, that's a sign you're having a problem during connect. We can increase connection_timeout to mitigate the issues with high load during deployment.

[Sergii Golovatiuk (sgolovatiuk)]

Make sure you have 2.5 GB for controllers as it's known issue when controllers have high swap in/out due to RAM issues. This causes sporadic issues with services.

[Vladimir Kuklin (vkuklin)]

possibly, the same connection error as in here: https://bugs.launchpad.net/fuel/+bug/1362163

[Vladimir Kuklin (vkuklin)]

we never faced this issue before. let's mark it as incomplete until we get a reproducer. looks like really a connectivity issue

[Jay Pipes (jaypipes)]

What is the bind_address value used in the my.cnf?

[Stanislaw Bogatkin (sbogatkin)]

Can reproduce. Diagnostic snapshot attached.

[Vladimir Kuklin (vkuklin)]

Stanislaw, snapshot that you have attached does not contain any errors in deployment.

[Vladimir Kuklin (vkuklin)]

I am marking this bug as incomplete as Stanislaw's snapshot does not contain errors and there are no other reproducers.

[Stanislaw Bogatkin (sbogatkin)]

Ok, I can't reproduce that anymore. Was tried another 5 times.