[systest] glance-manage db_sync returned 2 instead of one of [0]

[Egor Kotko]

VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "5.1"
  api: "1.0"
  build_number: "305"
  build_id: "2014-07-09_02-14-20"
  astute_sha: "c0ffd4fa1b1ea16931f174a7f4efeac701ec23e6"
  fuellib_sha: "fd5fd0d3f74c5a084adfc951a4ee24e3dc27e09c"
  ostf_sha: "f6f7cee46a85ca3e758f629c2df8b370e9de494a"
  nailgun_sha: "2001a30884f2c24d18a62fc9f9c76c6ed66691e3"
  fuelmain_sha: "9e441d9035fa852bdb00be1031355f0f89823231"

http://jenkins-product.srt.mirantis.net:8080/view/0_master_swarm/job/master_fuelmain.system_test.ubuntu.thread_4/107/testReport/%28root%29/deploy_ha_flat/deploy_ha_flat/

Steps to reproduce:
1. Create env: Ubuntu, 3 controllers 2 compute, flat dhcp

Expected result:
Deployed cluster

Actual Result:
Deployment failed with errors in puppet log

Wed Jul 09 04:15:19 +0000 2014 /Stage[main]/Glance::Registry/Exec[glance-manage db_sync]/returns (notice): glance-manage: error: argument command: invalid choice: 'version_control' (choose from 'db', 'db_version', 'db_upgrade', 'db_downgrade', 'db_version_control', 'db_sync')
Wed Jul 09 04:15:19 +0000 2014 /Stage[main]/Glance::Registry/Exec[glance-manage db_sync] (err): Failed to call refresh: glance-manage db_sync || (glance-manage version_control 0 && glance-manage db_sync) returned 2 instead of one of [0]
Wed Jul 09 04:15:19 +0000 2014 /Stage[main]/Glance::Registry/Exec[glance-manage db_sync] (err): glance-manage db_sync || (glance-manage version_control 0 && glance-manage db_sync) returned 2 instead of one of [0]

[Aleksandr Didenko (adidenko)]

The problems started with the following error on primary controller (node-1):

Wed Jul 09 04:03:22 +0000 2014 /Stage[main]/Cluster::Haproxy_ocf/Service[p_haproxy] (info): Starting to evaluate the resource
Wed Jul 09 04:03:22 +0000 2014 Service[p_haproxy](provider=pacemaker) (debug): getting last operations
Wed Jul 09 04:03:28 +0000 2014 Service[p_haproxy](provider=pacemaker) (debug): getting last ops on node-1 for p_haproxy
Wed Jul 09 04:03:28 +0000 2014 Service[p_haproxy](provider=pacemaker) (debug): ALL OPERATIONS:

 []
Wed Jul 09 04:03:28 +0000 2014 Service[p_haproxy](provider=pacemaker) (debug): COMPLETED OPERATIONS:

 []
Wed Jul 09 04:03:28 +0000 2014 Service[p_haproxy](provider=pacemaker) (debug): START/STOP OPERATIONS:

 []
Wed Jul 09 04:03:28 +0000 2014 Service[p_haproxy](provider=pacemaker) (debug): GOOD OPERATIONS :

 []
Wed Jul 09 04:03:28 +0000 2014 Service[p_haproxy](provider=pacemaker) (debug): LAST GOOD OPERATION :

 'nil' 'true' ''
Wed Jul 09 04:03:30 +0000 2014 /Stage[main]/Cluster::Haproxy_ocf/Service[p_haproxy] (err): Could not evaluate: resource p_haproxy not found

Which has broken resource dependencies and all the following errors were caused by it. I have not found any clue in pacemaker/corosync logs and was not able to reproduce the bug on the later ISOs (#317, #320).

If you reproduce this bug again, please provide access to the running env for further investigation.

[Andrew Woodward (xarses)]

Incomplete for more than one week. If this is still an issue please re-open the bug