[astute] There are two images "TestVM" in Horizon

[Anastasia Palkina]

"build_id": "2014-07-14_00-31-14",
"mirantis": "yes",
"build_number": "122",
"ostf_sha": "09b6bccf7d476771ac859bb3c76c9ebec9da9e1f",
"nailgun_sha": "4637ae61920b92f56154a03f7e717e3bd597525c",
"production": "docker",
"api": "1.0",
"fuelmain_sha": "8a8faa68f7939d095dde60665f16b86d068f696d",
"astute_sha": "5df009e8eab611750309a4c5b5c9b0f7b9d85806",
"release": "5.0.1",
"fuellib_sha": "2d1e1369c13bc9771e9473086cb064d257a21fc2"

1. Create new environment (Ubuntu, HA mode)
2. Choose GRE segmentation
3. Choose Ceph for images
4. Add 3 controllers, compute, cinder, 2 ceph
5. Start deployment. It was successful
6. Start OSTF tests. Tests with creating instance has faile
7. Login to Horizon
8. Try to create instance. There are two images "TestVM" (see screen)

[Anastasia Palkina (apalkina)]

Logs are here https://drive.google.com/a/mirantis.com/file/d/0B6SjzarTGFxaTG93WnE0MThqZjA/edit?usp=sharing

[Anastasia Palkina (apalkina)]

root@node-17:~# glance image-list
+--------------------------------------+--------+-------------+------------------+----------+--------+
| ID | Name | Disk Format | Container Format | Size | Status |
+--------------------------------------+--------+-------------+------------------+----------+--------+
| 533a04a3-f302-4064-92ab-571966ddaa3a | TestVM | qcow2 | bare | 14811136 | active |
| 504bba7e-7974-4da0-91ee-b68de626d2e7 | TestVM | qcow2 | bare | 14811136 | active |
+--------------------------------------+--------+-------------+------------------+----------+--------+
root@node-17:~# glance image-show 533a04a3-f302-4064-92ab-571966ddaa3a
+------------------------------+-------------------------------------------------+
| Property | Value |
+------------------------------+-------------------------------------------------+
| Property 'murano_image_info' | {"title": "Murano Demo", "type": "cirros.demo"} |
| checksum | 9d526403713579f446434dd6190f9e9e |
| container_format | bare |
| created_at | 2014-07-14T15:33:14 |
| deleted | False |
| disk_format | qcow2 |
| id | 533a04a3-f302-4064-92ab-571966ddaa3a |
| is_public | True |
| min_disk | 0 |
| min_ram | 64 |
| name | TestVM |
| owner | f68e2f9c45284c62864ae22c88b53771 |
| protected | False |
| size | 14811136 |
| status | active |
| updated_at | 2014-07-14T15:33:44 |
+------------------------------+-------------------------------------------------+
root@node-17:~# glance image-show 504bba7e-7974-4da0-91ee-b68de626d2e7
+------------------------------+-------------------------------------------------+
| Property | Value |
+------------------------------+-------------------------------------------------+
| Property 'murano_image_info' | {"title": "Murano Demo", "type": "cirros.demo"} |
| checksum | 9d526403713579f446434dd6190f9e9e |
| container_format | bare |
| created_at | 2014-07-14T15:34:13 |
| deleted | False |
| disk_format | qcow2 |...

[Timur Nurlygayanov (tnurlygayanov)]

https://github.com/stackforge/fuel-library/blob/783951e836fe4b867f1174fc339090d202d5bc53/deployment/puppet/murano/manifests/cirros.pp

[Serg Melikyan (smelikyan)]

Looks like image was re-imported twice due too issue with Astute:

https://github.com/stackforge/fuel-astute/blob/stable/5.0/spec/unit/upload_cirros_image_hook_spec.rb

из лога ./localhost/var/log/astute/astute.log:
2014-07-14T11:31:29 info: [398] b27d2582-35f6-411d-89d9-808a5ad445cc: Upload cirros image "TestVM" is done
2014-07-14T11:47:42 info: [422] 21d61dae-9db4-4a1d-b164-e32d66c7fedf: Upload cirros image "TestVM" is done
2014-07-14T13:56:46 info: [410] f6ac4b97-c0ea-4eab-aba3-db2759705764: Upload cirros image "TestVM" is done
2014-07-14T15:34:26 info: [414] a1623f65-a1a2-4331-8bde-968cb0e52d55: Upload cirros image "TestVM" is done
2014-07-14T15:36:22 info: [398] 0b222c6b-b73d-4c3f-bd76-c5e8013a1dbc: Upload cirros image "TestVM" is done

[Vladimir Sharshov (vsharshov)]

Mcollective do not give answer when Astute try to add image and repeat request. As result we got 2 images.

Keywords: 2014-07-14T15:33:38 debug: [414] Retry #1 to run mcollective agent on nodes: '17'.

Possible reason: master node was overloaded (2 deploy in parallel in one physical node).

Workaround for 5.0: just remove one image from Glance
Solution for 5.1: join 2 command into one (check existing image and add image by one command)

Full problem logs:
2014-07-14T15:32:14 info: [414] a1623f65-a1a2-4331-8bde-968cb0e52d55: Finished deployment of nodes => roles: [{"20"=>"compute"}, {"21"=>"cinder"}, {"22"=>"ceph-osd"}, {"23"=>"ceph-osd"}]
2014-07-14T15:32:38 debug: [414] a1623f65-a1a2-4331-8bde-968cb0e52d55: MC agent 'execute_shell_command', method 'execute', results: {:sender=>"17", :statuscode=>0, :statusmsg=>"OK", :data=>{:stdout=>"ID Name Disk Format Container Format Size \n------------------------------------ ------------------------------ -------------------- -------------------- --------------\n", :exit_code=>1, :stderr=>""}}
2014-07-14T15:32:38 debug: [414] a1623f65-a1a2-4331-8bde-968cb0e52d55: cmd: /usr/bin/glance -N http://192.168.0.7:5000/v2.0/ -T admin -I admin -K admin index && (/usr/bin/glance -N http://192.168.0.7:5000/v2.0/ -T admin -I admin -K admin index | grep TestVM)
2014-07-14T15:33:38 debug: [414] Retry #1 to run mcollective agent on nodes: '17'
2014-07-14T15:34:26 debug: [414] a1623f65-a1a2-4331-8bde-968cb0e52d55: MC agent 'execute_shell_command', method 'execute', results: {:sender=>"17", :statuscode=>0, :statusmsg=>"OK", :data=>{:stdout=>"+------------------------------+-------------------------------------------------+\n| Property | Value |\n+------------------------------+-------------------------------------------------+\n| Property 'murano_image_info' | {\"title\": \"Murano Demo\", \"type\": \"cirros.demo\"} |\n| checksum | 9d526403713579f446434dd6190f9e9e |\n| container_format | bare |\n| created_at | 2014-07-14T15:34:13 |\n| deleted | False |\n| deleted_at | None |\n| disk_format | qcow2 |\n| id | 504bba7e-7974-4da0-91ee-b68de626d2e7 |\n| is_public | True |\n| min_disk | 0 |\n| min_ram | 64 |\n| name | TestVM |\n| owner | f68e2f9c45284c62864ae22c88b53771 |\n| protect...

[Mike Scherbakov (mihgen)]

Downgrading to Medium as it doesn't really break deployment and can be easily workarounded.

[Tatyanka (tatyana-leontovich)]

so for now, the same tests failed according we have 2 image http://paste.openstack.org/show/97318/
http://jenkins-product.srt.mirantis.net:8080/view/0_master_swarm/job/master_fuelmain.system_test.centos.thread_2/143/consoleFull

[Vladimir Sharshov (vsharshov)]

Bug was not reproduced for a long time, but root of problem still present in code. It will be fixed soon.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/134893

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/134893
Committed: https://git.openstack.org/cgit/stackforge/fuel-astute/commit/?id=1aa7efb3905a9b5d48718b70a6321cf914c7aa71
Submitter: Jenkins
Branch: master

commit 1aa7efb3905a9b5d48718b70a6321cf914c7aa71
Author: Vladimir Sharshov (warpc) <email address hidden>
Date: Mon Nov 17 13:16:16 2014 +0300

    Check existing or upload cirros by one request

    Sometimes mcollective do not receive answer and
    we can upload 2 images instead of one. This
    solution join 2 command into 1: check existing
    and upload if not exist. Now this is idempotency
    operation.

    Change-Id: I920ab585a4324204bc8060850e2838a88db3b2e8
    Closes-Bug: #1342039

[Anastasia Palkina (apalkina)]

Verified on ISO #76

"build_id": "2015-01-23_22-54-45", "ostf_sha": "95054878eff472eb6a0d9d185ce65e3105dde632", "build_number": "76", "auth_required": true, "api": "1.0", "nailgun_sha": "e75ac13a46d6c40fe7c90af51c053c0e0a818808", "production": "docker", "fuelmain_sha": "", "astute_sha": "48829a27b8041c36b14e34cc2ae760c8f57703a9", "feature_groups": ["mirantis"], "release": "6.1", "release_versions": {"2014.2-6.1": {"VERSION": {"build_id": "2015-01-23_22-54-45", "ostf_sha": "95054878eff472eb6a0d9d185ce65e3105dde632", "build_number": "76", "api": "1.0", "nailgun_sha": "e75ac13a46d6c40fe7c90af51c053c0e0a818808", "production": "docker", "fuelmain_sha": "", "astute_sha": "48829a27b8041c36b14e34cc2ae760c8f57703a9", "feature_groups": ["mirantis"], "release": "6.1", "fuellib_sha": "4c61bf87a96453a750df3e81606ba9fe21c2059f"}}}, "fuellib_sha": "4c61bf87a96453a750df3e81606ba9fe21c2059f"

http://paste.openstack.org/show/162305/