Multiple OSTF failed on environment with ceilomenter

[Dmitry Tyzhnenko]

Platform Ceilometer OSTF tests failed, but cli commands work well.

Scenario:
1. Create new environment
2. Choose Neutron, VLAN
3. Choose Ceph for volumes and images, ceph for ephemeral and Rados GW for objects
4. Add 3 controller
5. Add 1 compute
6. Add 3 mongo nodes
7. Verify networks
8. Deploy cluster
9. Verify networks
10. Run OSTF

Expecter result:
 All step pass

Actual result:
  On 10 step failed multiply tests:

Ceilometer test to check alarm state and get Nova metrics - 4. Get instance pollsters.

Ceilometer test to check notifications from Glance - 2. Get image notifications.
Ceilometer test to check notifications from Keystone - 2. Get project notifications.
Ceilometer test to check notifications from Neutron - 2. Get network notifications.
Ceilometer test to check notifications from Cinder - 4. Get volume snapshot notifications.
Check stack autoscaling - 11. Wait for the 2nd instance to be launched

Logs snapshot - https://drive.google.com/a/mirantis.com/file/d/0B8U7EvTbuAOlaDljMncwV2M2WGM/view?usp=sharing

[Vitaly Gusev (vgusev)]

Tests fail because of missed options in ceilometer haproxy config, see bug https://bugs.launchpad.net/fuel/+bug/1533680

[Dmitry Tyzhnenko (dtyzhnenko)]

This bug dosn't related to #1533680. ostf.log in snapshot log contains nothing about HTTPSConnectionPool.

Reproduced on #429 iso

VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "8.0"
  api: "1.0"
  build_number: "429"
  build_id: "429"
  fuel-nailgun_sha: "12b15b2351e250af41cc0b10d63a50c198fe77d8"
  python-fuelclient_sha: "4f234669cfe88a9406f4e438b1e1f74f1ef484a5"
  fuel-agent_sha: "df16d41cd7a9445cf82ad9fd8f0d53824711fcd8"
  fuel-nailgun-agent_sha: "92ebd5ade6fab60897761bfa084aefc320bff246"
  astute_sha: "c7ca63a49216744e0bfdfff5cb527556aad2e2a5"
  fuel-library_sha: "3eaf4f4a9b88b287a10cc19e9ce6a62298cc4013"
  fuel-ostf_sha: "214e794835acc7aa0c1c5de936e93696a90bb57a"
  fuel-mirror_sha: "b62f3cce5321fd570c6589bc2684eab994c3f3f2"
  fuelmenu_sha: "85de57080a18fda18e5325f06eaf654b1b931592"
  shotgun_sha: "63645dea384a37dde5c01d4f8905566978e5d906"
  network-checker_sha: "9f0ba4577915ce1e77f5dc9c639a5ef66ca45896"
  fuel-upgrade_sha: "616a7490ec7199f69759e97e42f9b97dfc87e85b"
  fuelmain_sha: "e8e36cff332644576d7853c80b8a53d5b955420a"

[Dmitry Tyzhnenko (dtyzhnenko)]

Scenario for reproduce:
1. Create new environment
2. Choose Neutron, TUN
3. Choose Ceilometer
4. Add 3 controller+mongo
5. Add 1 compute
6. Add 1 cinder
7. Change disk configuration for all Mongo nodes. Change 'MongoDB' volume for vdc
8. Verify networks
9. Deploy the environment
10. Verify networks
11. Run OSTF tests

Log snapshot - https://drive.google.com/a/mirantis.com/file/d/0B8U7EvTbuAOlTUI5d2RtX2tkams/view?usp=sharing

[Dmitry Tyzhnenko (dtyzhnenko)]

Scenario:
1. Create new environment
2. Choose Neutron, TUN
3. Choose Ceph for volumes and Ceph for images
4. Choose Ceph RadosGW for objects
5. Choose Ceilometer
6. Add 3 controller+ceph
7. Add 1 compute
8. Add 3 mongo
9. Change disk configuration for both Mongo nodes. Change 'MongoDB' volume for vdc
10. Verify networks
11. Deploy the environment
12. Verify networks
13. Run OSTF tests

Log snapshot - https://drive.google.com/a/mirantis.com/file/d/0B8U7EvTbuAOlTUI5d2RtX2tkams/view?usp=sharing

[Dmitry Tyzhnenko (dtyzhnenko)]

Due reproduce since last acceptance cycle, rise to critical.

[Artem Panchenko (apanchenko-8)]

Guys, I'm pretty sure it's a duplicate of bug #1520664. MOS QA folks, could you confirm that?

[Timur Nurlygayanov (tnurlygayanov)]

Hi Artem, ok, we will take a look asap.

Vitaly, could you please check and change the status if this issue is already fixed?

Thank you!

[Vitaly Gusev (vgusev)]

Hi folks,

The tests were already fixed in patch https://review.openstack.org/#/c/268977/, Ceilometer SWARM tests successfully passed https://product-ci.infra.mirantis.net/view/8.0_swarm/job/8.0.system_test.ubuntu.services_ha.ceilometer/117/

[Nastya Urlapova (aurlapova)]

@Vitaliy,
we use very clear policy - attach version of Fuel, that was used for verification.

[Vitaly Gusev (vgusev)]

Verified on ISO:
VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "8.0"
  api: "1.0"
  build_number: "466"
  build_id: "466"
  fuel-nailgun_sha: "f81311bbd6fee2665e3f96dcac55f72889b2f38c"
  python-fuelclient_sha: "4f234669cfe88a9406f4e438b1e1f74f1ef484a5"
  fuel-agent_sha: "6823f1d4005a634b8436109ab741a2194e2d32e0"
  fuel-nailgun-agent_sha: "b2bb466fd5bd92da614cdbd819d6999c510ebfb1"
  astute_sha: "b81577a5b7857c4be8748492bae1dec2fa89b446"
  fuel-library_sha: "fe03d887361eb80232e9914eae5b8d54304df781"
  fuel-ostf_sha: "ab5fd151fc6c1aa0b35bc2023631b1f4836ecd61"
  fuel-mirror_sha: "b62f3cce5321fd570c6589bc2684eab994c3f3f2"
  fuelmenu_sha: "fac143f4dfa75785758e72afbdc029693e94ff2b"
  shotgun_sha: "63645dea384a37dde5c01d4f8905566978e5d906"
  network-checker_sha: "9f0ba4577915ce1e77f5dc9c639a5ef66ca45896"
  fuel-upgrade_sha: "616a7490ec7199f69759e97e42f9b97dfc87e85b"
  fuelmain_sha: "727f7076f04cb0caccc9f305b149a2b5b5c2af3a"