[regression] 2 Heat-related Tempest tests failed: test_exceed_max_template_size_fails and test_exceed_max_resources_per_stack

[Timur Nurlygayanov]

Detailed bug description:
2 Heat-related Tempest tests failed on MOS 9.0 builds.

Steps to reproduce:
1. Deploy environment with MOS 9.0 Mitaka ISO #68
2. Run full Tempest tests suite using Rally tool (rally verify)

Expected results:
All tests passed

Actual result:
117 test failed, 2 of them related to Heat and it looks like the root of the issue in Heat code / configuration.

Traceback (most recent call last):
  File "/home/rally/.rally/tempest/for-deployment-9f73a386-2a07-4078-a937-2caab60f6d45/tempest/api/orchestration/stacks/test_limits.py", line 48, in test_exceed_max_resources_per_stack
    stack_name, template)
  File "/usr/local/lib/python2.7/dist-packages/testtools/testcase.py", line 434, in assertRaises
    self.assertThat(our_callable, matcher)
  File "/usr/local/lib/python2.7/dist-packages/testtools/testcase.py", line 447, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: <bound method type.create_stack of <class 'tempest.api.orchestration.stacks.test_limits.TestServerStackLimits'>> returned 'tempest-heat-1973933374/d601e141-6033-4ac1-a7d0-76ba2b725646'

Reproducibility:
100 %. Reproduced several times on different configurations:

Workaround:
We don't know the workaround yet.

Impact:
Some Heat functionality doesn't work, all user who use Heat are affected.

Description of the environment:
- Operation system: Ubuntu 14.04
- Versions of components: -
- Reference architecture: Neutron VLAN, HA, Cinder LVM, Swift
- Network model: Neutron VLANs
- Related projects installed: -

[Timur Nurlygayanov (tnurlygayanov)]

This issue reproduced only on MOS 9.0 and don't reproduced on MOS 8.0 / MOS 7.0, so, it looks like regression issue.

[Sergey Kraynev (skraynev)]

It happens, because we change these both limits for correct work Sahara on big scale. So it's expected behavior and should be fixed by patching tempest test for MOS. Note, that in master these tests work without error, because uses default values.

[Timur Nurlygayanov (tnurlygayanov)]

I've described the defect in Tempest project: https://bugs.launchpad.net/tempest/+bug/1556135

[Timur Nurlygayanov (tnurlygayanov)]

it is possible to configure these parameters in Tempest configuration file:
https://github.com/openstack/tempest/blob/55df9d63ca846628929c16a8996b6a37fccd6d76/tempest/config.py#L876-L881

we need to add fix for these parameters in out Tempest jobs.

[Sergii Turivnyi (sturivnyi)]

need to add parameters to tempest.conf with values same as in /etc/heat/heat.conf

max_template_size = 544000000
max_resources_per_stack = 1000
max_json_body_size = 10880000

[Sergii Turivnyi (sturivnyi)]

Parameters added to CI job

[Sergii Turivnyi (sturivnyi)]

It works. Was tested

[Timur Nurlygayanov (tnurlygayanov)]

So is it a regression issue for 9.1?

[Oleksiy Butenko (obutenko)]

Hi! No, it's not regression.
This tests failed only with radosGW
we need skip this test on env with radosGW
I close this bug - Mark as duplicate https://bugs.launchpad.net/mos/+bug/1442193