OSTF test 'Advanced stack actions: suspend, resume and check' is failed duy to InternalServerError

[Tatyana Kuterina]

Detailed bug description:
Found on CI:
https://product-ci.infra.mirantis.net/job/9.x.acceptance.ubuntu.mixed_os_components/4/console

Test: Deployment with 3 controllers, NeutronTUN, with Murano, Sahara and Ceilometer
Test Group: mixed_components_murano_sahara_ceilometer

Steps to reproduce:
        1. Create new environment
        2. Choose Neutron, TUN
        3. Choose Murano
        4. Choose Sahara
        5. Choose Ceilometer
        6. Add 3 controller
        7. Add 1 compute
        8. Add 1 cinder
        9. Add 3 mongo
        10. Verify networks
        11. Deploy the environment
        12. Verify networks
        13. Run OSTF tests

Expected results:
    All ostf tests are passed
Actual results:
    Failed 1 OSTF test:
  - Advanced stack actions: suspend, resume and check (error)

Error in ostf log:
InternalServerError: Request Failed: internal server error while processing your request.

Description of the environment:
    9.1 snapshot #153

https://drive.google.com/a/mirantis.com/file/d/0Bz15vbpS5ZPNbDBYOHlaTkZvNmc/view?usp=sharing

[Alexander Nagovitsyn (gluk12189)]

Errors in logs:
The server could not comply with the request since it is either malformed or otherwise incorrect

500 Internal Server Error: The server has either erred or is incapable of performing the requested operation. (HTTP 500)\n", "type": "StackValidationFailed"}, "title": "Bad Request"}
NeutronError": {"message": "Unable to complete operation on network afd2304c-bfa0-4f6a-a555-dc000bbbf17a. There are one or more ports still in use on the network.", "type": "NetworkInUse", "detail": ""}}

It seems that the server cannt handle requests for tests.
Looking at the state of the node, is not surprising:
slave-01:
    Host node-2
    Roles:
       Hiera:
           - controller
       Nailgun:
           - controller
    Memory:
       RAM:
          used 2927
          cached 63
          free 82
          shared 7
          total 3009
          buffers 4
       SWAP:
          total 3071
          free 665
          used 2406

As you can see the VM uses the SWAP memory. I guess that is not running very fast. It is amazing that we dont see more serious problems in the OSTF tests using this configuration.

[Alexander Nagovitsyn (gluk12189)]

Also i found this error:
2016-09-09 13:54:10.408 2558 ERROR heat.engine.service [req-42c3ccea-8485-44a2-af50-1fc8de7e003c - - - - -] Service 4e3f36de-9232-4cc9-b0fc-230b160f850d update failed: (_mysql_exceptions.OperationalError) (2013, "Lost connection to MySQL server at 'reading initial communication packet', system error: 0") [SQL: u'SELECT 1']

It looks like a this bug https://bugs.launchpad.net/fuel/+bug/1621537

[Alexander Nagovitsyn (gluk12189)]

Also this bug https://bugs.launchpad.net/congress/+bug/1621268
and this https://bugs.launchpad.net/fuel/+bug/1621448

[Alexander Nagovitsyn (gluk12189)]

I cant reproduce this on any normal configuration.

[Timur Nurlygayanov (tnurlygayanov)]

The status changed to Invalid because QA can't reproduce the issue.