[check][scenario 10] overcloud deploy fails while adding security group rule to amphora image

[chandan kumar]

Scenario 10 container multinode check jobs are failing while deploying overcloud image with reate security group rule to open amphora management ssh port.

http://logs.openstack.org/98/604298/207/check/tripleo-ci-centos-7-scenario010-multinode-oooq-container/d8808f2/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2019-02-01_02_03_24

2019-02-01 02:03:24 | TASK [octavia-overcloud-config : setting fact for management network security group] ***
2019-02-01 02:03:24 | Friday 01 February 2019 02:03:11 +0000 (0:00:08.166) 0:00:49.839 *******
2019-02-01 02:03:24 | ok: [centos-7-rax-dfw-0002314752]
2019-02-01 02:03:24 |
2019-02-01 02:03:24 | TASK [octavia-overcloud-config : create security group rule to open amphora management ssh port] ***
2019-02-01 02:03:24 | Friday 01 February 2019 02:03:11 +0000 (0:00:00.236) 0:00:50.075 *******
2019-02-01 02:03:24 | fatal: [centos-7-rax-dfw-0002314752]: FAILED! => {
2019-02-01 02:03:24 | "changed": false,
2019-02-01 02:03:24 | "cmd": "if [[ $(openstack security group rule list lb-mgmt-sec-grp --protocol tcp --ingress -f value 2>&1 | grep \"0.0.0.0/0 22:22\") == \"\" ]]; then\n openstack security group rule create --protocol tcp --dst-port 22 lb-mgmt-sec-grp\n fi",
2019-02-01 02:03:24 | "delta": "0:00:06.359407",
2019-02-01 02:03:24 | "end": "2019-02-01 02:03:18.615807",
2019-02-01 02:03:24 | "rc": 1,
2019-02-01 02:03:24 | "start": "2019-02-01 02:03:12.256400"
2019-02-01 02:03:24 | }
2019-02-01 02:03:24 |
2019-02-01 02:03:24 | STDERR:
2019-02-01 02:03:24 |
2019-02-01 02:03:24 | Error while executing command: ResourceNotFound: 404, Security group 25e8e313-8962-4727-9747-f7448dcf4a55 does not exist
2019-02-01 02:03:24 |

I needs to be investigated why it is failing at that place.

[Carlos Goncalves (cgoncalves)]

There's something odd in Neutron. The security group is created but then adding rule to it fails:

create failed (client error): The resource could not be found.
POST /v2.0/security-group-rules HTTP/1.1" status: 404

http://logs.openstack.org/98/604298/207/check/tripleo-ci-centos-7-scenario010-multinode-oooq-container/d8808f2/logs/subnode-2/var/log/containers/neutron/server.log.txt.gz#_2019-02-01_02_03_18_389

Job started failing on Jan 25. I'm thinking https://review.openstack.org/#/c/628691/ is what caused this and https://review.openstack.org/#/c/633778/ will fix the regression. Awaiting for patch to merge (+W'd already) to recheck job.

[wes hayutin (weshayutin)]

This looks like it's in progress
https://review.openstack.org/#/c/643934/
Depends-On: I6fd6add76e21e0b3d8b4ee0fbe5798a1f2a25205
Depends-On: I8fd2874be23e41073d95d84fdce3e383b18060db

Brent, any updates from you?

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/634286
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=18eceb2195413380971edc38657eb38c2527678a
Submitter: Zuul
Branch: master

commit 18eceb2195413380971edc38657eb38c2527678a
Author: Brent Eagles <email address hidden>
Date: Thu Jan 31 17:36:39 2019 +0000

    Use Octavia's project for lb security group ops

    The security group is created under the service tenant but the
    operations are being run as the admin tenant causing rule creation
    operations to fail.

    Change-Id: I3cb1918d94ab56fc96663ae3db6015bcd5c9c547
    Closes-Bug: #1814228

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-common 10.6.1 release.