HAProxy marks swift backends as down because xinetd didn't start swiftcheck service

[Artem Panchenko]

After successful deployment BVT failed while checking HAProxy back-ends:

AssertionError: HAProxy back-ends are DOWN. {'exit_code': 0, 'stderr': [], 'stdout': ['swift node-4 Status: DOWN/L4CON Sessions: 0 Rate: 0 \n', 'swift node-5 Status: DOWN/L4CON Sessions: 0 Rate: 0 \n']}

Swift back-end was marked as DOWN on non-primary controllers, because 'swiftcheck' service didn't run on it. According to puppet logs 'xinetd' service was reloaded before 'swiftcheck 49001/tcp' line was added to /etc/services file (node-4):

http://paste.openstack.org/show/264129/

<27>Jun 4 19:29:00 node-4 xinetd[10557]: service/protocol combination not in /etc/services: swiftcheck/tcp

Steps to reproduce:

            1. Create cluster: CentOS, NovaVlan
            2. Add 3 nodes with controller roles
            3. Add 2 nodes with compute roles
            4. Set up cluster to use Network VLAN manager with 8 networks
            5. Deploy the cluster
            6. Check HAProxy back ends

Expected result:

- all back-ends are UP

Actual result:

- back-ends for Swift are DOWN on some controllers

Fuel version info (6.1 build #507): http://paste.openstack.org/show/264127/

Diagnostic snapshot is attached.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/188621

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/188621
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=907e1b99b31c7c88955ec5edc50efba5d9e5342c
Submitter: Jenkins
Branch: master

commit 907e1b99b31c7c88955ec5edc50efba5d9e5342c
Author: Artem Panchenko <email address hidden>
Date: Fri Jun 5 01:55:38 2015 +0300

    Reserve 49001/tcp port before starting swiftcheck

    Require adding of '49001/tcp' port as reserved for swiftcheck
    service to '/etc/services' before configuring it under xinetd.

    Change-Id: Ia01959d205d186ce2e21afec9673df0fd6d94736
    Closes-bug: #1462142

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/188704

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/188704
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=506e2840dabcb96b564e8a8f3d32a36bd31c6264
Submitter: Jenkins
Branch: master

commit 506e2840dabcb96b564e8a8f3d32a36bd31c6264
Author: Vladimir Kuklin <email address hidden>
Date: Fri Jun 5 11:41:34 2015 +0300

    Fix service order for galera check xinetd

    Otherwise galeracheck will not work if /etc/services
    is not edited before xinetd service is declared

    Change-Id: Ia336f722e8e9f8aa6e6d25cc7e5bed9081d02c39
    Related-bug: #1462142

[Tatyanka (tatyana-leontovich)]

verified 521 iso