Deployment with detached-keystone plugin failed - undefined method `collect' for nil:NilClass

[Vladimir Khlyunev]

ISO #182

Steps:
1) Build&Install detach-db and detach-keystone.
https://github.com/stackforge/fuel-plugin-detach-keystone
https://github.com/stackforge/fuel-plugin-detach-database
2) Create cluster:
HA, Neutron VLAN, Cinder, No additional components
3) Add:
3 controllers, 1 database, 3 keystones, 2 cinder+compute
4) Deploy the cluster

Result: (complete trace http://paste.openstack.org/show/420219/ )
2015-08-17 14:56:53 ERR Could not prefetch keystone_endpoint provider 'openstack': undefined method `collect' for nil:NilClass

Snapshot will be added after deployment stops.

[Vladimir Khlyunev (vkhlyunev)]

After investigating https://bugs.launchpad.net/bugs/1484963 :
This bug is not duplicate because main error is
2015-08-17 14:56:53 ERR Could not prefetch keystone_endpoint provider 'openstack': undefined method `collect' for nil:NilClass

[Vladimir Khlyunev (vkhlyunev)]

Matthew have no ability to fix this bug. Bartlomiej, could you help him?

[Vladimir Khlyunev (vkhlyunev)]

This bug still blocking QA, Matthew is working on another critical bug, Bartlomiej is not responding. Library-team, please decide who will fix this bug and when.

[Bartłomiej Piotrowski (bpiotrowski)]

Unfortunately I have hit different issue locally that I'll investigate later. Looking at Vladimir's lab, deployment failed because of failure of getting endpoint list via openstackclient, although I'm not completely sure why it happens.

From 'openstack --debug endpoint list' output:

ConnectionRefused: Unable to establish connection to https://public.fuel.local:5000/v2.0/tokens

The problem is that in /etc/hosts, public.fuel.local points to wrong IP – public_vip instead of service_endpoint_vip. If user modifies /etc/hosts manually, openstackclient successfully returns the endpoint list.

[Bartłomiej Piotrowski (bpiotrowski)]

Fixed by https://review.openstack.org/#/c/213743/ and https://review.openstack.org/#/c/214109/ .

[Matthew Mosesohn (raytrac3r)]

public_vip is used for non-ssl deployments in keystone.conf, breaking token generation. It should be set to public_service_endpoint VIP for non-SSL deployments.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/218851

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/218851
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=e055af9dee6fbaf84cb8e86a9ebff9b25b07792e
Submitter: Jenkins
Branch: master

commit e055af9dee6fbaf84cb8e86a9ebff9b25b07792e
Author: Matthew Mosesohn <email address hidden>
Date: Mon Aug 31 16:18:33 2015 +0300

    Use public_service_endpoint if defined for keystone public addr

    Keystone with detached services should advertise the correct
    public address for issuing tokens.

    Note that for SSL deployment, the public SSL hostname is used
    and is not impacted.

    Change-Id: I866446364651a98bb742414640b1e41362570beb
    Closes-Bug: #1485641

[Maksym Strukov (unbelll)]

Verified as fixed in 7.0-287

{"build_id": "287", "build_number": "287", "release_versions": {"2015.1.0-7.0": {"VERSION": {"build_id": "287", "build_number": "287", "api": "1.0", "fuel-library_sha": "43224223dab8cf9627b5ecf737e60216a3fdd114", "nailgun_sha": "46a7a2177a0b7ef91422284c1c90295fee8f5c84", "feature_groups": ["mirantis"], "fuel-nailgun-agent_sha": "d7027952870a35db8dc52f185bb1158cdd3d1ebd", "openstack_version": "2015.1.0-7.0", "fuel-agent_sha": "082a47bf014002e515001be05f99040437281a2d", "production": "docker", "python-fuelclient_sha": "1ce8ecd8beb640f2f62f73435f4e18d1469979ac", "astute_sha": "a717657232721a7fafc67ff5e1c696c9dbeb0b95", "fuel-ostf_sha": "1f08e6e71021179b9881a824d9c999957fcc7045", "release": "7.0", "fuelmain_sha": "6b83d6a6a75bf7bca3177fcf63b2eebbf1ad0a85"}}}, "auth_required": true, "api": "1.0", "fuel-library_sha": "43224223dab8cf9627b5ecf737e60216a3fdd114", "nailgun_sha": "46a7a2177a0b7ef91422284c1c90295fee8f5c84", "feature_groups": ["mirantis"], "fuel-nailgun-agent_sha": "d7027952870a35db8dc52f185bb1158cdd3d1ebd", "openstack_version": "2015.1.0-7.0", "fuel-agent_sha": "082a47bf014002e515001be05f99040437281a2d", "production": "docker", "python-fuelclient_sha": "1ce8ecd8beb640f2f62f73435f4e18d1469979ac", "astute_sha": "a717657232721a7fafc67ff5e1c696c9dbeb0b95", "fuel-ostf_sha": "1f08e6e71021179b9881a824d9c999957fcc7045", "release": "7.0", "fuelmain_sha": "6b83d6a6a75bf7bca3177fcf63b2eebbf1ad0a85"}