Fuel for OpenStack

keystoneclient.session Failure: SSL exception connecting to ...

Bug #1478649 reported by Max Yatsenko
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Released
High
Max Yatsenko
Fuel for OpenStack 7.0

Bug Description

The following "Health Checks" were failed:
- Create volume and boot instance from it
- Create volume and attach it to instance

ISO: http://jenkins-product.srt.mirantis.net:8080/view/custom_iso/job/custom_7.0_iso/655/

I get an error when I try to create an instance (boot instance from volume) :

# nova boot --flavor 1 --boot-volume a1d150a7-6883-48ba-a863-b87726a642d2 test_instance_from_volume1
ERROR (BadRequest): Block Device Mapping is Invalid: failed to get volume 1ca99f45-bed5-4111-b19b-b6d0f87311fe. (HTTP 400) (Request-ID: req-2c0fa805-3dc9-4ecf-b1ea-d6bf751ea138)

--------------------------------------------------------------------------------------

/var/log/nova/nova-api.log contains the following errors:

# grep 2c0fa805-3dc9-4ecf-b1ea-d6bf751ea138 /var/log/* -r
/var/log/nova/nova-api.log:2015-07-27 13:37:28.517 12255 INFO keystoneclient.session [req-2c0fa805-3dc9-4ecf-b1ea-d6bf751ea138 638538f9a80b4a43a1c52857916ee4d5 559bffdb72e64cc28ba3f8ef832eaedd - - -] Failure: SSL exception connecting to https://172.16.51.20:8776/v2/559bffdb72e64cc28ba3f8ef832eaedd/volumes/1ca99f45-bed5-4111-b19b-b6d0f87311fe: hostname '172.16.51.20' doesn't match u'public.fuel.local'. Retrying in 0.5s.
/var/log/nova/nova-api.log:2015-07-27 13:37:29.024 12255 INFO keystoneclient.session [req-2c0fa805-3dc9-4ecf-b1ea-d6bf751ea138 638538f9a80b4a43a1c52857916ee4d5 559bffdb72e64cc28ba3f8ef832eaedd - - -] Failure: Unable to establish connection to https://172.16.51.20:8776/v2/559bffdb72e64cc28ba3f8ef832eaedd/volumes/1ca99f45-bed5-4111-b19b-b6d0f87311fe. Retrying in 1.0s.
/var/log/nova/nova-api.log:2015-07-27 13:37:30.071 12255 INFO keystoneclient.session [req-2c0fa805-3dc9-4ecf-b1ea-d6bf751ea138 638538f9a80b4a43a1c52857916ee4d5 559bffdb72e64cc28ba3f8ef832eaedd - - -] Failure: SSL exception connecting to https://172.16.51.20:8776/v2/559bffdb72e64cc28ba3f8ef832eaedd/volumes/1ca99f45-bed5-4111-b19b-b6d0f87311fe: hostname '172.16.51.20' doesn't match u'public.fuel.local'. Retrying in 2.0s.
/var/log/nova/nova-api.log:2015-07-27 13:37:32.083 12255 INFO nova.api.openstack.wsgi [req-2c0fa805-3dc9-4ecf-b1ea-d6bf751ea138 638538f9a80b4a43a1c52857916ee4d5 559bffdb72e64cc28ba3f8ef832eaedd - - -] HTTP exception thrown: Block Device Mapping is Invalid: failed to get volume 1ca99f45-bed5-4111-b19b-b6d0f87311fe.
/var/log/nova/nova-api.log:2015-07-27 13:37:32.108 12255 INFO nova.osapi_compute.wsgi.server [req-2c0fa805-3dc9-4ecf-b1ea-d6bf751ea138 638538f9a80b4a43a1c52857916ee4d5 559bffdb72e64cc28ba3f8ef832eaedd - - -] 192.168.0.3 "POST /v2/559bffdb72e64cc28ba3f8ef832eaedd/os-volumes_boot HTTP/1.1" status: 400 len: 351 time: 3.6823351
/var/log/nova-all.log:<182>Jul 27 13:37:28 node-1 nova-api 2015-07-27 13:37:28.517 12255 INFO keystoneclient.session [req-2c0fa805-3dc9-4ecf-b1ea-d6bf751ea138 638538f9a80b4a43a1c52857916ee4d5 559bffdb72e64cc28ba3f8ef832eaedd - - -] Failure: SSL exception connecting to https://172.16.51.20:8776/v2/559bffdb72e64cc28ba3f8ef832eaedd/volumes/1ca99f45-bed5-4111-b19b-b6d0f87311fe: hostname '172.16.51.20' doesn't match u'public.fuel.local'. Retrying in 0.5s.
/var/log/nova-all.log:<182>Jul 27 13:37:29 node-1 nova-api 2015-07-27 13:37:29.024 12255 INFO keystoneclient.session [req-2c0fa805-3dc9-4ecf-b1ea-d6bf751ea138 638538f9a80b4a43a1c52857916ee4d5 559bffdb72e64cc28ba3f8ef832eaedd - - -] Failure: Unable to establish connection to https://172.16.51.20:8776/v2/559bffdb72e64cc28ba3f8ef832eaedd/volumes/1ca99f45-bed5-4111-b19b-b6d0f87311fe. Retrying in 1.0s.
/var/log/nova-all.log:<182>Jul 27 13:37:30 node-1 nova-api 2015-07-27 13:37:30.071 12255 INFO keystoneclient.session [req-2c0fa805-3dc9-4ecf-b1ea-d6bf751ea138 638538f9a80b4a43a1c52857916ee4d5 559bffdb72e64cc28ba3f8ef832eaedd - - -] Failure: SSL exception connecting to https://172.16.51.20:8776/v2/559bffdb72e64cc28ba3f8ef832eaedd/volumes/1ca99f45-bed5-4111-b19b-b6d0f87311fe: hostname '172.16.51.20' doesn't match u'public.fuel.local'. Retrying in 2.0s.
/var/log/nova-all.log:<182>Jul 27 13:37:32 node-1 nova-api 2015-07-27 13:37:32.083 12255 INFO nova.api.openstack.wsgi [req-2c0fa805-3dc9-4ecf-b1ea-d6bf751ea138 638538f9a80b4a43a1c52857916ee4d5 559bffdb72e64cc28ba3f8ef832eaedd - - -] HTTP exception thrown: Block Device Mapping is Invalid: failed to get volume 1ca99f45-bed5-4111-b19b-b6d0f87311fe.
/var/log/nova-all.log:<182>Jul 27 13:37:32 node-1 nova-api 2015-07-27 13:37:32.108 12255 INFO nova.osapi_compute.wsgi.server [req-2c0fa805-3dc9-4ecf-b1ea-d6bf751ea138 638538f9a80b4a43a1c52857916ee4d5 559bffdb72e64cc28ba3f8ef832eaedd - - -] 192.168.0.3 "POST /v2/559bffdb72e64cc28ba3f8ef832eaedd/os-volumes_boot HTTP/1.1" status: 400 len: 351 time: 3.6823351
/var/log/upstart/nova-api.log:2015-07-27 13:37:28.517 12255 INFO keystoneclient.session [req-2c0fa805-3dc9-4ecf-b1ea-d6bf751ea138 638538f9a80b4a43a1c52857916ee4d5 559bffdb72e64cc28ba3f8ef832eaedd - - -] Failure: SSL exception connecting to https://172.16.51.20:8776/v2/559bffdb72e64cc28ba3f8ef832eaedd/volumes/1ca99f45-bed5-4111-b19b-b6d0f87311fe: hostname '172.16.51.20' doesn't match u'public.fuel.local'. Retrying in 0.5s.
/var/log/upstart/nova-api.log:2015-07-27 13:37:29.024 12255 INFO keystoneclient.session [req-2c0fa805-3dc9-4ecf-b1ea-d6bf751ea138 638538f9a80b4a43a1c52857916ee4d5 559bffdb72e64cc28ba3f8ef832eaedd - - -] Failure: Unable to establish connection to https://172.16.51.20:8776/v2/559bffdb72e64cc28ba3f8ef832eaedd/volumes/1ca99f45-bed5-4111-b19b-b6d0f87311fe. Retrying in 1.0s.
/var/log/upstart/nova-api.log:2015-07-27 13:37:30.071 12255 INFO keystoneclient.session [req-2c0fa805-3dc9-4ecf-b1ea-d6bf751ea138 638538f9a80b4a43a1c52857916ee4d5 559bffdb72e64cc28ba3f8ef832eaedd - - -] Failure: SSL exception connecting to https://172.16.51.20:8776/v2/559bffdb72e64cc28ba3f8ef832eaedd/volumes/1ca99f45-bed5-4111-b19b-b6d0f87311fe: hostname '172.16.51.20' doesn't match u'public.fuel.local'. Retrying in 2.0s.
/var/log/upstart/nova-api.log:2015-07-27 13:37:32.083 12255 INFO nova.api.openstack.wsgi [req-2c0fa805-3dc9-4ecf-b1ea-d6bf751ea138 638538f9a80b4a43a1c52857916ee4d5 559bffdb72e64cc28ba3f8ef832eaedd - - -] HTTP exception thrown: Block Device Mapping is Invalid: failed to get volume 1ca99f45-bed5-4111-b19b-b6d0f87311fe.
/var/log/upstart/nova-api.log:2015-07-27 13:37:32.108 12255 INFO nova.osapi_compute.wsgi.server [req-2c0fa805-3dc9-4ecf-b1ea-d6bf751ea138 638538f9a80b4a43a1c52857916ee4d5 559bffdb72e64cc28ba3f8ef832eaedd - - -] 192.168.0.3 "POST /v2/559bffdb72e64cc28ba3f8ef832eaedd/os-volumes_boot HTTP/1.1" status: 400 len: 351 time: 3.6823351

--------------------------------------------------------------------------------------

To solve the issue "publicur"l endpoint for service: "cinderv2" was udpated
from: https://172.16.51.20:8776/v2/%(tenant_id)s
to: https://public.fuel.local:8776/v2/%(tenant_id)s

Max Yatsenko (myatsenko)
Changed in mos:
milestone: none → 7.0
Max Yatsenko (myatsenko)
no longer affects: mos
Nastya Urlapova (aurlapova)
Changed in fuel:
milestone: none → 7.0
assignee: nobody → Fuel Library Team (fuel-library)
importance: Undecided → High
status: New → Confirmed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (master)

Fix proposed to branch: master
Review: https://review.openstack.org/206455

Changed in fuel:
assignee: Fuel Library Team (fuel-library) → Vasyl Saienko (vsaienko)
status: Confirmed → In Progress
Revision history for this message
Max Yatsenko (myatsenko) wrote :

The same issue we have for the following endpoints in keystone endpoint list:
- nova
- nuetron
- heat
- swift
- glance

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/206493

Changed in fuel:
assignee: Vasyl Saienko (vsaienko) → Max Yatsenko (myatsenko)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to fuel-library (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/206517

OpenStack Infra (hudson-openstack)
Changed in fuel:
assignee: Max Yatsenko (myatsenko) → Vasyl Saienko (vsaienko)
OpenStack Infra (hudson-openstack)
Changed in fuel:
assignee: Vasyl Saienko (vsaienko) → Max Yatsenko (myatsenko)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (master)

Fix proposed to branch: master
Review: https://review.openstack.org/206959

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to fuel-library (master)

Reviewed: https://review.openstack.org/206517
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=f74af445f26afb1740d9762a1bb958e9975feffc
Submitter: Jenkins
Branch: master

commit f74af445f26afb1740d9762a1bb958e9975feffc
Author: Sergey Kolekonov <email address hidden>
Date: Tue Jul 28 15:20:26 2015 +0300

    Fix using of public address with SSL for Neutron

    Neutron public endpoint should use public ssl hostname

    Change-Id: Ibbc5430495ddc4d1408a517517b658b93b2e0eea
    Related-bug: #1478649

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (master)

Reviewed: https://review.openstack.org/206455
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=9520ea62b97ea254de2464a53efb440c079292cd
Submitter: Jenkins
Branch: master

commit 9520ea62b97ea254de2464a53efb440c079292cd
Author: vsaienko <email address hidden>
Date: Tue Jul 28 12:58:55 2015 +0300

    Fix using of public address with SSL for Cinder

     * Cinder public endpoint should use hostname
       instead of ip.
     * add noop tests for cinder::keystone::auth

    Closes-Bug: #1478649

    Change-Id: Ife533fa514241b98f75173620ac01be909abe39f

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to fuel-library (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/208070

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Related fix proposed to branch: master
Review: https://review.openstack.org/208111

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (master)

Reviewed: https://review.openstack.org/206493
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=42a29aa19face1ad6e91404014c29bc79deff4ab
Submitter: Jenkins
Branch: master

commit 42a29aa19face1ad6e91404014c29bc79deff4ab
Author: Maksym Yatsenko <email address hidden>
Date: Tue Jul 28 14:15:23 2015 +0300

    Fix using of public address with SSL for Heat

    * Heat public endpoint should use public ssl hostname instead ip
    address.
    * Replace {public|admin|internal}_{address|protocol} with {public|admin}_url
    * Update noop tests

    Closes-Bug: #1478649

    Change-Id: Id7c5dba88a573c5460a9c253074a78ecad09f976

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Reviewed: https://review.openstack.org/206959
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=d1cdc5d99de94c88437ad1b2aba7d1f9374b645d
Submitter: Jenkins
Branch: master

commit d1cdc5d99de94c88437ad1b2aba7d1f9374b645d
Author: Maksym Yatsenko <email address hidden>
Date: Wed Jul 29 12:28:58 2015 +0300

     Fix using of public address with SSL for Nova

    * Nova public endpoint should use public ssl hostname instead ip
    address
    * Update noop tests

    Closes-Bug: #1478649

    Change-Id: If5ea922f7279d94cc8790f0265c7828046f3158e

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on fuel-library (master)

Change abandoned by Alex Schultz (<email address hidden>) on branch: master
Review: https://review.openstack.org/208111
Reason: This was done in https://review.openstack.org/#/c/206977/

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to fuel-library (master)

Reviewed: https://review.openstack.org/208070
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=368eb133ef7761c229ac4fc3a908dbd8bf71ae4b
Submitter: Jenkins
Branch: master

commit 368eb133ef7761c229ac4fc3a908dbd8bf71ae4b
Author: Alex Schultz <email address hidden>
Date: Fri Jul 31 17:19:06 2015 -0500

    Fix ceilometer using public address with ssl

    The ceilometer endpoing should use the ssl hostname instead of the
    address. Additionally we are replacing the deprecated address/protocol
    options with urls.

    Change-Id: Ibdef7cd2be7d73aa650f16ffd8ad4039dbbb318b
    Related-Bug: 1478649
    Related-Bug: 1476850

Revision history for this message
Vladimir Kuklin (vkuklin) wrote :

I rebased the change as it had jenkins issue.

Changed in fuel:
status: Fix Committed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (master)

Reviewed: https://review.openstack.org/206936
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=6916c08c1ca6096a5c51a896f898415ff88d843d
Submitter: Jenkins
Branch: master

commit 6916c08c1ca6096a5c51a896f898415ff88d843d
Author: Maksym Yatsenko <email address hidden>
Date: Wed Jul 29 10:53:45 2015 +0300

     Fix using of public address with SSL for Swift

    * Swift public endpoint should use public ssl hostname instead ip
    address
    * Replace {public|admin|internal}_address, public_protocol with {public|admin}_url
    * Update noop tests

    Closes-Bug: #1478649

    Change-Id: I09200a93b69d8f0f7565ef23cb3928d43df1c102

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
Alexander Kurenyshev (akurenyshev) wrote :

Verified on the #265 ISO.

Steps used to verify:

1) Install master node
2) Create cluster with 3 controllers, 2 computes and 1 cinder. Enables SSL for horizon
3) Deploy cluster
4) Run OSTF

Result:
All tests, include:
- Create volume and boot instance from it
- Create volume and attach it to instance

are passed.

Changed in fuel:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.