g-api raise 500 error if filesystem_store_datadirs and filesystem_store_datadir both specified

[Abhishek Kekane]

g-api raise 500 error if filesystem_store_datadirs and filesystem_store_datadir both specified

If both filesystem_store_datadirs and filesystem_store_datadir parameters specified in glance-api.conf file then while creating new image 500 internal server error will be raised. Ideally it should raise 'BadStoreConfiguration' exception and glance-api service should not be started.

Stack trace on the console:

2014-09-23 03:15:24.407 7594 ERROR glance.api.v1.upload_utils [f351b844-8b1e-429c-8943-a79b331311be 41d56c1d7e134fdb8a1dcfe4ea3c82de 73078aaf41fb47b5bb0cfd4e9fdc79fb - - -] Failed to upload image 288f7386-bddf-4b2b-a97f-49d2814f7b99
2014-09-23 03:15:24.407 7594 TRACE glance.api.v1.upload_utils Traceback (most recent call last):
2014-09-23 03:15:24.407 7594 TRACE glance.api.v1.upload_utils File "/opt/stack/glance/glance/api/v1/upload_utils.py", line 106, in upload_data_to_store
2014-09-23 03:15:24.407 7594 TRACE glance.api.v1.upload_utils store)
2014-09-23 03:15:24.407 7594 TRACE glance.api.v1.upload_utils File "/usr/local/lib/python2.7/dist-packages/glance_store/backend.py", line 342, in store_add_to_backend
2014-09-23 03:15:24.407 7594 TRACE glance.api.v1.upload_utils (location, size, checksum, metadata) = store.add(image_id, data, size)
2014-09-23 03:15:24.407 7594 TRACE glance.api.v1.upload_utils File "/usr/local/lib/python2.7/dist-packages/glance_store/driver.py", line 149, in add_disabled
2014-09-23 03:15:24.407 7594 TRACE glance.api.v1.upload_utils raise exceptions.StoreAddDisabled
2014-09-23 03:15:24.407 7594 TRACE glance.api.v1.upload_utils StoreAddDisabled: None

Steps to reproduce:
1. edit glance-api.conf file and specify values for both filesystem_store_datadirs and filesystem_store_datadir options
2. Restart the glance-api service
3. Hit create-image api/upload image using horizon

[Stuart McLaren (stuart-mclaren)]

Thanks Abhishek.

filesystem_store_datadirs and filesystem_store_datadir are mutually exclusive, but the code isn't handling that properly.

[Abhishek Kekane (abhishek-kekane)]

This error is raised because at the time of service starts when filesystem store is initialized, it calls configure_add method and raises 'BadStoreConfiguration' if filesystem_store_datadirs and filesystem_store_datadir are specified. This exception is caught in glance_store/driver.py [1], where this exception is suppressed in the warning and add method is disabled for that particular method which is raising 'StoreAddDisabled' exception without any valid message which is not good in end user's point of view.

So when user tries to upload the image the instead of add, add_disabled method gets called and 500 error is returned to the user.

Possible Solutions:

1. Introduce new exception and restrict glance-api service from starting (as its working previously when this filesystem_store_datadirs option is introduced, ref: https://review.openstack.org/#/c/58997).

2. Pass meaningful message to 'StoreAddDisabled' exception

Please suggest.

[1] https://github.com/openstack/glance_store/blob/master/glance_store/driver.py#L66

[Tushar Patil (tpatil)]

IMO, it should return 410 error instead of 500.
What do others think?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/124302

[Zhi Yan Liu (lzy-dev)]

Abhishek, after the checking on configure_add() of filesystem driver, I consider current driver initialization logic is correct, so I prefer we fix this issue with above #2 approach. I just reviewed the its implementation at https://review.openstack.org/#/c/124302/ , and few minor suggestions added.

Tushar, +1 for 410 (HTTP Gone).

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/124302
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=a0f96d437e810d799161452e7164ecebc38909a0
Submitter: Jenkins
Branch: master

commit a0f96d437e810d799161452e7164ecebc38909a0
Author: abhishekkekane <email address hidden>
Date: Wed Sep 24 04:36:13 2014 -0700

    g-api raises 500 error while uploading image

    If both filesystem_store_datadirs and filesystem_store_datadir parameters
    specified in glance-api.conf file then while creating new image 500
    internal server error is raised.

    Caught StoreAddDisabled exception for v1 and v2 api while uploading
    image and raised HTTPGone exception to return HTTP 410 response to
    user.

    Closes-Bug: #1372888
    Change-Id: Iccbf5b88634d3c956d41e8d8a0126648c64b34eb

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: proposed/juno
Review: https://review.openstack.org/127247

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/127247
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=ace6197bbaaba7d53c05ff6eb2c39a642cfc463d
Submitter: Jenkins
Branch: proposed/juno

commit ace6197bbaaba7d53c05ff6eb2c39a642cfc463d
Author: abhishekkekane <email address hidden>
Date: Wed Sep 24 04:36:13 2014 -0700

    g-api raises 500 error while uploading image

    If both filesystem_store_datadirs and filesystem_store_datadir parameters
    specified in glance-api.conf file then while creating new image 500
    internal server error is raised.

    Caught StoreAddDisabled exception for v1 and v2 api while uploading
    image and raised HTTPGone exception to return HTTP 410 response to
    user.

    Closes-Bug: #1372888
    Change-Id: Iccbf5b88634d3c956d41e8d8a0126648c64b34eb

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/128928

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/128928
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=858cd9d4fdf32b4cc52e02ec771d01bfb463aa77
Submitter: Jenkins
Branch: master

commit 96e28428655aa7122ed74b045ff1bda1984255b1
Author: Nikhil Komawar <email address hidden>
Date: Tue Oct 14 13:09:48 2014 -0400

    Fix options and their groups - etc/glance-api.conf

    As per the docs at [0] , some of the options should have been moved
    around in the etc/glance-api.conf. This patch changes the conf file to:

        1. indicate new default values
        2. change the group of some of the configs in order to adhere to
           new groups as expected by the deployer.
        3. deprecated configs have been removed or replaced with new ones.

    [0] http://docs.openstack.org/trunk/config-reference/content/glance-conf-changes-master.html

    Fixes bug: 1380689

    Change-Id: I5b5ab96b050b502007e6660a7a613e252404d4e8

commit 9b176a278116849c8f7b7f4d9a987f37ec52779c
Author: Andy McCrae <email address hidden>
Date: Sat Oct 11 20:56:36 2014 +0100

    Adjust authentication.rst doc to reference "identity_uri"

    The "auth_port", "auth_host", and "auth_protocol" variables were
    deprecated in favour of a single "identity_uri" variable.

    * Adjust authentication.rst doc to reference "identity_uri"

    Change-Id: I48de53f21b8d767b276858ed274066015d765f0e
    Closes-Bug: #1361613

commit 08f83f543bc992ae8f2787fb405e58c33dadba73
Author: Jamie Lennox <email address hidden>
Date: Wed Aug 6 18:24:05 2014 +1000

    Use identity_uri instead of older fragments

    This has been the default option in middleware for a while now and we
    should recommend the default options.

    Change-Id: Ief347c897cf15ab4101936a56404e3a378021b15
    (cherry picked from commit e7110a9c6e0119b3d0c6f5cdb3b9675a82b76039)

commit c0d90a580f87dbbf71e3a5d5c1b5cf8d7c7245b2
Author: Stuart McLaren <email address hidden>
Date: Wed Jul 16 13:33:32 2014 +0000

    Prevent setting swift+config locations

    Forbid setting 'swift+config' locations in a similar
    manner to 'file' for security reasons; knowledge of
    the reference name should not be exploitable.

    Setting swift+config had been prevented when swift
    was the default store, this patch changes to forbid
    setting no matter which store is the default.

    As with change id I75af34145521f533dcd6f5fd7690f5a68f3b44b3
    this is v1 only for now.

    Change-Id: I62c4980bd5c2f3dd77fc40cd007bc1067eca63a4
    Closes-bug: 1334196

commit f259cac74d3e988b4012dcc2abd30091df27f5ce
Author: Wayne Okuma <email address hidden>
Date: Wed Oct 8 08:17:20 2014 -0700

    Metadef schema column name is a reserved word in MySQL

    The metadef_properties and metadef_objects tables both have
    a column named schema. Unfortunately, schema is a reserved word
    in some relational database products, including MySQL and PostgreSQL.
    The metadef_properties.schema and metadef_objects.schema
    columns should be renamed to a non reserved word.

    Conflicts:
     glance/db/sqlalchemy/metadata.py
     glance/tests/unit/test_m...