Incorrect value for swift_store_large_object_size in default Glance configuration file

Bug #1373813 reported by Timur Nurlygayanov
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
High
Timur Nurlygayanov
5.1.x
Fix Released
High
Timur Nurlygayanov
6.0.x
Fix Released
High
Timur Nurlygayanov

Bug Description

Steps To Reproduce:
1. Deploy OpenStack cloud with the default parameters (1 controller, 1 compute node, without Ceph, only Glance+Swift).
2. Upload image with Ubuntu 14.04 to Glance.
3. Create VM with this image.
4. Install packages, like (to increase the size of hard disk):
build-essential cvs subversion git-core mercurial python-setuptools python-dev python2.7-dev python-software-properties
libpq-dev libtiff4-dev libjpeg8-dev zlib1g-dev libfreetype6-dev liblcms2-dev libwebp-dev tcl8.5-dev tk8.5-dev python-pip
5. Try to create snapshot of this VM.

Expected Result:
User can successfully create snapshots.

Observed Result:
User can't create large (>300 Mb) snapshots, but can create small snapshots, for example, if we will use TestVM image for Virtual Machine.

In Glance log files we can see the following error:
________________
<147>Sep 23 17:23:24 node-2 glance-glance.store.swift ERROR: Failed to add object to Swift.
Got error from Swift: put_object('glance', 'ec4d05e0-1626-48c2-a838-15d3d02affd1', ...) failure and no ability to reset contents for reupload.
<147>Sep 23 17:23:24 node-2 glance-glance.api.v1.upload_utils ERROR: Failed to upload image ec4d05e0-1626-48c2-a838-15d3d02affd1
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/glance/api/v1/upload_utils.py", line 99, in upload_data_to_store
store)
File "/usr/lib/python2.6/site-packages/glance/store/__init__.py", line 380, in store_add_to_backend
(location, size, checksum, metadata) = store.add(image_id, data, size)
File "/usr/lib/python2.6/site-packages/glance/store/swift.py", line 503, in add
raise glance.store.BackendException(msg)
BackendException: Failed to add object to Swift.
Got error from Swift: put_object('glance', 'ec4d05e0-1626-48c2-a838-15d3d02affd1', ...) failure and no ability to reset contents for reupload.
________________

How To Fix:
We found the workaround for this problem, need to set parameter 'swift_store_large_object_size=200' in /etc/glance/glance-api.conf file. In this case glance will slice all large snapshots to 200Mb files and Swift will successfully store it.

tags: added: glance
Revision history for this message
Timur Nurlygayanov (tnurlygayanov) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (master)

Fix proposed to branch: master
Review: https://review.openstack.org/123992

Changed in fuel:
assignee: nobody → Timur Nurlygayanov (tnurlygayanov)
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (stable/5.1)

Fix proposed to branch: stable/5.1
Review: https://review.openstack.org/124024

Changed in fuel:
assignee: Timur Nurlygayanov (tnurlygayanov) → Aleksandr Didenko (adidenko)
Changed in fuel:
assignee: Aleksandr Didenko (adidenko) → Timur Nurlygayanov (tnurlygayanov)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (master)

Reviewed: https://review.openstack.org/123992
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=5efc17969ab66b54f5daf9472d8cad30fce9c9be
Submitter: Jenkins
Branch: master

commit 5efc17969ab66b54f5daf9472d8cad30fce9c9be
Author: TimurNurlygayanov <email address hidden>
Date: Thu Sep 25 13:44:12 2014 +0400

    Fixed issue with max size for Swift objects

    Fixed issue with max size of object, which Glance
    should store in Swift without slicing.
    Default value doesn't work when user attempts to make
    a snapshots with size > 300 Mb.

    Change-Id: Ic3b3120605f1e1536a34bedc4ad5c0c2f4c8510b
    Closes-Bug: #1373813

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (stable/5.1)

Reviewed: https://review.openstack.org/124024
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=6b7a115afacd97227a3e808962863b179a85a27f
Submitter: Jenkins
Branch: stable/5.1

commit 6b7a115afacd97227a3e808962863b179a85a27f
Author: TimurNurlygayanov <email address hidden>
Date: Thu Sep 25 15:42:54 2014 +0400

    Fixed issue with max size for Swift objects

    Fixed issue with max size of object, which Glance
    should store in Swift without slicing.
    Default value doesn't work when user attempts to make
    a snapshots with size > 300 Mb.

    Change-Id: Ic3b3120605f1e1536a34bedc4ad5c0c2f4c8510b
    Closes-Bug: #1373813

tags: added: release-notes
Revision history for this message
Mykhaylo Slobodyan (mykhaylo-slobodyan) wrote :

Verified on 5.1.1 ISO #37, with 1.3gb snapshot.

<150>Nov 26 18:03:17 node-1 glance-registry 2014-11-26 18:03:17.593 22058 INFO glance.registry.api.v1.images [2e074cac-4163-480a-93fa-8714cee994bd c79706805fd24105aebda7d48a73e7bb 072b37d7b1e546e18768a174247428ce - - -] Suc
cessfully created image f136d58c-f7bf-4a1d-9595-ed29396ad993
<151>Nov 26 18:04:35 node-1 glance-registry 2014-11-26 18:04:35.845 22058 DEBUG glance.registry.api.v1.images [16b92ed6-fab8-4b4b-9f6f-593cabe2cd63 c79706805fd24105aebda7d48a73e7bb 072b37d7b1e546e18768a174247428ce - - -] Updating image f136d58c-f7bf-4a1d-9595-ed29396ad993 with metadata: {u'name': u'test367', u'container_format': u'bare', u'disk_format': u'qcow2', u'is_public': False, u'properties': {u'kernel_id': None, u'image_location': u'snapshot', u'image_state': u'available', u'ramdisk_id': None, u'owner_id': u'072b37d7b1e546e18768a174247428ce'}, u'size': 1434058752} update /usr/lib/python2.7/dist-packages/glance/registry/api/v1/images.py:445

VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "5.1.1"
  api: "1.0"
  build_number: "37"
  build_id: "2014-11-24_21-01-00"
  astute_sha: "dade74af41d4972fe05a1c16ae1db2a2e60c6715"
  fuellib_sha: "444339cae21c369c1d95e96c1059d4099077138e"
  ostf_sha: "64cb59c681658a7a55cc2c09d079072a41beb346"
  nailgun_sha: "24b956d739e8e9d8f728701522c6fa8364526c45"
  fuelmain_sha: "3797f2f2eff42812715840293a618de73fdef26f"

Revision history for this message
Andrey Sledzinskiy (asledzinskiy) wrote :

verified on {

    "build_id": "2014-12-26_14-25-46",
    "ostf_sha": "a9afb68710d809570460c29d6c3293219d3624d4",
    "build_number": "58",
    "auth_required": true,
    "api": "1.0",
    "nailgun_sha": "5f91157daa6798ff522ca9f6d34e7e135f150a90",
    "production": "docker",
    "fuelmain_sha": "81d38d6f2903b5a8b4bee79ca45a54b76c1361b8",
    "astute_sha": "16b252d93be6aaa73030b8100cf8c5ca6a970a91",
    "feature_groups": [
        "mirantis"
    ],
    "release": "6.0",
    "release_versions": {
        "2014.2-6.0": {
            "VERSION": {
                "build_id": "2014-12-26_14-25-46",
                "ostf_sha": "a9afb68710d809570460c29d6c3293219d3624d4",
                "build_number": "58",
                "api": "1.0",
                "nailgun_sha": "5f91157daa6798ff522ca9f6d34e7e135f150a90",
                "production": "docker",
                "fuelmain_sha": "81d38d6f2903b5a8b4bee79ca45a54b76c1361b8",
                "astute_sha": "16b252d93be6aaa73030b8100cf8c5ca6a970a91",
                "feature_groups": [
                    "mirantis"
                ],
                "release": "6.0",
                "fuellib_sha": "fde8ba5e11a1acaf819d402c645c731af450aff0"
            }
        }
    },
    "fuellib_sha": "fde8ba5e11a1acaf819d402c645c731af450aff0"

}

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.