Session timeout on huge image uploads

[Sudheer Kalla]

When user tries to upload an image(via horizon) which is huge in size while uploading the image if the session expires the process will be stopped in between and image upload fails.

So when ever the user tries to upload a huge image or file then there should be a mechansim which will make sure the session time out did not occurs and make user will not losse any data

[Matt Borland (palecrow)]

This is being worked on in a blueprint: https://blueprints.launchpad.net/openstack/?searchtext=horizon-glance-large-image-upload

[Timur Sufiev (tsufiev-x)]

Once Django patch https://review.openstack.org/#/c/230434/ is merged (or its Angular counterpart https://review.openstack.org/#/c/317456/ ), Horizon will have nothing to do with session expiry while uploading large images via CORS - the issue will have to be fixed on Glance side.

[Matt Borland (palecrow)]

Timur, I agree that we should use those patches as the way forward.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/317456
Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=0e1279d05c3b2de7127f9e4ee752d5771514cc74
Submitter: Jenkins
Branch: master

commit 0e1279d05c3b2de7127f9e4ee752d5771514cc74
Author: Timur Sufiev <email address hidden>
Date: Tue May 24 14:54:54 2016 +0300

    [NG] Support local file upload in Create Image workflow

    First, now there are 2 '/api/glance/images/ API wrapper endpoints for
    creating a new image - POST and PUT. The POST endpoint which existed
    before now resides at PUT. This was done to support legacy
    (i.e. proxied by web-server) file uploads in Angular Create Image,
    because Django (which we need to use in that case) doesn't correctly
    process PUT request. So, if local file binary payload is added to the
    form contents, we send it using POST request.

    Second, speaking of '/api/glance/images' PUT request previously known
    as POST... Where before the POST call to Horizon REST API wrappers
    returned the final image object that Glance just created, now there
    are two possibilities for what happens after PUT is sent.

    * When Create Image form Source Type is set to URL, then everything is
      going as before.
    * When Source Type is set to File, then just a file name instead of an
      actual Blob is sent to '/api/glance/images', then the Glance Image
      is queued for creation and Horizon web-server responds with an Image
      object which dict() representation has 2 additional keys:
      'upload_url' and 'token_id'. The 'upload_url' tells the location for
      the subsequent CORS request, while 'token_id' is passed as a header
      in that request, so Keystone would let it in. CORS upload is started
      immediately as Image is queued for creation (first promise is
      resolved) and returns the second promise, which is resolved once the
      upload finishes. The modal form hangs until second promise resolves
      to indicate that upload is in progress. Upload progress notification
      is added in a follow-up patch.

    DEPLOY NOTES

    The client-side code relies on CORS being enabled for Glance service
    (otherwise browser would forbid the PUT request to a location
    different from the one form content came from). In a Devstack setup
    you'll need to edit [cors] section of glance-api.conf file, setting
    `allowed_origin` setting to the full hostname of the web server (say,
    http://<HOST_IP>/dashboard).

    Related-Bug: #1467890
    Implements blueprint: horizon-glance-large-image-upload
    Change-Id: I5d842d614c16d3250380ea1dc1c6e0289d206fb5

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/230434
Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=93af461e40fca76eae58066cc1f9de83d7b716ea
Submitter: Jenkins
Branch: master

commit 93af461e40fca76eae58066cc1f9de83d7b716ea
Author: Timur Sufiev <email address hidden>
Date: Tue May 17 13:16:15 2016 +0300

    [Django] Allow to upload the image directly to Glance service

    Since large Glance images even temporarily stored on dashboard side
    tend to fill up Web Server filesystem, it is desirable to route image
    payload directly to Glance service (which usually streams it to
    storage backend, which in turn has plenty of space).

    To make it possible we need to trick Django into thinking that a file
    was selected inside FileInput, while its contents are not actually
    transferred to Django server. Then, once image is created client-side
    code needs to know the exact url the image payload needs to be
    transferred to. Both tasks are solved via using ExternalFileField /
    ExternalUploadMeta classes which allow to work around the usual Django
    form processing workflow with minimal changes to CreateImage form
    business logic.

    The client-side code relies on CORS being enabled for Glance service
    (otherwise browser would forbid the PUT request to a location
    different from the one form content came from). In a Devstack setup
    you'll need to edit [cors] section of glance-api.conf file, setting
    `allowed_origin` setting to the full hostname of the web server (say,
    http://<HOST_IP>/dashboard) and restart glance-api process.

    A progress bar is implemented to track the progress of a file upload,
    in case a really huge image is transferred.

    The new machinery could be easily switched on/off with a single
    setting `HORIZON_IMAGES_UPLOAD_MODE` set to 'direct' / 'legacy'.

    Related-Bug: #1467890
    Closes-Bug: #1403129
    Implements blueprint: horizon-glance-large-image-upload
    Change-Id: I01d02f75268186b43066df6fd966aa01c08e01d7

[Rob Cresswell (robcresswell-deactivatedaccount)]

As of Newton with the new Images work, this should no longer be an issue