images with bigger size failed to push through openstack dashboard

[ashutosh gautam]

Hi,

We faced an issue while pushing the image through opensatck dashboard UI.

Problem:
we tried to to upload a 5.5G image through Horizon webui. That fails, but the CLI works.
We ran some tests and found that /tmp is being used to hold the temporary file.
```
8.5M /tmp/systemd-private-bf0f1d5622ae4b1bba0d7c34c7530e4f-apache2.service-sAg1Hg/tmp/tmpr5z6fm55.upload.img
```

There is no option for direct glance transfer right now.

It seems that the default and only available mode for horizon I can see right now is to use tmpfs (/tmp) to store the image on disk first and then push it to glance. Its current size is only 2G
```
root@juju-e9cbb4-5-lxd-19:~# df -h /tmp
Filesystem Size Used Avail Use% Mounted on
tmpfs 2.0G 139M 1.9G 7% /tmp
```

Eventhough it is true we have a workaround(Increase the size of fstab), we do need to make the functionality work in Horizon.

we would request in order to allow HORIZON_IMAGES_UPLOAD_MODE to be configurable through openstack-dashboard charm, so that we can direct upload images to glance without using /tmp as an intermediate location.

Thanks

[Alex Kavanagh (ajkavanagh)]

Essentially, I think this bug is about the option [1] `HORIZON_IMAGES_UPLOAD_MODE` which can be set to "legacy" or "direct".

Currently, the openstack-dashboard charm has this in it's config template:

    # Set to 'legacy' or 'direct' to allow users to upload images to glance via
    # Horizon server. When enabled, a file form field will appear on the create
    # image form. If set to 'off', there will be no file form field on the create
    # image form. See documentation for deployment considerations.
    #HORIZON_IMAGES_UPLOAD_MODE = 'legacy'

i.e. it defaults to legacy.

This means that it does use the horizon unit to upload the image to first, and then upload it to glance.

If the option is set to direct, then from [1]:

    HORIZON_IMAGES_UPLOAD_MODE
    New in version 10.0.0(Newton).

    Default: "legacy"

    Valid values are "direct", "legacy" (default) and "off". "off" disables the ability to upload images via Horizon. legacy enables local file upload by piping the image file through the Horizon’s web-server. direct sends the image file directly from the web browser to Glance. This bypasses Horizon web-server which both reduces network hops and prevents filling up Horizon web-server’s filesystem. direct is the preferred mode, but due to the following requirements it is not the default. The direct setting requires a modern web browser, network access from the browser to the public Glance endpoint, and CORS support to be enabled on the Glance API service. Without CORS support, the browser will forbid the PUT request to a location different than the Horizon server. To enable CORS support for Glance API service, you will need to edit [cors] section of glance-api.conf file (see here how to do it). Set allowed_origin to the full hostname of Horizon web-server (e.g. http://<HOST_IP>/dashboard) and restart glance-api process.

An issue here is setting CORS for the user's browser to be able to upload directly to the glance endpoint, if the endpoint domain name is different to the horizon dashboard domain name. Thus, this complicates it a little.

What I think needs to happen is that an option is added to the dashboard to enable "direct_glance_uploads" as a boolean, and for the [cors] section of the glance-api.conf file to be updated automatically depending on the endpoint for glance.

[1] https://docs.openstack.org/horizon/latest/configuration/settings.html#horizon-images-upload-mode

[Alex Kavanagh (ajkavanagh)]

Triaging to medium. A temporary workaround would be to manually increase the tmpfs size on the unit to a larger size (obviously will impact memory usage on the unit), or move tmpfs to a disk. The scope of this bug should be enabling the direct option to allow direct uploads to glance.

[Alex Kavanagh (ajkavanagh)]

Added in horizon-k8s operator as it has the same default setting of 'legacy' in its local_settings.py.j2 template.