command image-update fails when data is not required, stdin is not a tty and stdin is open

[Hugh Saunders]

If stdin is not a tty and stdin is open, glance client will assume that stdin contains data that must be passed to glance. This is not always the case.

Bug #1166263 fixes the case where stdin is not a tty and is closed, however in some scenarios (exerstack testing for example) glance client will run in a situation where stdin is not a tty and is open, but no data should be read.

Steps to reproduce:
1) provide glance with non tty stdin when no data is required, for example renaming an image:
:| glance image-update testing --name testing

Example session:
---------------
root@devstack:~/devstack# glance show bd0593f4-1018-4328-af1e-546784d82cf2 |egrep '^(Id|Name)'
Id: bd0593f4-1018-4328-af1e-546784d82cf2
Name: testing
root@devstack:~/devstack# glance image-update testing --name testing >/dev/null; echo $?
0
root@devstack:~/devstack# :| glance image-update testing --name testing
Request returned failure status.
409 Conflict
There was a conflict when trying to complete your request.
 Cannot upload to an unqueued image (HTTP 409)
---------------

[Hugh Saunders (hughsaunders)]

I suggest two modifications
 * check the image status in v1/shell.py/do_image_update. Only attempt to read data is status is queued.
 * check for available data size in v1/shell.py/_set_data_field, dont read if 0.

Will submit a review after testing.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/27536

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/27536
Committed: http://github.com/openstack/python-glanceclient/commit/a3585ef62dbaa2c67f5b400f41e02cc686be5e7c
Submitter: Jenkins
Branch: master

commit a3585ef62dbaa2c67f5b400f41e02cc686be5e7c
Author: Hugh Saunders <email address hidden>
Date: Fri Apr 26 08:26:11 2013 +0100

    Don't attempt to read stdin if it is empty.

     * Check for available data size in v1/shell.py/_set_data_field, don't
       read if 0.
     * Add test_shell.py including tests for 3x stdin scenarios:
     * closed
      * open and empty
     * open with data

    Change-Id: I6ff65b0e226be509de9cd3f021560081529283b0
    Fixes: bug #1173044

[Simon Pasquier (simon-pasquier)]

The fix apparently introduces another bug, see https://bugs.launchpad.net/python-glanceclient/+bug/1184566

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/31529

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/31529
Committed: http://github.com/openstack/python-glanceclient/commit/9fda0dc815f042272c7c2b8ae18424cf1b6505e8
Submitter: Jenkins
Branch: master

commit 9fda0dc815f042272c7c2b8ae18424cf1b6505e8
Author: Hugh Saunders <email address hidden>
Date: Mon Jun 3 15:24:51 2013 +0100

    Fix problem where image data is not read from a pipe.

    For image-updae and image-create commands, glanceclient attempts to
    determine whether image data should be uploaded based on the presence
    of data on stdin. Unforunately it is difficult to determine if data is
    available, especially when standard in is from a pipe.

    This is especially problematic for update operations, where data must
    only be uploaded if the image is in queued state. For example data may
    be uploaded when the user only wants to rename an image, but the rename
    will be rejected because data cannot be uploaded to an unqueued image.

    This patch removes the check that attempts to determine if data is
    available to read as it didn't work for pipes. It also re-introduces a
    check for image state in the update operation, so that glanceclient only
    attempts to read data if the image being updated is in queued state.

    The image state check is part of the original patchset that was removed
    so the patchset could have a single focus [1]

    This patch also removes a test for handling empty stdin, and adds a test
    for reading stdin from a pipe.

    [1] https://review.openstack.org/#/c/27536/3/glanceclient/v1/shell.py

    Fixes: bug 1184566
    Related to: bug 1173044

    Change-Id: I8d37f6412a0bf9ca21cbd75cde6a4d5a174e5545

[Hugh Saunders (hughsaunders)]

9fda0dc815f042272c7c2b8ae18424cf1b6505e8 as referenced above fixes this for all cases except an image in queued state. This bug is still an issue for images in queued state.

However it would require a redesign of the glance cli create/update commands to fix this properly. This change is to require a --data flag to specify when data should be read. --data - for stdin, or --data <path> to read from a file. Unfortunately this would break lots of existing tools so isn't possible in the short term. This bug should probably be closed wont_fix, but I don't have permission to set that.

[Jaroslav Henner (jhenner)]

There is probably no workaround when there is no tty available (on Jenkins). It would be good to fix. What about adding some argument like --no-data that would made the behaviour correct when updating only the metadata?

[Jaroslav Henner (jhenner)]

Fortunately, there is a workaround -- closing the stdin using 0<&-

0<&- glance image-update --is-public True ${SNAPSHOT_ID}

[Jaroslav Henner (jhenner)]

Unfortunately there is no workaround. I must have done some error when investigating that. It seems python opens the stdin if it was closed:

 <&- python -c 'import sys; print sys.stdin'
 <open file '<stdin>', mode 'r' at 0x7fa8ae1eb150>

[Jaroslav Henner (jhenner)]

It helped to add

+ if image.status == 'queued':
        _set_data_field(fields, args)

in def do_image_update(gc, args):
in v1/shell.py of python-glanceclient-0.9.0-3.fc19.noarch

[Andrey Pavlov (apavlov-e)]

Is there any new information about this bug?
I am trying to use this feature in my autotest scripts but it doesn't work.
It works well locally but doesn't work in gating -

2015-04-20 16:03:43.690 | + glance image-update image-8484825d --container-format ami --disk-format ami
2015-04-20 16:03:45.185 | Unable to upload image data to an image which is active.

I investigated that this message now is produced by code of this change -
https://review.openstack.org/#/c/136520/7

[Denis Egorenko (degorenko)]

I have the same issue for gating, during update image name:

# glance image-update old-name --name new-name
Unable to upload image data to an image which is active.

[Andrey Pavlov (apavlov-e)]

@Denis, I've found that it can be done via openstack cli tool.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/216692

[OpenStack Infra (hudson-openstack)]

Change abandoned by Darja Shakhray (<email address hidden>) on branch: master
Review: https://review.openstack.org/216692

[Ian Cordasco (icordasc)]

This seems to have been mostly fixed and the last case seems to add far too much complexity in order to fix it. I'm marking this as won't fix until it can be shown to be a severe bug.