image-create command fails when reading from stdin

[Simon Pasquier]

The following command creates an empty image in queued state:
$ glance image-create --name cirros-0.3.1-x86_64-rootfs --public --container-format=bare --disk-format raw < <(zcat /opt/devstack/files/cirros-0.3.1-x86_64-rootfs.img.gz)
+------------------+--------------------------------------+
| Property | Value |
+------------------+--------------------------------------+
| checksum | None |
| container_format | bare |
| created_at | 2013-05-27T10:38:57 |
| deleted | False |
| deleted_at | None |
| disk_format | raw |
| id | 98bdf205-3f95-4826-82bf-438729e209d9 |
| is_public | True |
| min_disk | 0 |
| min_ram | 0 |
| name | cirros-0.3.1-x86_64-rootfs |
| owner | 08ca2c466e3d41c5ab9c2e0c9a2f3b82 |
| protected | False |
| size | 0 |
| status | queued |
| updated_at | 2013-05-27T10:38:57 |
+------------------+--------------------------------------+

The same problem exists using standard pipes:
zcat /opt/devstack/files/cirros-0.3.1-x86_64-rootfs.img.gz | glance image-create --name cirros-0.3.1-x86_64-rootfs --public --container-format=bare --disk-format raw
+------------------+--------------------------------------+
| Property | Value |
+------------------+--------------------------------------+
| checksum | None |
| container_format | bare |
| created_at | 2013-05-27T11:48:07 |
| deleted | False |
| deleted_at | None |
| disk_format | raw |
| id | 3436ac82-3158-47dc-a304-b75c6e10c547 |
| is_public | True |
| min_disk | 0 |
| min_ram | 0 |
| name | cirros-0.3.1-x86_64-rootfs |
| owner | 7087a00934c745e58c2e26460451ecf5 |
| protected | False |
| size | 0 |
| status | queued |
| updated_at | 2013-05-27T11:48:07 |
+------------------+--------------------------------------+

This used to work in Folsom & Grizzly releases. The problem was introduced when fixing bug #1173044 (https://bugs.launchpad.net/python-glanceclient/+bug/1173044).

As a side note, this prevents using devstack with the LXC driver.

[Hugh Saunders (hughsaunders)]

I can reproduce this, and will investigate.

[Hugh Saunders (hughsaunders)]

St_size does not give a useful value for pipes:

root@devstack:/opt/stack/devstack/files# python -c 'import os; print os.fstat(0)' </etc/hostname
posix.stat_result(st_mode=33188, st_ino=99893, st_dev=51713L, st_nlink=1, st_uid=0, st_gid=0, st_size=9, st_atime=1370096199, st_mtime=1370096199, st_ctime=1370096199)

root@devstack:/opt/stack/devstack/files# python -c 'import os; print os.fstat(0)' <(cat /etc/hostname)
posix.stat_result(st_mode=8576, st_ino=26, st_dev=11L, st_nlink=1, st_uid=0, st_gid=5, st_size=0, st_atime=1370264462, st_mtime=1370264462, st_ctime=1370249304)

The test for this didn't pick this up as it read from a file rather than a pipe: https://github.com/openstack/python-glanceclient/blob/master/tests/v1/test_shell.py#L401-L426.

Will add another test involving reading from a pipe, and backout the st_size check.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/31529

[Hugh Saunders (hughsaunders)]

Hi,
I have submitted a patch, but will now be AFK for a week. Please feel free to take this bug, improve/change the patch so that it can be merged and this bug fixed.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/31529
Committed: http://github.com/openstack/python-glanceclient/commit/9fda0dc815f042272c7c2b8ae18424cf1b6505e8
Submitter: Jenkins
Branch: master

commit 9fda0dc815f042272c7c2b8ae18424cf1b6505e8
Author: Hugh Saunders <email address hidden>
Date: Mon Jun 3 15:24:51 2013 +0100

    Fix problem where image data is not read from a pipe.

    For image-updae and image-create commands, glanceclient attempts to
    determine whether image data should be uploaded based on the presence
    of data on stdin. Unforunately it is difficult to determine if data is
    available, especially when standard in is from a pipe.

    This is especially problematic for update operations, where data must
    only be uploaded if the image is in queued state. For example data may
    be uploaded when the user only wants to rename an image, but the rename
    will be rejected because data cannot be uploaded to an unqueued image.

    This patch removes the check that attempts to determine if data is
    available to read as it didn't work for pipes. It also re-introduces a
    check for image state in the update operation, so that glanceclient only
    attempts to read data if the image being updated is in queued state.

    The image state check is part of the original patchset that was removed
    so the patchset could have a single focus [1]

    This patch also removes a test for handling empty stdin, and adds a test
    for reading stdin from a pipe.

    [1] https://review.openstack.org/#/c/27536/3/glanceclient/v1/shell.py

    Fixes: bug 1184566
    Related to: bug 1173044

    Change-Id: I8d37f6412a0bf9ca21cbd75cde6a4d5a174e5545