image-create command fails when reading from stdin

Bug #1184566 reported by Simon Pasquier on 2013-05-27
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
python-glanceclient
Undecided
Hugh Saunders

Bug Description

The following command creates an empty image in queued state:
$ glance image-create --name cirros-0.3.1-x86_64-rootfs --public --container-format=bare --disk-format raw < <(zcat /opt/devstack/files/cirros-0.3.1-x86_64-rootfs.img.gz)
+------------------+--------------------------------------+
| Property | Value |
+------------------+--------------------------------------+
| checksum | None |
| container_format | bare |
| created_at | 2013-05-27T10:38:57 |
| deleted | False |
| deleted_at | None |
| disk_format | raw |
| id | 98bdf205-3f95-4826-82bf-438729e209d9 |
| is_public | True |
| min_disk | 0 |
| min_ram | 0 |
| name | cirros-0.3.1-x86_64-rootfs |
| owner | 08ca2c466e3d41c5ab9c2e0c9a2f3b82 |
| protected | False |
| size | 0 |
| status | queued |
| updated_at | 2013-05-27T10:38:57 |
+------------------+--------------------------------------+

The same problem exists using standard pipes:
zcat /opt/devstack/files/cirros-0.3.1-x86_64-rootfs.img.gz | glance image-create --name cirros-0.3.1-x86_64-rootfs --public --container-format=bare --disk-format raw
+------------------+--------------------------------------+
| Property | Value |
+------------------+--------------------------------------+
| checksum | None |
| container_format | bare |
| created_at | 2013-05-27T11:48:07 |
| deleted | False |
| deleted_at | None |
| disk_format | raw |
| id | 3436ac82-3158-47dc-a304-b75c6e10c547 |
| is_public | True |
| min_disk | 0 |
| min_ram | 0 |
| name | cirros-0.3.1-x86_64-rootfs |
| owner | 7087a00934c745e58c2e26460451ecf5 |
| protected | False |
| size | 0 |
| status | queued |
| updated_at | 2013-05-27T11:48:07 |
+------------------+--------------------------------------+

This used to work in Folsom & Grizzly releases. The problem was introduced when fixing bug #1173044 (https://bugs.launchpad.net/python-glanceclient/+bug/1173044).

As a side note, this prevents using devstack with the LXC driver.

summary: - image-create command fails when using Bash process substitution
+ image-create command fails when reading from stdin
description: updated
Hugh Saunders (hughsaunders) wrote :

I can reproduce this, and will investigate.

Changed in python-glanceclient:
assignee: nobody → Hugh Saunders (hughsaunders)
status: New → Confirmed
Hugh Saunders (hughsaunders) wrote :

St_size does not give a useful value for pipes:

root@devstack:/opt/stack/devstack/files# python -c 'import os; print os.fstat(0)' </etc/hostname
posix.stat_result(st_mode=33188, st_ino=99893, st_dev=51713L, st_nlink=1, st_uid=0, st_gid=0, st_size=9, st_atime=1370096199, st_mtime=1370096199, st_ctime=1370096199)

root@devstack:/opt/stack/devstack/files# python -c 'import os; print os.fstat(0)' <(cat /etc/hostname)
posix.stat_result(st_mode=8576, st_ino=26, st_dev=11L, st_nlink=1, st_uid=0, st_gid=5, st_size=0, st_atime=1370264462, st_mtime=1370264462, st_ctime=1370249304)

The test for this didn't pick this up as it read from a file rather than a pipe: https://github.com/openstack/python-glanceclient/blob/master/tests/v1/test_shell.py#L401-L426.

Will add another test involving reading from a pipe, and backout the st_size check.

Changed in python-glanceclient:
status: Confirmed → In Progress
Hugh Saunders (hughsaunders) wrote :

Hi,
I have submitted a patch, but will now be AFK for a week. Please feel free to take this bug, improve/change the patch so that it can be merged and this bug fixed.

Changed in python-glanceclient:
status: In Progress → Confirmed
assignee: Hugh Saunders (hughsaunders) → nobody
Changed in python-glanceclient:
assignee: nobody → Hugh Saunders (hughsaunders)

Reviewed: https://review.openstack.org/31529
Committed: http://github.com/openstack/python-glanceclient/commit/9fda0dc815f042272c7c2b8ae18424cf1b6505e8
Submitter: Jenkins
Branch: master

commit 9fda0dc815f042272c7c2b8ae18424cf1b6505e8
Author: Hugh Saunders <email address hidden>
Date: Mon Jun 3 15:24:51 2013 +0100

    Fix problem where image data is not read from a pipe.

    For image-updae and image-create commands, glanceclient attempts to
    determine whether image data should be uploaded based on the presence
    of data on stdin. Unforunately it is difficult to determine if data is
    available, especially when standard in is from a pipe.

    This is especially problematic for update operations, where data must
    only be uploaded if the image is in queued state. For example data may
    be uploaded when the user only wants to rename an image, but the rename
    will be rejected because data cannot be uploaded to an unqueued image.

    This patch removes the check that attempts to determine if data is
    available to read as it didn't work for pipes. It also re-introduces a
    check for image state in the update operation, so that glanceclient only
    attempts to read data if the image being updated is in queued state.

    The image state check is part of the original patchset that was removed
    so the patchset could have a single focus [1]

    This patch also removes a test for handling empty stdin, and adds a test
    for reading stdin from a pipe.

    [1] https://review.openstack.org/#/c/27536/3/glanceclient/v1/shell.py

    Fixes: bug 1184566
    Related to: bug 1173044

    Change-Id: I8d37f6412a0bf9ca21cbd75cde6a4d5a174e5545

Changed in python-glanceclient:
status: Confirmed → Fix Committed
Louis Taylor (kragniz) on 2014-09-21
Changed in python-glanceclient:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers