Glance client no longer outputs to stdout

[Stuart McLaren]

For several years glance client output to stdout by default.

I would rely on uploading a small text file as a pseudo image and verify it existed with, for example:

$ glance image-download 96ed6a3f-4ec9-4e74-a1bd-30e33324be80

The behaviour seems to have changed:

$ glance image-download 96ed6a3f-4ec9-4e74-a1bd-30e33324be80
No redirection or local file specified for downloaded image data. Please specify a local file with --file to save downloaded image or redirect output to another source.

A couple of things

1) It concerns me that this basic behaviour, which has been constant for around 3 years was changed. It signals that any command cannot be considered stable.

2) the behaviour is, in my view, non-standard

If I do:

$ glance image-download 96ed6a3f-4ec9-4e74-a1bd-30e33324be80
No redirection or local file specified for downloaded image data. Please specify a local file with --file to save downloaded image or redirect output to another source.

And I note that the exit status is '0' I must then presume that the content of my image is the text string 'No redirection or local file specified for downloaded image data. Please specify a local file with --file to save downloaded image or redirect output to another source.' (The message is printed to stdout not to stderr.)

and if I then do:

 $ glance image-download 96ed6a3f-4ec9-4e74-a1bd-30e33324be80 > /tmp/img1

I would expect /tmp/img1 to contain the same string: "No redirection or local file specified for downloaded image data. Please specify a local file with --file to save downloaded image or redirect output to another source." (That's how things typically work.)

3) it is not symmetric

I can create an image without specifying an input file

 $ glance image-create ....

we used to be able to download without specifying an output file

4) I can no longer display to stdout without invoking another command

I need to do

 $ glance image-download 96ed6a3f-4ec9-4e74-a1bd-30e33324be80 | cat

to display the image without first piping it into a file.

Are folks open to reverting the behaviour?

[Erno Kuvaja (jokke)]

I do see that Cindy had good intentions with the change, but I totally agree with you on this one.

[Ian Cordasco (icordasc)]

So a few replies:

1. "It signals that any command cannot be considered stable." That's a bit exaggerated if not outright misleading.

2. I agree, the exit code probably shouldn't be 0. That said, it was able to retrieve the image data (so technically it did download the image data) it just didn't save it to disk or anywhere else (if you consider stdout to be saving it somewhere). An appropriate exit code could be non-zero but I don't know what the right exit code is.

3. Things don't need to be symmetric. In fact, I'd argue that symmetry that is actively harmful to a user-experience is not worth keeping. As far as I can tell, you're the only person who was relying on this behaviour. Most people, when they download a file, are spoiled by their browsers that suggest an appropriate filename via Content-Disposition headers. None of our documentation suggests this as a way to test that Glance was successfully setup, in fact, all documentation suggests uploading an image and booting it (e.g., cirros is suggested in the install guide).

4. If only all of openstack were this composable.

"Are folks open to reverting the behaviour?"

I'm not. Like I alluded to above, I think most users who were seeing image data printed to stdout were not doing it intentionally. I know that I for one have used image-download and seen the image data to stdout and uttered some unsavory words in response.

[Ian Cordasco (icordasc)]

Two other points:

"""
 $ glance image-download 96ed6a3f-4ec9-4e74-a1bd-30e33324be80 > /tmp/img1

I would expect /tmp/img1 to contain the same string: "No redirection or local file specified for downloaded image data. Please specify a local file with --file to save downloaded image or redirect output to another source." (That's how things typically work.)
"""

The message says "No redirction [...] specified for downloaded image data. Please [...] redirect output to another source" So why do you expect that to not work? That makes no sense

---

Also, OpenStack uses SemVer, correct? SemVer says, and I quote,

"Major version zero (0.y.z) is for initial development. Anything may change at any time. The public API should not be considered stable."

I'm not saying I agree or that this is justification for breaking this behaviour on you unexpectedly, but, those are the rules by which OpenStack operates. If you want glanceclient to have a real guarantee of API stability, we need to release 1.0.0.

[Stuart McLaren (stuart-mclaren)]

@Ian

I think it's healthy we finally disagree on something :-)

I sense I'm not going to convince you on this.

What irks me about the new behaviour is really this:

If, using a bash shell I do (where commandx is any command):

 $ commandx > /tmp/out
 $ cat /tmp/out

and I then do:

 $ commandx

I expect to get the same result as before to stdout. If I don't, then the behaviour -- for me anyway -- doesn't pass the test of least surprise.

''it did download the image data' -- I'm not sure this is the case. It will download the full image if it is 'small', but it will only partially download the image if it's big (it won't iterate over the full body).

[Stuart McLaren (stuart-mclaren)]

To recap, currently a download with no redirection will:

1) *partially* download the image
2) print a message to stdout (not stderr)
3) return 0 (success)

Is this the desired behaviour?

[Erno Kuvaja (jokke)]

My personal take is that the current behavior, specifically doing partial download is less than desirable. I'd be still happy to revert the change caused this and open for discussion if/how we want to proceed with the std output.

[Niall Bunting (niall-bunting)]

I agree the current behavior is not particularly great, especially as it returns a 0, whilst not actually being a success. I don't think we should download data just to throw it away though if a file is not specified.

However if you look at the behavior of a few other commands such as wget it does not do what the user expects it to do either.

In the case of wget:
wget google.com
Saving to index.html

wget google.com > test.com
Saving to index.html
//also creates an empty test.com

In many cases the downloading of something may not be very helpful in the users terminal. We should try to give the behavior that is most useful to the user. And i am unsure with a large image that showing that to the user would be very helpful especially as outside of the test case the data would not mean much to the user.

[Ian Cordasco (icordasc)]

> My personal take is that the current behavior, specifically doing partial download is less than desirable.

Let me ask this question, since we're not doing anything with the data (in the instance the user doesn't tell us where to put the data), why is not downloading all 30GB of data for no particularly good reason desirable? No, really, why would a user want to wait for that to download in its entirety before being able to do what they actually wanted?

Here's the thing, we can add a flag that says --no-really-I-want-the-raw-bytes-printed-to-my-terminal, but I'm very much against reverting this because the "partial download" issue only affects the shell and doesn't affect users of the Python API.

I can see how this makes it harder for people to make an end-run around swift not being exposed to the user, but it doesn't make it *that* much harder.

[Stuart McLaren (stuart-mclaren)]

@Ian,

I agree 100% that downloading and dumping all 30GB isn't where we want to go here.
(I'm not sure anyone is suggesting that.)

I'm reasonably sure that the current behaviour (downloading then dumping a subset of the image bytes) was never actually intended, so don't see why it has to be preserved.

"--file -" might be reasonable to indicate you want the bytes on stdout.

I do think the partial download should be removed. (My first preference it just to revert.)

[Ian Cordasco (icordasc)]

> I do think the partial download should be removed. (My first preference it just to revert.)

Let's be clear, it only ever partially downloads the image or it only partially downloads the image if you don't provide a file? Which is it because no one bringing up partial downloads is being explicit.

If there's no file, we should probably just not ever make a request at all. That said, this changes the behaviour in a different way: Users won't be able to tell in advance if the image exists. So I'm not quite sure how we're planning to avoid a partial download if the user doesn't specify a --file (I also don't have time to dig through the code and find exactly what portion you're talking about).

[OpenStack Infra (hudson-openstack)]

Change abandoned by Glance Bot (<email address hidden>) on branch: master
Review: https://review.openstack.org/203015

[Kairat Kushaev (kkushaev)]

At the current moment, glance image-download <SOME_ID> returns 1 and outputs the error message to stderr (tested on master for Glance V2 api).
See the link for info: https://github.com/openstack/python-glanceclient/blob/master/glanceclient/common/utils.py#L286.
I also tried to reproduce partial download for output to the terminal. It seems that glanceclient iterating through the body here:
https://github.com/openstack/python-glanceclient/blob/master/glanceclient/common/utils.py#L309. I tested at least on Ubuntu.
Is the issue still valid for the current master?

[Kairat Kushaev (kkushaev)]

Marking as incomplete due to previous comment and no activity.

[Kairat Kushaev (kkushaev)]

*as invalid