TRACEBACK if no output file specified while image download

[Ravi Shekhar Jethani]

While downloading image data via 'glance client' we require to explicitly specify where the output should go. We need to specify output file either via '--file' command option or via stdout redirection. If we do not explicitly specify we get an error message which is fine but current implementation(on master) causes a TRACEBACK on the service side. Although the TRACEBACK is not fatal but it does look bad.

Steps to reproduce:

$ glance image-download <IMAGE ID>

Sample command output:

vagrant@ubuntu16:~$ glance image-download af9105b5-fc18-47b4-890d-c7f46e97b190
No redirection or local file specified for downloaded image data. Please specify a local file with --file to save downloaded image or redirect output to another source.

Sample glance api log output:

2016-10-18 15:26:25.481 DEBUG glance.api.middleware.version_negotiation [-] Using url versioning from (pid=4737) process_request /o
pt/stack/glance/glance/api/middleware/version_negotiation.py:58
2016-10-18 15:26:25.482 DEBUG glance.api.middleware.version_negotiation [-] Matched version: v2 from (pid=4737) process_request /op
t/stack/glance/glance/api/middleware/version_negotiation.py:70
2016-10-18 15:26:25.483 DEBUG glance.api.middleware.version_negotiation [-] new path /v2/images/af9105b5-fc18-47b4-890d-c7f46e97b190/file from (pid=4737) process_request /opt/stack/glance/glance/api/middleware/version_negotiation.py:71
2016-10-18 15:26:25.528 DEBUG glance.api.middleware.cache [req-42c4c736-baa2-41fe-b0ff-a9ba16dcbf0a 32b19608a63e45268f2afcac14f59539 9cecc6ab160e405cb854d1671657a983] Cache hit for image 'af9105b5-fc18-47b4-890d-c7f46e97b190' from (pid=4737) process_request /opt/stack/glance/glance/api/middleware/cache.py:165
2016-10-18 15:26:25.532 INFO eventlet.wsgi.server [req-42c4c736-baa2-41fe-b0ff-a9ba16dcbf0a 32b19608a63e45268f2afcac14f59539 9cecc6ab160e405cb854d1671657a983] Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/eventlet/wsgi.py", line 506, in handle_one_response
    write(b''.join(towrite))
  File "/usr/local/lib/python2.7/dist-packages/eventlet/wsgi.py", line 452, in write
    wfile.writelines(towrite)
  File "/usr/lib/python2.7/socket.py", line 338, in writelines
    self.flush()
  File "/usr/lib/python2.7/socket.py", line 307, in flush
    self._sock.sendall(view[write_offset:write_offset+buffer_size])
  File "/usr/local/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 385, in sendall
    tail = self.send(data, flags)
  File "/usr/local/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 379, in send
    return self._send_loop(self.fd.send, data, flags)
  File "/usr/local/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 366, in _send_loop
    return send_method(data, *args)
error: [Errno 104] Connection reset by peer

We have same behavior for both v1 and v2 glance clients.

We can avoid this TRACEBACK by simply moving checks for command options before even connecting to glance service

[Ian Cordasco (icordasc)]

This is the correct behaviour for python-glanceclient. The fact that we close the connection before consuming the socket on the client side indicates a problem with the way eventlet handles this. This shouldn't result in an error in our logs and the entirety of that traceback comes from eventlet.

[Ravi Shekhar Jethani (ravishekar-jethani)]

Hi Ian,

I do not meant changing the behavior of glance client what I meant was that we can easily avoid the traceback by making little change to the way we are checking for command options. Kindly see updated bug

[Ian Cordasco (icordasc)]

Ravi, in that case this is a bug against the glanceclient, which is a separate project. That said, I'm not convinced we need to fix this. Here's why:

1. By not checking for the arguments, we can then issue a proper error if there is no image. Meaning that not stopping early, a user will see if they can actually download that image.

2. While not common on small clouds, connection errors like this are almost certainly fairly common in very large environments, especially with unreliable internet connections for clients.

Are you concerned by this connection error appearing in your logs? Or is there some other reason you think this needs fixing?

[Ravi Shekhar Jethani (ravishekar-jethani)]

Hi Ian,

IMO A good command line tool should always validate the options/arguments given by the user then only it should start (expensive)calculations. A simple example of 'ls' command may help here. I simply execute:

$ ls --abc /root
ls: unrecognized option '--abc'
Try 'ls --help' for more information.

By looking at the output we can see 'ls' is validating options first before actually trying to list contents of '/root'(which will fail)

This is what I want for glance client. I reported a similar bug: https://bugs.launchpad.net/python-novaclient/+bug/1502866 for nova client earlier.

It is better to FORCE the user to use the command correctly irrespective of the cases whether image exists or does he has proper permissions etc. The glance command is doing that by giving proper error but we can improve a bit more just by reordering some lines. It will simply avoid few service requests.

[Stuart McLaren (stuart-mclaren)]

(From memory) a full description of the behaviour here is that the image will actually be partially downloaded by the client before the traceback occurs. Ideally it should be possible to detect that an invalid command has been entered and not connect to the server at all. That would prevent both the wasted partial download and a 'false positive' server side log entry which seems to indicate a network problem of some kind.

I think I slightly disagree with Ian on '1' above. My preference generally would be to 'fail early'. Being able to access image X at time 't' doesn't guarantee you'll be able to access it at time 't+1' when you use a corrected command.

[Ravi Shekhar Jethani (ravishekar-jethani)]

Ian, Stuart,

Can we mark this as a valid bug?