uploading ticket fails with broken pipe (does not retry on broken connection to Swift)

[Vincent Ladeuil]

.venv/bin/ubuntu-ci create_ticket -t "Testing kernel" -d "A ticket for Testing kernel" -a linux-image-3.11.0-19-generic -o vila -s ~/ci/kernel/linux_3.11.0-19.33ubuntu1~vila1ubuntu1_source.changes

failed with:

ubuntu-ci ERROR Unexpected exception
Traceback (most recent call last):
  File "/home/vila/ci/ubuntu-ci-services-itself/trunk/cli/ubuntu-ci", line 121, in main
    args.func(args)
  File "/home/vila/ci/ubuntu-ci-services-itself/trunk/cli/ci_libs/ticket.py", line 153, in new_ticket
    new_ticket.add_new_ticket(args)
  File "/home/vila/ci/ubuntu-ci-services-itself/trunk/cli/ci_libs/ticket.py", line 146, in add_new_ticket
    subticket._process()
  File "/home/vila/ci/ubuntu-ci-services-itself/trunk/cli/ci_libs/ticket.py", line 55, in _process
    location = upload_files(file_path, self.ticket_id)
  File "/home/vila/ci/ubuntu-ci-services-itself/trunk/cli/ci_libs/file_handler.py", line 128, in upload_files
    location = ds.put_file(file_path, fp)
  File "/home/vila/ci/ubuntu-ci-services-itself/trunk/ci-utils/ci_utils/data_store/__init__.py", line 69, in put_file
    content_type=content_type)
  File "/home/vila/ci/ubuntu-ci-services-itself/trunk/.venv/local/lib/python2.7/site-packages/swiftclient/client.py", line 890, in put_object
    conn.send('%x\r\n%s\r\n' % (len(chunk), chunk))
  File "/usr/lib/python2.7/httplib.py", line 805, in send
    self.sock.sendall(data)
  File "/usr/lib/python2.7/ssl.py", line 293, in sendall
    v = self.send(data[count:])
  File "/usr/lib/python2.7/ssl.py", line 262, in send
    v = self._sslobj.write(data)
error: [Errno 32] Broken pipe

Note that the traceback is similar to bug #1289567 but for a different purpose.

Here, I was trying to create a ticket for kernel testing.

[Vincent Ladeuil (vila)]

For this specific case uploading >100M seems like:
- a blocker with my actual network connection,
- a pain for anyone wanting to create such a ticket.

I thought there was a way to let launchpad find the source tarball itself but I don't know the specifics.

If that was possible then creating the ticket may be possible with uploading only the .dsc and .source_changes which have far more reasonable sizes ?

[Vincent Ladeuil (vila)]

assigning to Ursula for feedback about skipping the tar.gz upload with help from lp

[Celso Providelo (cprov)]

Since CLI is using tempurls for uploading and downloading swift objects now and the inner functions, implemented with python `requests` expect payloads to be huge (using response.iter_content() for GET and file objects for PUT), this is not longer an issue.