Error in uploading built DU to glance

Bug #1441828 reported by Devdatta Kulkarni
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Solum
Invalid
High
Devdatta Kulkarni

Bug Description

Testing with the following settings:

image_format=docker
image_storage=glance
language_pack=auto

saw this in worker screen:

2015-04-08 19:19:24.825 28420 DEBUG solum.worker.handlers.shell [-] Build logs stored at /var/log/solum/worker/build-decff9bb-38f1-4399-a0fe-85e71f57a60c.log _do_build /opt/stack/solum/solum/worker/handlers/shell.py:371

(---CHECK THIS---) 500 Internal Server Error: Failed to upload image 5c471b54-d53c-47d6-a654-f4f8c0d8e3cc (HTTP 500)

2015-04-08 19:21:49.486 28420 DEBUG solum.worker.handlers.shell [-] User log upload strategy: swift upload_task_log /opt/stack/solum/solum/worker/handlers/shell.py:62

(---CHECK THIS---) 2015-04-08 19:21:49.487 28420 DEBUG solum.uploaders.common [-] Could not parse json line: { "@timestamp": "2015-04-08T19:19:49+0000", "project_id": "b13e17a7527f46c7b943eb10da3317ee", "commit_id": "a553ff-----> Python app detected"}058737", "build_id": "3c051db818d545ab7c5a6f578fc08391f6793d94e4a66cad9ce28704add00ac5", "task": "build", "message": "
 transform_jsonlog /opt/stack/solum/solum/uploaders/common.py:83
2015-04-08 19:21:49.487 28420 DEBUG solum.uploaders.common [-] Could not parse json line: { "@timestamp": "2015-04-08T19:19:57+0000", "project_id": "b13e17a7527f46c7b943eb10da3317ee", "commit_id": "a553ff-----> Installing dependencies with pip"}ld_id": "3c051db818d545ab7c5a6f578fc08391f6793d94e4a66cad9ce28704add00ac5", "task": "build", "message": "
 transform_jsonlog /opt/stack/solum/solum/uploaders/common.py:83
2015-04-08 19:21:49.487 28420 DEBUG solum.uploaders.common [-] Could not parse json line: { "@timestamp": "2015-04-08T19:19:58+0000", "project_id": "b13e17a7527f46c7b943eb10da3317ee", "commit_id": "a553ff You are using pip version 6.1.0.dev0, however version 6.1.1 is available."}c08391f6793d94e4a66cad9ce28704add00ac5", "task": "build", "message": "
 transform_jsonlog /opt/stack/solum/solum/uploaders/common.py:83
2015-04-08 19:21:49.488 28420 DEBUG solum.uploaders.common [-] Could not parse json line: { "@timestamp": "2015-04-08T19:19:58+0000", "project_id": "b13e17a7527f46c7b943eb10da3317ee", "commit_id": "a553ff You should consider upgrading via the 'pip install --upgrade pip' command."}08391f6793d94e4a66cad9ce28704add00ac5", "task": "build", "message": "
 transform_jsonlog /opt/stack/solum/solum/uploaders/common.py:83
2015-04-08 19:21:49.488 28420 DEBUG solum.uploaders.common [-] Could not parse json line: { "@timestamp": "2015-04-08T19:19:59+0000", "project_id": "b13e17a7527f46c7b943eb10da3317ee", "commit_id": "a553ff"}2c7631fe9908029a0e76aa59f3058737", "build_id": "3c051db818d545ab7c5a6f578fc08391f6793d94e4a66cad9ce28704add00ac5", "task": "build", "message": "
 transform_jsonlog /opt/stack/solum/solum/uploaders/common.py:83
2015-04-08 19:21:49.488 28420 DEBUG solum.uploaders.common [-] Could not parse json line: { "@timestamp": "2015-04-08T19:20:02+0000", "project_id": "b13e17a7527f46c7b943eb10da3317ee", "commit_id": "a553ff-----> Discovering process types"}", "build_id": "3c051db818d545ab7c5a6f578fc08391f6793d94e4a66cad9ce28704add00ac5", "task": "build", "message": "
 transform_jsonlog /opt/stack/solum/solum/uploaders/common.py:83
2015-04-08 19:21:49.488 28420 DEBUG solum.uploaders.common [-] Could not parse json line: { "@timestamp": "2015-04-08T19:20:08+0000", "project_id": "b13e17a7527f46c7b943eb10da3317ee", "commit_id": "a553ff-----> Compiled slug size is 33M"}", "build_id": "3c051db818d545ab7c5a6f578fc08391f6793d94e4a66cad9ce28704add00ac5", "task": "build", "message": "
 transform_jsonlog /opt/stack/solum/solum/uploaders/common.py:83
2015-04-08 19:21:49.488 28420 DEBUG solum.uploaders.common [-] Could not parse json line: { "@timestamp": "2015-04-08T19:20:10+0000", "project_id": "b13e17a7527f46c7b943eb10da3317ee", "commit_id": "a553ff"}nding build context to Docker daemon 33.78 MB: "3c051db818d545ab7c5a6f578fc08391f6793d94e4a66cad9ce28704add00ac5", "task": "build", "message": "Sending build context to Docker daemon 557.1 kB
 transform_jsonlog /opt/stack/solum/solum/uploaders/common.py:83
2015-04-08 19:21:49.489 28420 DEBUG solum.uploaders.common [-] Could not parse json line: { "@timestamp": "2015-04-08T19:20:10+0000", "project_id": "b13e17a7527f46c7b943eb10da3317ee", "commit_id": "a553ff"}2c7631fe9908029a0e76aa59f3058737", "build_id": "3c051db818d545ab7c5a6f578fc08391f6793d94e4a66cad9ce28704add00ac5", "task": "build", "message": "Sending build context to Docker daemon
 transform_jsonlog /opt/stack/solum/solum/uploaders/common.py:83
2015-04-08 19:21:49.489 28420 DEBUG solum.uploaders.swift [-] Uploading log to Swift. solum-logs, cherrypy-71177b9f-db83-4b0b-b04a-c05c5d96f0dd/build-decff9bb-38f1-4399-a0fe-85e71f57a60c.log upload_log /opt/stack/solum/solum/uploaders/swift.py:42
2015-04-08 19:21:49.621 28420 DEBUG swiftclient [-] REQ: curl -i http://10.0.2.15:8080/v1/AUTH_b13e17a7527f46c7b943eb10da3317ee/solum-logs -X PUT -H "Content-Length: 0" -H "X-Auth-Token: c5cfa4c834c943efb86803da76c6efcf" http_log /usr/local/lib/python2.7/dist-packages/swiftclient/client.py:99
2015-04-08 19:21:49.621 28420 DEBUG swiftclient [-] RESP STATUS: 202 Accepted http_log /usr/local/lib/python2.7/dist-packages/swiftclient/client.py:100
2015-04-08 19:21:49.621 28420 DEBUG swiftclient [-] RESP HEADERS: [('date', 'Wed, 08 Apr 2015 19:21:49 GMT'), ('content-length', '76'), ('content-type', 'text/html; charset=UTF-8'), ('connection', 'keep-alive'), ('x-trans-id', 'tx7bb1c30c7bad4bb5b9315-0055257fcd')] http_log /usr/local/lib/python2.7/dist-packages/swiftclient/client.py:101
2015-04-08 19:21:49.621 28420 DEBUG swiftclient [-] RESP BODY: <html><h1>Accepted</h1><p>The request is accepted for processing.</p></html> http_log /usr/local/lib/python2.7/dist-packages/swiftclient/client.py:103
2015-04-08 19:21:49.708 28420 WARNING requests.packages.urllib3.connectionpool [-] Connection pool is full, discarding connection: 10.0.2.15
2015-04-08 19:21:49.708 28420 DEBUG swiftclient [-] REQ: curl -i http://10.0.2.15:8080/v1/AUTH_b13e17a7527f46c7b943eb10da3317ee/solum-logs/cherrypy-71177b9f-db83-4b0b-b04a-c05c5d96f0dd/build-decff9bb-38f1-4399-a0fe-85e71f57a60c.log -X PUT -H "X-Auth-Token: c5cfa4c834c943efb86803da76c6efcf" http_log /usr/local/lib/python2.7/dist-packages/swiftclient/client.py:99
2015-04-08 19:21:49.708 28420 DEBUG swiftclient [-] RESP STATUS: 201 Created http_log /usr/local/lib/python2.7/dist-packages/swiftclient/client.py:100
2015-04-08 19:21:49.709 28420 DEBUG swiftclient [-] RESP HEADERS: [('content-length', '0'), ('last-modified', 'Wed, 08 Apr 2015 19:21:50 GMT'), ('connection', 'keep-alive'), ('etag', 'a07f25c0e2d01cf7b4e8ebcebd7cde22'), ('x-trans-id', 'txa2f6a3d7f92c4e6988a57-0055257fcd'), ('date', 'Wed, 08 Apr 2015 19:21:49 GMT'), ('content-type', 'text/html; charset=UTF-8')] http_log /usr/local/lib/python2.7/dist-packages/swiftclient/client.py:101
2015-04-08 19:21:49.709 28420 DEBUG solum.uploaders.swift [-] Logfile uploaded to Swift. upload_log /opt/stack/solum/solum/uploaders/swift.py:49

(---CHECK THIS---) 2015-04-08 19:21:49.718 28420 DEBUG solum.worker.handlers.shell [-] created_image_id: ID DOCKER_IMAGE_TAG=b13e17a7527f46c7b943eb10da3317ee-web-20150408t192009942602884-a553ffbd2c7631fe9908029a0e76aa59f3058737 _do_build /opt/stack/solum/solum/worker/handlers/shell.py:411

2015-04-08 19:21:49.718 28420 DEBUG solum.worker.handlers.shell [req-ce2190f4-bdf7-4327-bc84-d1e60c011ddd] sbuild id:5 READY (built successfully) ID DOCKER_IMAGE_TAG=b13e17a7527f46c7b943eb10da3317ee-web-20150408t192009942602884-a553ffbd2c7631fe9908029a0e76aa59f3058737 5

----

From the trace there are at least three issues (marked with ---CHECK THIS---):

1) Uploading to glance seems to be failing

2) Parsing in uploader common seems to be failing

3) created_image_id seems to be not generated correctly

Changed in solum:
assignee: nobody → Devdatta (devdatta-kulkarni)
Adrian Otto (aotto)
Changed in solum:
status: New → Triaged
importance: Undecided → High
milestone: none → 2015.1.1
Changed in solum:
status: Triaged → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.