Error in uploading built DU to glance

Bug #1441828 reported by Devdatta Kulkarni on 2015-04-08
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Solum
High
Devdatta Kulkarni

Bug Description

Testing with the following settings:

image_format=docker
image_storage=glance
language_pack=auto

saw this in worker screen:

2015-04-08 19:19:24.825 28420 DEBUG solum.worker.handlers.shell [-] Build logs stored at /var/log/solum/worker/build-decff9bb-38f1-4399-a0fe-85e71f57a60c.log _do_build /opt/stack/solum/solum/worker/handlers/shell.py:371

(---CHECK THIS---) 500 Internal Server Error: Failed to upload image 5c471b54-d53c-47d6-a654-f4f8c0d8e3cc (HTTP 500)

2015-04-08 19:21:49.486 28420 DEBUG solum.worker.handlers.shell [-] User log upload strategy: swift upload_task_log /opt/stack/solum/solum/worker/handlers/shell.py:62

(---CHECK THIS---) 2015-04-08 19:21:49.487 28420 DEBUG solum.uploaders.common [-] Could not parse json line: { "@timestamp": "2015-04-08T19:19:49+0000", "project_id": "b13e17a7527f46c7b943eb10da3317ee", "commit_id": "a553ff-----> Python app detected"}058737", "build_id": "3c051db818d545ab7c5a6f578fc08391f6793d94e4a66cad9ce28704add00ac5", "task": "build", "message": "
 transform_jsonlog /opt/stack/solum/solum/uploaders/common.py:83
2015-04-08 19:21:49.487 28420 DEBUG solum.uploaders.common [-] Could not parse json line: { "@timestamp": "2015-04-08T19:19:57+0000", "project_id": "b13e17a7527f46c7b943eb10da3317ee", "commit_id": "a553ff-----> Installing dependencies with pip"}ld_id": "3c051db818d545ab7c5a6f578fc08391f6793d94e4a66cad9ce28704add00ac5", "task": "build", "message": "
 transform_jsonlog /opt/stack/solum/solum/uploaders/common.py:83
2015-04-08 19:21:49.487 28420 DEBUG solum.uploaders.common [-] Could not parse json line: { "@timestamp": "2015-04-08T19:19:58+0000", "project_id": "b13e17a7527f46c7b943eb10da3317ee", "commit_id": "a553ff You are using pip version 6.1.0.dev0, however version 6.1.1 is available."}c08391f6793d94e4a66cad9ce28704add00ac5", "task": "build", "message": "
 transform_jsonlog /opt/stack/solum/solum/uploaders/common.py:83
2015-04-08 19:21:49.488 28420 DEBUG solum.uploaders.common [-] Could not parse json line: { "@timestamp": "2015-04-08T19:19:58+0000", "project_id": "b13e17a7527f46c7b943eb10da3317ee", "commit_id": "a553ff You should consider upgrading via the 'pip install --upgrade pip' command."}08391f6793d94e4a66cad9ce28704add00ac5", "task": "build", "message": "
 transform_jsonlog /opt/stack/solum/solum/uploaders/common.py:83
2015-04-08 19:21:49.488 28420 DEBUG solum.uploaders.common [-] Could not parse json line: { "@timestamp": "2015-04-08T19:19:59+0000", "project_id": "b13e17a7527f46c7b943eb10da3317ee", "commit_id": "a553ff"}2c7631fe9908029a0e76aa59f3058737", "build_id": "3c051db818d545ab7c5a6f578fc08391f6793d94e4a66cad9ce28704add00ac5", "task": "build", "message": "
 transform_jsonlog /opt/stack/solum/solum/uploaders/common.py:83
2015-04-08 19:21:49.488 28420 DEBUG solum.uploaders.common [-] Could not parse json line: { "@timestamp": "2015-04-08T19:20:02+0000", "project_id": "b13e17a7527f46c7b943eb10da3317ee", "commit_id": "a553ff-----> Discovering process types"}", "build_id": "3c051db818d545ab7c5a6f578fc08391f6793d94e4a66cad9ce28704add00ac5", "task": "build", "message": "
 transform_jsonlog /opt/stack/solum/solum/uploaders/common.py:83
2015-04-08 19:21:49.488 28420 DEBUG solum.uploaders.common [-] Could not parse json line: { "@timestamp": "2015-04-08T19:20:08+0000", "project_id": "b13e17a7527f46c7b943eb10da3317ee", "commit_id": "a553ff-----> Compiled slug size is 33M"}", "build_id": "3c051db818d545ab7c5a6f578fc08391f6793d94e4a66cad9ce28704add00ac5", "task": "build", "message": "
 transform_jsonlog /opt/stack/solum/solum/uploaders/common.py:83
2015-04-08 19:21:49.488 28420 DEBUG solum.uploaders.common [-] Could not parse json line: { "@timestamp": "2015-04-08T19:20:10+0000", "project_id": "b13e17a7527f46c7b943eb10da3317ee", "commit_id": "a553ff"}nding build context to Docker daemon 33.78 MB: "3c051db818d545ab7c5a6f578fc08391f6793d94e4a66cad9ce28704add00ac5", "task": "build", "message": "Sending build context to Docker daemon 557.1 kB
 transform_jsonlog /opt/stack/solum/solum/uploaders/common.py:83
2015-04-08 19:21:49.489 28420 DEBUG solum.uploaders.common [-] Could not parse json line: { "@timestamp": "2015-04-08T19:20:10+0000", "project_id": "b13e17a7527f46c7b943eb10da3317ee", "commit_id": "a553ff"}2c7631fe9908029a0e76aa59f3058737", "build_id": "3c051db818d545ab7c5a6f578fc08391f6793d94e4a66cad9ce28704add00ac5", "task": "build", "message": "Sending build context to Docker daemon
 transform_jsonlog /opt/stack/solum/solum/uploaders/common.py:83
2015-04-08 19:21:49.489 28420 DEBUG solum.uploaders.swift [-] Uploading log to Swift. solum-logs, cherrypy-71177b9f-db83-4b0b-b04a-c05c5d96f0dd/build-decff9bb-38f1-4399-a0fe-85e71f57a60c.log upload_log /opt/stack/solum/solum/uploaders/swift.py:42
2015-04-08 19:21:49.621 28420 DEBUG swiftclient [-] REQ: curl -i http://10.0.2.15:8080/v1/AUTH_b13e17a7527f46c7b943eb10da3317ee/solum-logs -X PUT -H "Content-Length: 0" -H "X-Auth-Token: c5cfa4c834c943efb86803da76c6efcf" http_log /usr/local/lib/python2.7/dist-packages/swiftclient/client.py:99
2015-04-08 19:21:49.621 28420 DEBUG swiftclient [-] RESP STATUS: 202 Accepted http_log /usr/local/lib/python2.7/dist-packages/swiftclient/client.py:100
2015-04-08 19:21:49.621 28420 DEBUG swiftclient [-] RESP HEADERS: [('date', 'Wed, 08 Apr 2015 19:21:49 GMT'), ('content-length', '76'), ('content-type', 'text/html; charset=UTF-8'), ('connection', 'keep-alive'), ('x-trans-id', 'tx7bb1c30c7bad4bb5b9315-0055257fcd')] http_log /usr/local/lib/python2.7/dist-packages/swiftclient/client.py:101
2015-04-08 19:21:49.621 28420 DEBUG swiftclient [-] RESP BODY: <html><h1>Accepted</h1><p>The request is accepted for processing.</p></html> http_log /usr/local/lib/python2.7/dist-packages/swiftclient/client.py:103
2015-04-08 19:21:49.708 28420 WARNING requests.packages.urllib3.connectionpool [-] Connection pool is full, discarding connection: 10.0.2.15
2015-04-08 19:21:49.708 28420 DEBUG swiftclient [-] REQ: curl -i http://10.0.2.15:8080/v1/AUTH_b13e17a7527f46c7b943eb10da3317ee/solum-logs/cherrypy-71177b9f-db83-4b0b-b04a-c05c5d96f0dd/build-decff9bb-38f1-4399-a0fe-85e71f57a60c.log -X PUT -H "X-Auth-Token: c5cfa4c834c943efb86803da76c6efcf" http_log /usr/local/lib/python2.7/dist-packages/swiftclient/client.py:99
2015-04-08 19:21:49.708 28420 DEBUG swiftclient [-] RESP STATUS: 201 Created http_log /usr/local/lib/python2.7/dist-packages/swiftclient/client.py:100
2015-04-08 19:21:49.709 28420 DEBUG swiftclient [-] RESP HEADERS: [('content-length', '0'), ('last-modified', 'Wed, 08 Apr 2015 19:21:50 GMT'), ('connection', 'keep-alive'), ('etag', 'a07f25c0e2d01cf7b4e8ebcebd7cde22'), ('x-trans-id', 'txa2f6a3d7f92c4e6988a57-0055257fcd'), ('date', 'Wed, 08 Apr 2015 19:21:49 GMT'), ('content-type', 'text/html; charset=UTF-8')] http_log /usr/local/lib/python2.7/dist-packages/swiftclient/client.py:101
2015-04-08 19:21:49.709 28420 DEBUG solum.uploaders.swift [-] Logfile uploaded to Swift. upload_log /opt/stack/solum/solum/uploaders/swift.py:49

(---CHECK THIS---) 2015-04-08 19:21:49.718 28420 DEBUG solum.worker.handlers.shell [-] created_image_id: ID DOCKER_IMAGE_TAG=b13e17a7527f46c7b943eb10da3317ee-web-20150408t192009942602884-a553ffbd2c7631fe9908029a0e76aa59f3058737 _do_build /opt/stack/solum/solum/worker/handlers/shell.py:411

2015-04-08 19:21:49.718 28420 DEBUG solum.worker.handlers.shell [req-ce2190f4-bdf7-4327-bc84-d1e60c011ddd] sbuild id:5 READY (built successfully) ID DOCKER_IMAGE_TAG=b13e17a7527f46c7b943eb10da3317ee-web-20150408t192009942602884-a553ffbd2c7631fe9908029a0e76aa59f3058737 5

----

From the trace there are at least three issues (marked with ---CHECK THIS---):

1) Uploading to glance seems to be failing

2) Parsing in uploader common seems to be failing

3) created_image_id seems to be not generated correctly

Changed in solum:
assignee: nobody → Devdatta (devdatta-kulkarni)
Adrian Otto (aotto) on 2015-06-12
Changed in solum:
status: New → Triaged
importance: Undecided → High
milestone: none → 2015.1.1
Changed in solum:
status: Triaged → Invalid
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers