| 2018-04-11 10:57:42 |
Christopher Bartz |
bug |
|
|
added bug |
| 2018-04-11 11:01:34 |
Christopher Bartz |
description |
It is possible to create a DLO Manifest with a segments container containing a question mark inside. E.g.
curl -XPUT ... -H"x-object-manifest: a%3Fcontainer/prefix"
But if this DLO Object should be retrieved (GET or HEAD), a 404 (or incorrect 2xx) is returned.
The reason lies in the DLO Middleware: It unquotes the container [1], but does not quote it again when doing the container listing [2]. Thus the internal container listing request looks like this in the proxy log:
Apr 11 12:24:21 swift proxy-server: - - 11/Apr/2018/10/24/21 GET /v1/AUTH_account/a%3Fformat%3Djson%26prefix%3Dprefix HTTP/1.0 404
(be aware that the log line is quoted again)
This means the container listing is done on the container called "a" and not "a?container". If this container does not exist, a 404 is returned. If the container does exist, an incorrect 2xx response would be returned.
[1] https://github.com/openstack/swift/blob/master/swift/common/middleware/dlo.py#L233
[2] https://github.com/openstack/swift/blob/master/swift/common/middleware/dlo.py#L147 |
It is possible to create a DLO Manifest with a segments container containing a question mark inside. E.g.
curl -XPUT ... -H"x-object-manifest: a%3Fcontainer/prefix"
But if this DLO Object should be retrieved (GET or HEAD), a 404 (or incorrect 2xx) is returned.
The reason lies in the DLO Middleware: It unquotes the container [1], but does not quote it again when doing the container listing [2]. Because the path contains now an '?', the '?' is used as the path/query separator, and everything after the '?' is cutted. Thus the internal container listing request looks like this in the proxy log:
Apr 11 12:24:21 swift proxy-server: - - 11/Apr/2018/10/24/21 GET /v1/AUTH_account/a%3Fformat%3Djson%26prefix%3Dprefix HTTP/1.0 404
(be aware that the log line is quoted again)
This means the container listing is done on the container called "a" and not "a?container". If this container does not exist, a 404 is returned. If the container does exist, an incorrect 2xx response would be returned.
[1] https://github.com/openstack/swift/blob/master/swift/common/middleware/dlo.py#L233
[2] https://github.com/openstack/swift/blob/master/swift/common/middleware/dlo.py#L147 |
|
| 2018-04-11 11:58:56 |
Christopher Bartz |
summary |
dlo does not work when segments container contains a '?' |
DLO GET/HEAD does not work if manifest value contains a '?' |
|
| 2018-04-11 12:03:48 |
Christopher Bartz |
description |
It is possible to create a DLO Manifest with a segments container containing a question mark inside. E.g.
curl -XPUT ... -H"x-object-manifest: a%3Fcontainer/prefix"
But if this DLO Object should be retrieved (GET or HEAD), a 404 (or incorrect 2xx) is returned.
The reason lies in the DLO Middleware: It unquotes the container [1], but does not quote it again when doing the container listing [2]. Because the path contains now an '?', the '?' is used as the path/query separator, and everything after the '?' is cutted. Thus the internal container listing request looks like this in the proxy log:
Apr 11 12:24:21 swift proxy-server: - - 11/Apr/2018/10/24/21 GET /v1/AUTH_account/a%3Fformat%3Djson%26prefix%3Dprefix HTTP/1.0 404
(be aware that the log line is quoted again)
This means the container listing is done on the container called "a" and not "a?container". If this container does not exist, a 404 is returned. If the container does exist, an incorrect 2xx response would be returned.
[1] https://github.com/openstack/swift/blob/master/swift/common/middleware/dlo.py#L233
[2] https://github.com/openstack/swift/blob/master/swift/common/middleware/dlo.py#L147 |
It is possible to create a DLO Manifest with a segments container containing a question mark inside. E.g.
curl -XPUT ... -H"x-object-manifest: a%3Fcontainer/prefix"
But if this DLO Object should be retrieved (GET or HEAD), a 404 (or incorrect 2xx) is returned.
The reason lies in the DLO Middleware: It unquotes the container [1], but does not quote it again when doing the container listing [2]. Because the path contains now an '?', the '?' is used as the path/query separator, and everything after the '?' is cutted. Thus the internal container listing request looks like this in the proxy log:
Apr 11 12:24:21 swift proxy-server: - - 11/Apr/2018/10/24/21 GET /v1/AUTH_account/a%3Fformat%3Djson%26prefix%3Dprefix HTTP/1.0 404
(be aware that the log line is quoted again)
This means the container listing is done on the container called "a" and not "a?container". If this container does not exist, a 404 is returned. If the container does exist, an incorrect 2xx response would be returned.
A similar problem arises, if the object prefix contains a '?' . Traceback in this case:
Apr 11 13:54:20 swift proxy-server: - - 11/Apr/2018/11/54/20 GET /v1/AUTH_account/test_segments/serv%3F.json/1523441992.908/0%3Fmultipart-manifest%3Dget HTTP/1.0 499 - - 70 - tx5157e02bd5ab4e6fa345a-005acdf76c - 0.0202 DLO - 1523447660.865747929 1523447660.885992050 0
Apr 11 13:54:20 swift proxy-server: ERROR: An error occurred while retrieving segments: #012Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/swift/common/request_helpers.py", line 442, in _internal_iter#012 (self.name, seg_resp.status_int, seg_req.path))#012SegmentError: ERROR: While processing manifest /v1/AUTH_account/test/serv%3F.json, got 404 while retrieving /v1/AUTH_account/test_segments/serv (txn: tx5157e02bd5ab4e6fa345a-005acdf76c)
[1] https://github.com/openstack/swift/blob/master/swift/common/middleware/dlo.py#L233-L234
[2] https://github.com/openstack/swift/blob/master/swift/common/middleware/dlo.py#L147 |
|
| 2018-04-12 09:32:03 |
Christopher Bartz |
description |
It is possible to create a DLO Manifest with a segments container containing a question mark inside. E.g.
curl -XPUT ... -H"x-object-manifest: a%3Fcontainer/prefix"
But if this DLO Object should be retrieved (GET or HEAD), a 404 (or incorrect 2xx) is returned.
The reason lies in the DLO Middleware: It unquotes the container [1], but does not quote it again when doing the container listing [2]. Because the path contains now an '?', the '?' is used as the path/query separator, and everything after the '?' is cutted. Thus the internal container listing request looks like this in the proxy log:
Apr 11 12:24:21 swift proxy-server: - - 11/Apr/2018/10/24/21 GET /v1/AUTH_account/a%3Fformat%3Djson%26prefix%3Dprefix HTTP/1.0 404
(be aware that the log line is quoted again)
This means the container listing is done on the container called "a" and not "a?container". If this container does not exist, a 404 is returned. If the container does exist, an incorrect 2xx response would be returned.
A similar problem arises, if the object prefix contains a '?' . Traceback in this case:
Apr 11 13:54:20 swift proxy-server: - - 11/Apr/2018/11/54/20 GET /v1/AUTH_account/test_segments/serv%3F.json/1523441992.908/0%3Fmultipart-manifest%3Dget HTTP/1.0 499 - - 70 - tx5157e02bd5ab4e6fa345a-005acdf76c - 0.0202 DLO - 1523447660.865747929 1523447660.885992050 0
Apr 11 13:54:20 swift proxy-server: ERROR: An error occurred while retrieving segments: #012Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/swift/common/request_helpers.py", line 442, in _internal_iter#012 (self.name, seg_resp.status_int, seg_req.path))#012SegmentError: ERROR: While processing manifest /v1/AUTH_account/test/serv%3F.json, got 404 while retrieving /v1/AUTH_account/test_segments/serv (txn: tx5157e02bd5ab4e6fa345a-005acdf76c)
[1] https://github.com/openstack/swift/blob/master/swift/common/middleware/dlo.py#L233-L234
[2] https://github.com/openstack/swift/blob/master/swift/common/middleware/dlo.py#L147 |
It is possible to create a DLO Manifest with a segments container containing a question mark inside. E.g.
curl -XPUT ... -H"x-object-manifest: a%3Fcontainer/prefix"
But if this DLO Object should be retrieved (GET or HEAD), a 404 (or incorrect 2xx) is returned.
The reason lies in the DLO Middleware: It unquotes the container [1], but does not quote it again when doing the container listing [2]. Because the path contains now an '?', the '?' is used as the path/query separator, and everything after the '?' is cutted. Thus the internal container listing request looks like this in the proxy log:
Apr 11 12:24:21 swift proxy-server: - - 11/Apr/2018/10/24/21 GET /v1/AUTH_account/a%3Fformat%3Djson%26prefix%3Dprefix HTTP/1.0 404
(be aware that the log line is quoted again)
This means the container listing is done on the container called "a" and not "a?container". If this container does not exist, a 404 is returned. If the container does exist, an incorrect 2xx response would be returned.
A similar problem arises, if the object prefix contains a '?' and a GET is performed. A 409 conflict is returned (if filesize > 0) and the traceback looks like:
Apr 11 13:54:20 swift proxy-server: - - 11/Apr/2018/11/54/20 GET /v1/AUTH_account/test_segments/serv%3F.json/1523441992.908/0%3Fmultipart-manifest%3Dget HTTP/1.0 499 - - 70 - tx5157e02bd5ab4e6fa345a-005acdf76c - 0.0202 DLO - 1523447660.865747929 1523447660.885992050 0
Apr 11 13:54:20 swift proxy-server: ERROR: An error occurred while retrieving segments: #012Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/swift/common/request_helpers.py", line 442, in _internal_iter#012 (self.name, seg_resp.status_int, seg_req.path))#012SegmentError: ERROR: While processing manifest /v1/AUTH_account/test/serv%3F.json, got 404 while retrieving /v1/AUTH_account/test_segments/serv (txn: tx5157e02bd5ab4e6fa345a-005acdf76c)
[1] https://github.com/openstack/swift/blob/master/swift/common/middleware/dlo.py#L233-L234
[2] https://github.com/openstack/swift/blob/master/swift/common/middleware/dlo.py#L147 |
|
| 2018-04-12 17:19:35 |
Tim Burke |
swift: status |
New |
Confirmed |
|
| 2019-03-21 16:41:46 |
OpenStack Infra |
swift: status |
Confirmed |
Fix Released |
|
| 2019-03-28 21:12:47 |
OpenStack Infra |
tags |
|
in-feature-losf |
|
