Defects observed in SoF with the patch - https://review.openstack.org/#/c/159285/

[Pushkaraj]

Hello,

I was testing SoF with the patch - https://review.openstack.org/#/c/159285/
In that below defect was found: (I am reporting it here, fixing it is conditional to checkin of the patch)

Issues:
1. new argument - frag_index is being passed to DiskFile and DiskFileManager; added **kwargs to signature of __init__().
    Aug 19 09:28:15 brokentooth proxy-server: ERROR __call__ error with PUT /sdb9/175/AUTH_test/sofsos1/requirements.txt :
    Traceback (most recent call last):
    File "/home/swift/swift/swift/obj/server.py", line 909, in __call__
        res = method(req)
    File "/home/swift/swift/swift/common/utils.py", line 2667, in wrapped
        return func(*a, **kw)
    File "/home/swift/swift/swift/common/utils.py", line 1207, in _timing_stats
        resp = func(ctrl, *args, **kwargs)
    File "/home/swift/swiftonfile/swiftonfile/swift/obj/server.py", line 74, in PUT
        return server.ObjectController.PUT(self, request)
    File "/home/swift/swift/swift/common/utils.py", line 2667, in wrapped
        return func(*a, **kw)
    File "/home/swift/swift/swift/common/utils.py", line 1207, in _timing_stats
        resp = func(ctrl, *args, **kwargs) File "/home/swift/swift/swift/obj/server.py", line 508, in PUT
        policy=policy, frag_index=frag_index)
    File "/home/swift/swift/swift/obj/server.py", line 198, in get_diskfile
        device, partition, account, container, obj, policy, **kwargs)
    File "/home/swift/swiftonfile/swiftonfile/swift/obj/diskfile.py", line 220, in get_diskfile
        policy=policy, **kwargs)
    TypeError: __init__() got an unexpected keyword argument 'frag_index' (txn: txc2d501fdbd734d659690d-0055d4846f) (client_ip: 127.0.0.1)

2. It seems that the base class - swift.obj.diskfile.DiskFileManager is not getting initialized, so for now, I have initialized few required members. Pl check the attached patch for details.

3. swift client is reporting error while downloading as etag is encapsulated in double quote.

    [swift@brokentooth swiftonfile]$ swift download sofsos1 requirements.txt
    Pushkaraj: "a662a581b0412faa910ba7df7d23e4a5"
    Error downloading object 'sofsos1/requirements.txt': 'Error downloading requirements.txt: md5sum != etag, a662a581b0412faa910ba7df7d23e4a5 != "a662a581b0412faa910ba7df7d23e4a5"'

    [swift@brokentooth swiftonfile]$ curl -i http://127.0.0.1:8080/v1/AUTH_test/sofsos1/requirements.txt -X GET -H "X-Auth-Token: AUTH_tk32b03f9ff9884df5b211e3c3eef784d7" | grep Etag
    % Total % Received % Xferd Average Speed Time Time Time Current
                                    Dload Upload Total Spent Left Speed
    100 366 100 366 0 0 28162 0 --:--:-- --:--:-- --:--:-- 30500
    Etag: "a662a581b0412faa910ba7df7d23e4a5"

    This may not be related to this code change.

Thx

[Pushkaraj (pushkarajthorat)]

FYI, this was basic SAIO setup with the patch applied, and single-process.conf points to swiftonfile egg.

Also, the object POST (metadata update) failed with below error.

metadata POST fails with error:
Traceback (most recent call last):
  File "/home/swift/swift/swift/obj/server.py", line 908, in __call__
    res = method(req)
  File "/home/swift/swift/swift/common/utils.py", line 2667, in wrapped
    return func(*a, **kw)
  File "/home/swift/swift/swift/common/utils.py", line 1207, in _timing_stats
    resp = func(ctrl, *args, **kwargs)
  File "/home/swift/swiftonfile/swiftonfile/swift/obj/server.py", line 74, in PUT
    return server.ObjectController.PUT(self, request)
  File "/home/swift/swift/swift/common/utils.py", line 2667, in wrapped
    return func(*a, **kw)
  File "/home/swift/swift/swift/common/utils.py", line 1207, in _timing_stats
    resp = func(ctrl, *args, **kwargs)
  File "/home/swift/swift/swift/obj/server.py", line 584, in PUT
    for chunk in iter(timeout_reader, ''):
  File "/home/swift/swift/swift/obj/server.py", line 370, in timeout_reader
    return file_like.read(self.network_chunk_size)
  File "/home/swift/swift/swift/common/utils.py", line 2794, in read
    chunk = self.wsgi_input.read(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/eventlet/wsgi.py", line 190, in read
    return self._do_read(self.rfile.read, length)
  File "/usr/lib/python2.7/site-packages/eventlet/wsgi.py", line 124, in _do_read
    if length and length > self.content_length - self.position:
TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'

[Thiago da Silva (thiagodasilva)]

@Pushkaraj. I was able to reproduce the problem and this looks related to the same issue I mentioned earlier which is SP not currently supporting COPY. Swift has the option object_post_as_copy set to true, which means when you send a POST, it gets changed to a copy request. If you change that object_post_as_copy to false and send a POST again you won't run into this problem.

Still, copy is an current issue with SP. I'm going to test SP with the copy middleware patch as I think that should solve the problem.