Errors appeared in my log:
Aug 6 10:21:05 sw2 object-server: ssync subrequest failed with 499: PUT /cold5/0/AUTH_cff692e324/%D0%97%D0%B0%D0%BA%D0%B0%D0%B7%D1%8B/%D0%A2%D1%80%D0%B5%D0%B1%D0%BE%D0%B2%D0%B0%D0%BD%D0%B8%D1
%8F%20%D0%BE%20%D0%BF%D0%BE%D0%B3%D0%B0%D1%88%D0%B5%D0%BD%D0%B8%D0%B8%20%D0%B4%D0%BE%D0%BB%D0%B3%D0%B0%20%28%D0%B8%D1%81%D0%BA%D0%BE%D0%B2%D0%BE%D0%B5%29/28174.pdf
Aug 6 10:21:05 sw2 object-server: STDERR: Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/eventlet/wsgi.py", line 599, in handle_one_response#012 write(b'')#012 File "/usr/lib/python2.7/site-packages/eventlet/wsgi.py", line 538, in write#012 wfile.flush()#012 File "/usr/lib64/python2.7/socket.py", line 303, in flush#012 self._sock.sendall(view[write_offset:write_offset+buffer_size])#012 File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 403, in sendall#012 tail = self.send(data, flags)#012 File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 397, in send#012 return self._send_loop(self.fd.send, data, flags)#012 File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 384, in _send_loop#012 return send_method(data, *args)#012error: [Errno 32] Broken pipe
All services and all servers are available.
Information about the object:
swift-object-info /srv/node/object2/cold5/objects/0/14a/05ca2b8d9ad708ea9dba3978ca49f14a/1593157868.69436#1#d.data Path: /AUTH_cff692e3df0a4f83a24/Зазы/Треия (lkj)/28174.pdf
Account: AUTH_cff692e3df0a4f8389b24
Container: Зазы
Object: Треия (lkj)/28174.pdf
Object hash: 05ca2b8d9ad708ea9dba3978ca49f14a
Content-Type: application/pdf
Timestamp: 2020-06-26T07:51:08.694360 (1593157868.69436)
System Metadata:
X-Object-Sysmeta-Ec-Content-Length: 0
X-Object-Sysmeta-Ec-Etag: d41d8cd98f00b204e9800998ecf8427e
X-Object-Sysmeta-Ec-Frag-Index: 1
X-Object-Sysmeta-Ec-Scheme: liberasurecode_rs_vand 2+2
X-Object-Sysmeta-Ec-Segment-Size: 1048576
Transient System Metadata:
No metadata found
User Metadata:
No metadata found
Other Metadata:
X-Object-Manifest: %D0%97%D0%B0%D0%BA%D0%B0%D0%B7%D1%8B_segments/%D0%A2%D1%80%D0%B5%D0%B1%D0%BE%D0%B2%D0%B0%D0%BD%D0%B8%D1%8F%20%D0%BE%20%D0%BF%D0%BE%D0%B3%D0%B0%D1%88%D0%B5%D0%BD%D0%B8%D0%B8%20%D0%B4%D0%BE%D0%BB%D0%B3%D0%B0%20%28%D0%B8%D1%81%D0%BA%D0%BE%D0%B2%D0%BE%D0%B5%29/28174.pdf/slo/1593157868.688028/67108864
ETag: d41d8cd98f00b204e9800998ecf8427e (valid)
Content-Length: 0 (valid)
Partition 0
Hash 05ca2b8d9ad708ea9dba3978ca49f14a
According to the information, the file was created more than a month ago.
2020-06-26T07:51:08.694360
A month ago, the file migrated to another disk:
-rwxr-xr-x 1 swift swift 0 Jul 6 03:53 /srv/node/object2/cold5/objects/0/14a/05ca2b8d9ad708ea9dba3978ca49f14a/1593157868.69436#1#d.data
Why is the PUT command called and it ends with an error?
How do I know what's wrong with an object?
I found the reason for the error.
A DLO object was uploaded to the cloud storage.
Then the object type was changed to SLO.
But for some reason, the change only occurred on one server.
On all other servers, the object remains DLO.
Currently on the same server: pdf;swift_ bytes=1089467 26T07:51: 09.214980 (1593157869.21498) Sysmeta- Container- Update- Override- Etag: 8d3a35d5c6405e6 55b58aaa4f9964e 2d; slo_etag= ae9051fbff7dc04 99ee00ce3bcde9f ff Sysmeta- Ec-Content- Length: 462 Sysmeta- Ec-Etag: 8d3a35d5c6405e6 55b58aaa4f9964e 2d Sysmeta- Ec-Frag- Index: 3 Sysmeta- Ec-Scheme: liberasurecode_ rs_vand 2+2 Sysmeta- Ec-Segment- Size: 1048576 Sysmeta- Slo-Etag: ae9051fbff7dc04 99ee00ce3bcde9f ff Sysmeta- Slo-Size: 1089467 Large-Object: True 2743dec30d8036b 9d (valid) a9dba3978ca49f1 4a
Content-Type: application/
Timestamp: 2020-06-
System Metadata:
X-Object-
X-Object-
X-Object-
X-Object-
X-Object-
X-Object-
X-Object-
X-Object-
Transient System Metadata:
No metadata found
User Metadata:
No metadata found
Other Metadata:
X-Static-
ETag: 537a67d6e47c9b2
Content-Length: 312 (valid)
Partition 0
Hash 05ca2b8d9ad708e
On all other servers: 26T07:51: 08.694360 (1593157868.69436) Sysmeta- Ec-Content- Length: 0 Sysmeta- Ec-Etag: d41d8cd98f00b20 4e9800998ecf842 7e Sysmeta- Ec-Frag- Index: 1 Sysmeta- Ec-Scheme: liberasurecode_ rs_vand 2+2 Sysmeta- Ec-Segment- Size: 1048576 Manifest: %D0%97% D0%B0%D0% BA%D0%B0% D0%B7%D1% 8B_segments/ %D0%A2% D1%80%D0% B5%D0%B1% D0%BE%D0% B2%D0%B0% D0%BD%D0% B8%D1%8F% 20%D0%BE% 20%D0%BF% D0%BE%D0% B3%D0%B0% D1%88%D0% B5%D0%BD% D0%B8%D0% B8%20%D0% B4%D0%BE% D0%BB%D0% B3%D0%B0% 20%28%D0% B8%D1%81% D0%BA%D0% BE%D0%B2% D0%BE%D0% B5%29/28174. pdf/slo/ 1593157868. 688028/ 67108864 4e9800998ecf842 7e (valid) a9dba3978ca49f1 4a
Content-Type: application/pdf
Timestamp: 2020-06-
System Metadata:
X-Object-
X-Object-
X-Object-
X-Object-
X-Object-
Transient System Metadata:
No metadata found
User Metadata:
No metadata found
Other Metadata:
X-Object-
ETag: d41d8cd98f00b20
Content-Length: 0 (valid)
Partition 0
Hash 05ca2b8d9ad708e
Only on one server the object became SLO, and on all other servers the object remained DLO
Why did this happen?