EC sysmeta can show up on replicated data

Bug #1885857 reported by Tim Burke
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Object Storage (swift)
Fix Released
Low
Unassigned

Bug Description

Apparently we don't strip EC sysmeta when copying data between policies. I'm *pretty sure* if you're going between two different EC policies, it'll be fine (surely we would've heard about that bug by now otherwise, right?). Going from EC to replicated still *works* (since the replicated data path won't look at the headers), but it leads to some head-scratchers when debugging:

vagrant@saio:~/swift$ swift post c -H x-storage-policy:ec
vagrant@saio:~/swift$ swift upload c tox.ini
tox.ini
vagrant@saio:~/swift$ swift post c2 -H x-storage-policy:replicated
vagrant@saio:~/swift$ swift copy c tox.ini --destination /c2
created container c2
c/tox.ini copied to /c2/tox.ini
vagrant@saio:~/swift$ swift-get-nodes /etc/swift/object-1.ring.gz AUTH_test/c2/tox.ini | grep curl
curl -g -I -XHEAD "http://127.0.0.3:6036/sdb3/236/AUTH_test/c2/tox.ini" -H "X-Backend-Storage-Policy-Index: 1"
curl -g -I -XHEAD "http://127.0.0.1:6016/sdb1/236/AUTH_test/c2/tox.ini" -H "X-Backend-Storage-Policy-Index: 1"
curl -g -I -XHEAD "http://127.0.0.4:6046/sdb4/236/AUTH_test/c2/tox.ini" -H "X-Backend-Storage-Policy-Index: 1"
curl -g -I -XHEAD "http://127.0.0.2:6026/sdb2/236/AUTH_test/c2/tox.ini" -H "X-Backend-Storage-Policy-Index: 1" # [Handoff]
vagrant@saio:~/swift$ curl -g -I -XHEAD "http://127.0.0.3:6036/sdb3/236/AUTH_test/c2/tox.ini" -H "X-Backend-Storage-Policy-Index: 1"
HTTP/1.1 200 OK
Content-Type: application/octet-stream
X-Object-Sysmeta-Container-Update-Override-Etag: 0XqAb6vL1glzuefZ9COiEjByMg2Z2IVYNgHCMLV4+5c=; swift_meta=%7B%22cipher%22%3A+%22AES_CTR_256%22%2C+%22iv%22%3A+%22KsLvGHOR4YmK89uKQQ785A%3D%3D%22%2C+%22key_id%22%3A+%7B%22path%22%3A+%22%2FAUTH_test%2Fc2%2Ftox.ini%22%2C+%22secret_id%22%3A+%222018%22%2C+%22v%22%3A+%222%22%7D%7D
X-Object-Sysmeta-Ec-Etag: 1d2d2a97ac44b3b0a26b5379f427af69
X-Object-Sysmeta-Ec-Content-Length: 4788
X-Object-Sysmeta-Ec-Frag-Index: 4
X-Object-Sysmeta-Ec-Scheme: liberasurecode_rs_vand 4+2
X-Object-Sysmeta-Ec-Segment-Size: 1048576
X-Object-Transient-Sysmeta-Crypto-Meta-Mtime: St6CSFvUClCuZRBBld5Ynik=; swift_meta=%7B%22cipher%22%3A+%22AES_CTR_256%22%2C+%22iv%22%3A+%226SObRXmr8tWSSF6%2FobRFxg%3D%3D%22%7D
X-Object-Transient-Sysmeta-Crypto-Meta: %7B%22cipher%22%3A+%22AES_CTR_256%22%2C+%22key_id%22%3A+%7B%22path%22%3A+%22%2FAUTH_test%2Fc2%2Ftox.ini%22%2C+%22secret_id%22%3A+%222018%22%2C+%22v%22%3A+%222%22%7D%7D
X-Object-Sysmeta-Crypto-Etag: FgnZOAjxo2Rb0DIBbVeEX25LcrspMDW0/WHBk8NqKFM=; swift_meta=%7B%22cipher%22%3A+%22AES_CTR_256%22%2C+%22iv%22%3A+%22q02K12yOKR1tAVkitnD4pQ%3D%3D%22%7D
X-Object-Sysmeta-Crypto-Body-Meta: %7B%22body_key%22%3A+%7B%22iv%22%3A+%22eLyfKWyPM7ghVpTrBXtJJw%3D%3D%22%2C+%22key%22%3A+%22L8Vb1%2B2w4GjEXljd3kvitUOkZBMJt9%2BTQIWFlJdSN5s%3D%22%7D%2C+%22cipher%22%3A+%22AES_CTR_256%22%2C+%22iv%22%3A+%22KTHSdioYOinQTzoaMkIpDw%3D%3D%22%2C+%22key_id%22%3A+%7B%22path%22%3A+%22%2FAUTH_test%2Fc2%2Ftox.ini%22%2C+%22secret_id%22%3A+%222018%22%2C+%22v%22%3A+%222%22%7D%7D
X-Object-Sysmeta-Crypto-Etag-Mac: j7a0gL63t8MxcSlBx1+GJ+7O1zmxwiLFCQHW1fqsJLU=
Etag: "236f1845de751090ce30b740081a5c4b"
Last-Modified: Wed, 01 Jul 2020 04:57:02 GMT
X-Timestamp: 1593579421.76668
X-Backend-Timestamp: 1593579421.76668
X-Backend-Data-Timestamp: 1593579421.76668
X-Backend-Durable-Timestamp: 1593579421.76668
Content-Length: 4788
Date: Wed, 01 Jul 2020 04:57:56 GMT

Note how Content-Length and X-Object-Sysmeta-Ec-Content-Length match *exactly* -- that should basically *never* happen for real EC data.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to swift (master)

Reviewed: https://review.opendev.org/c/openstack/swift/+/783981
Committed: https://opendev.org/openstack/swift/commit/399bda8a410893b788bfc439d227b772a525c095
Submitter: "Zuul (22348)"
Branch: master

commit 399bda8a410893b788bfc439d227b772a525c095
Author: Tim Burke <email address hidden>
Date: Tue Mar 30 15:25:13 2021 -0700

    ec: Don't copy EC metadata to replicated objects

    Change-Id: Ia6c1b75a24d2b46720d7a3b6e3a126444a38be0e
    Closes-Bug: #1885857

Changed in swift:
status: New → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/swift 2.28.0

This issue was fixed in the openstack/swift 2.28.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.