Reconstruction with liberasurecode_rs_vand can get memory leak

Bug #1665242 reported by Kota Tsuyuzaki
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Object Storage (swift)
Fix Released
Undecided
Unassigned
PyECLib
Fix Released
Undecided
Unassigned
liberasurecode
Fix Released
Undecided
Unassigned

Bug Description

liberasurecode_reconstruct function with liberasurecode_rs_vand (built-in backend) can leak the allocated memory when trying to reconstruct a *parity* fragment.

That was found when we checking all memory leak in the test code at https://review.openstack.org/#/c/434513.
And this is affected also pyeclib and the users (OpenStack Swift is an user of the library)

I confirmed the leak can be reproduced at (at least) PyECLib layer [1] so that object-reconstructor of OpenStack Swift can be affected the leak.

Note that this leak caused at the liberasurecode layer so what both pyeclib and Swift users can do is just upgrading your *liberasurecode* after the fix get merged.

1: https://gist.github.com/bloodeagle40234/bf9eada4ecc41d6516c55fa014c0d78c

summary: - Reconstruction with liberasurecode_rs_vand get memory leak
+ Reconstruction with liberasurecode_rs_vand can get memory leak
description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to liberasurecode (master)

Reviewed: https://review.openstack.org/434162
Committed: https://git.openstack.org/cgit/openstack/liberasurecode/commit/?id=c9136a62b6e7cc8701cd7206ef0367520a20b8b9
Submitter: Jenkins
Branch: master

commit c9136a62b6e7cc8701cd7206ef0367520a20b8b9
Author: Kota Tsuyuzaki <email address hidden>
Date: Wed Feb 15 01:08:25 2017 -0800

    Fix valgrind-check and memory leak

    Can you believe that we ware testing the memory leak with valgrind
    to just *bash scripts* instead of actual binaries on liberasurecode_test
    and libec_slap?

    That is why we cannot find such an easy memory leak[1] at the gate.
    Now this patch enable to run the valgrind against to the binaries.

    With this fix, we found various memory leak at liberasurecode_test as
    follows and this patch also fixes them:

    - If we create fake fragments, we're responsible for freeing all of the
      frags as well as the array holding the pointers to the frags.
    - If we allocate any space, we're responsible for freeing it.
    - If we create an EC descriptor, we're responsible for destroying it.
    - If we create a fragment or skip array, we're responsible for freeing it.
    - If that happens inside a loop, we're responsible for doing it *inside
      that same loop*.

    In addition to the test fix, this patch fixes following memory leaks at
    the code which is affected to other users (pyeclib, OpenStack Swift)

    * Refuse to decode fragments that aren't even long enough to include
      fragment headers.
    * Fix a small memory leak in the builtin rs_vand implementation.

    Closes-Bug: #1665242

    Co-Authored-By: Tim Burke <email address hidden>

    1: https://review.openstack.org/#/c/431812

    Change-Id: I96f124e4e536bbd7544208acc084de1cda5c19b2

Changed in liberasurecode:
status: New → Fix Released
Revision history for this message
Thiago da Silva (thiagodasilva) wrote :

The fix has been released in liberasurecode, setting to released also in pyeclib to close the bug

Changed in pyeclib:
status: New → Fix Released
Revision history for this message
Thiago da Silva (thiagodasilva) wrote :

The fix has been released in liberasurecode, setting to released also in swift to close the bug

Changed in swift:
status: New → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.