Brick SEGFAULTs in 11.1

[Nick O'Connor]

[ Impact ]

 * Users experience brick SEGFAULTs under certain not-yet-understood scenarios. Some reports include a high percentage of small file I/O. I encountered the issue roughly every hour with Minio backed by GlusterFS on ZFS.

 * This bug introduces an increased risk of data loss or corruption depending on the user's configuration and timing of brick crashes.

 * Core dumps from multiple users revealed that the SEGFAULTs are caused by a stack overflow when namespaced inodes are destroyed.

 * The patch removes the recursive call to inode_unref when a namespaced inode is destroyed.

[ Test Plan ]

 * The triggers for the issue are not well understood. I have compiled the patch against version 11.1-4build3 of glusterfs and it resolved the issue on my system.

[ Where problems could occur ]

 * It's conceivable that this patch introduces undesired behavior when inodes are destroyed, however I highly doubt this scenario as __inode_destroy was not recursive before the change which introduced the bug.

[ Other Info ]

 * PR which introduced the bug: https://github.com/gluster/glusterfs/pull/1763
 * PR which added this patch: https://github.com/gluster/glusterfs/pull/4302
 * Issue discussion: https://github.com/gluster/glusterfs/issues/4295

[Nick O'Connor (nick-oconnor)]

I've recompiled glusterfs locally with the changes. I can confirm the fix linked above addresses the issue.

[Athos Ribeiro (athos-ribeiro)]

Thanks, Nick.

I am adding this to the server team backlog so someone can start working on this one soon.

If you are willing to drive this one, please let us know so we can aid you through the SRU process and sponsor uploads on your behalf in case it is needed.

Otherwise, someone in the server team will start checking/driving this one soon.

[Nick O'Connor (nick-oconnor)]

SGTM. I can drive this. Let me know what needs to be done.

[Nick O'Connor (nick-oconnor)]

This is ready for review/sponsorship.