Activity log for bug #1774336

Date Who What changed Old value New value Message
2018-05-31 06:04:08 Daniel Axtens bug added bug
2018-05-31 06:04:14 Daniel Axtens linux (Ubuntu): status New Confirmed
2018-05-31 06:04:17 Daniel Axtens linux (Ubuntu): assignee Daniel Axtens (daxtens)
2018-06-04 16:58:36 Sam bug added subscriber Sam
2018-06-07 15:55:19 Khaled El Mously nominated for series Ubuntu Trusty
2018-06-07 15:55:19 Khaled El Mously nominated for series Ubuntu Artful
2018-06-07 15:55:19 Khaled El Mously nominated for series Ubuntu Bionic
2018-06-07 15:55:19 Khaled El Mously nominated for series Ubuntu Xenial
2018-06-07 16:07:08 Kleber Sacilotto de Souza bug task added linux (Ubuntu Artful)
2018-06-07 16:07:15 Kleber Sacilotto de Souza bug task added linux (Ubuntu Bionic)
2018-06-07 16:07:21 Kleber Sacilotto de Souza bug task added linux (Ubuntu Trusty)
2018-06-07 16:07:27 Kleber Sacilotto de Souza bug task added linux (Ubuntu Xenial)
2018-06-12 10:05:36 Kleber Sacilotto de Souza linux (Ubuntu Trusty): status New Fix Committed
2018-06-12 10:05:38 Kleber Sacilotto de Souza linux (Ubuntu Xenial): status New Fix Committed
2018-06-12 10:05:40 Kleber Sacilotto de Souza linux (Ubuntu Artful): status New Fix Committed
2018-06-12 10:05:42 Kleber Sacilotto de Souza linux (Ubuntu Bionic): status New Fix Committed
2018-06-13 07:01:37 Brad Figg tags verification-needed-artful
2018-06-13 11:04:01 Brad Figg tags verification-needed-artful verification-needed-artful verification-needed-xenial
2018-06-14 11:02:12 Brad Figg tags verification-needed-artful verification-needed-xenial verification-needed-artful verification-needed-bionic verification-needed-xenial
2018-06-15 10:33:34 Brad Figg tags verification-needed-artful verification-needed-bionic verification-needed-xenial verification-needed-artful verification-needed-bionic verification-needed-trusty verification-needed-xenial
2018-06-19 17:24:42 David Coronel tags verification-needed-artful verification-needed-bionic verification-needed-trusty verification-needed-xenial verification-done-xenial verification-needed-artful verification-needed-bionic verification-needed-trusty
2018-07-02 08:27:17 Launchpad Janitor linux (Ubuntu Artful): status Fix Committed Fix Released
2018-07-02 08:27:17 Launchpad Janitor cve linked 2018-1130
2018-07-02 08:27:17 Launchpad Janitor cve linked 2018-11508
2018-07-02 08:27:17 Launchpad Janitor cve linked 2018-5750
2018-07-02 08:27:17 Launchpad Janitor cve linked 2018-5803
2018-07-02 08:27:17 Launchpad Janitor cve linked 2018-6927
2018-07-02 08:27:17 Launchpad Janitor cve linked 2018-7755
2018-07-02 08:27:17 Launchpad Janitor cve linked 2018-7757
2018-07-02 08:28:24 Launchpad Janitor linux (Ubuntu Trusty): status Fix Committed Fix Released
2018-07-02 08:28:24 Launchpad Janitor cve linked 2017-12154
2018-07-02 08:28:24 Launchpad Janitor cve linked 2017-12193
2018-07-02 08:28:24 Launchpad Janitor cve linked 2017-15265
2018-07-02 08:28:24 Launchpad Janitor cve linked 2018-3665
2018-07-02 08:27:55 Launchpad Janitor linux (Ubuntu Bionic): status Fix Committed Fix Released
2018-07-02 08:29:08 Launchpad Janitor linux (Ubuntu Xenial): status Fix Committed Fix Released
2018-07-02 08:29:08 Launchpad Janitor cve linked 2018-3639
2018-07-26 05:13:52 Launchpad Janitor linux (Ubuntu): status Confirmed Fix Released
2018-08-02 03:56:38 Daniel Axtens description == SRU Justification == [Impact] Oops during heavy NFS + FSCache use: [81738.886634] FS-Cache: [81738.888281] FS-Cache: Assertion failed [81738.889461] FS-Cache: 6 == 5 is false [81738.890625] ------------[ cut here ]------------ [81738.891706] kernel BUG at /build/linux-hVVhWi/linux-4.4.0/fs/fscache/operation.c:494! 6 == 5 represents an operation being DEAD when it was not expected to be. [Cause] There is a race in fscache and cachefiles. One thread is in cachefiles_read_waiter: 1) object->work_lock is taken. 2) the operation is added to the to_do list. 3) the work lock is dropped. 4) fscache_enqueue_retrieval is called, which takes a reference. Another thread is in cachefiles_read_copier: 1) object->work_lock is taken 2) an item is popped off the to_do list. 3) object->work_lock is dropped. 4) some processing is done on the item, and fscache_put_retrieval() is called, dropping a reference. Now if the this process in cachefiles_read_copier takes place *between* steps 3 and 4 in cachefiles_read_waiter, a reference will be dropped before it is taken, which leads to the objects reference count hitting zero, which leads to lifecycle events for the object happening too soon, leading to the assertion failure later on. (This is simplified and clarified from the original upstream analysis for this patch at https://www.redhat.com/archives/linux-cachefs/2018-February/msg00001.html and from a similar patch with a different approach to fixing the bug at https://www.redhat.com/archives/linux-cachefs/2017-June/msg00002.html) [Fix] Move fscache_enqueue_retrieval under the lock in cachefiles_read_waiter. This means that the object cannot be popped off the to_do list until it is in a fully consistent state with the reference taken. [Testcase] A user has run ~100 hours of NFS stress tests and not seen this bug recur. [Regression Potential] - Limited to fscache/cachefiles. - The change makes things more conservative (doing more under lock) so that's reassuring. - There may be performance impacts but none have been observed so far. == SRU Justification == [Impact] Oops during heavy NFS + FSCache use: [81738.886634] FS-Cache: [81738.888281] FS-Cache: Assertion failed [81738.889461] FS-Cache: 6 == 5 is false [81738.890625] ------------[ cut here ]------------ [81738.891706] kernel BUG at /build/linux-hVVhWi/linux-4.4.0/fs/fscache/operation.c:494! 6 == 5 represents an operation being DEAD when it was not expected to be. [Cause] There is a race in fscache and cachefiles. One thread is in cachefiles_read_waiter:  1) object->work_lock is taken.  2) the operation is added to the to_do list.  3) the work lock is dropped.  4) fscache_enqueue_retrieval is called, which takes a reference. Another thread is in cachefiles_read_copier:  1) object->work_lock is taken  2) an item is popped off the to_do list.  3) object->work_lock is dropped.  4) some processing is done on the item, and fscache_put_retrieval() is called, dropping a reference. Now if the this process in cachefiles_read_copier takes place *between* steps 3 and 4 in cachefiles_read_waiter, a reference will be dropped before it is taken, which leads to the objects reference count hitting zero, which leads to lifecycle events for the object happening too soon, leading to the assertion failure later on. (This is simplified and clarified from the original upstream analysis for this patch at https://www.redhat.com/archives/linux-cachefs/2018-February/msg00001.html and from a similar patch with a different approach to fixing the bug at https://www.redhat.com/archives/linux-cachefs/2017-June/msg00002.html) [Fix] (Old sauce patch being reverted) Move fscache_enqueue_retrieval under the lock in cachefiles_read_waiter. This means that the object cannot be popped off the to_do list until it is in a fully consistent state with the reference taken. (New upstream patch) Explicitly take a reference to the object while it is being enqueued. Adjust another part of the code to deal with the greater range of object states this exposes. [Testcase] A user has run ~100 hours of NFS stress tests and not seen this bug recur. [Regression Potential]  - Limited to fscache/cachefiles.  - The change makes things more conservative (taking more references) so that's reassuring.  - There may be performance impacts but none have been observed so far.
2018-08-28 14:59:59 David Coronel bug added subscriber David Coronel
2019-02-14 15:52:08 Andy Whitcroft tags verification-done-xenial verification-needed-artful verification-needed-bionic verification-needed-trusty kernel-fixup-verification-needed-bionic verification-done-xenial verification-needed-artful verification-needed-trusty
2019-02-14 16:09:25 Brad Figg tags kernel-fixup-verification-needed-bionic verification-done-xenial verification-needed-artful verification-needed-trusty kernel-fixup-verification-needed-bionic verification-done-xenial verification-needed-artful verification-needed-bionic verification-needed-trusty
2019-02-14 16:12:08 Andy Whitcroft tags kernel-fixup-verification-needed-bionic verification-done-xenial verification-needed-artful verification-needed-bionic verification-needed-trusty kernel-fixup-verification-needed-bionic verification-done-xenial verification-needed-artful verification-needed-trusty
2019-02-14 16:17:50 Andy Whitcroft tags kernel-fixup-verification-needed-bionic verification-done-xenial verification-needed-artful verification-needed-trusty kernel-fixup-verification-needed-bionic verification-done-bionic verification-done-xenial verification-needed-artful verification-needed-trusty
2019-07-24 21:24:44 Brad Figg tags kernel-fixup-verification-needed-bionic verification-done-bionic verification-done-xenial verification-needed-artful verification-needed-trusty cscc kernel-fixup-verification-needed-bionic verification-done-bionic verification-done-xenial verification-needed-artful verification-needed-trusty