Comment 5 for bug 1567557

Revision history for this message
Colin Ian King (colin-king) wrote : Re: Performance degradation of "zfs clone" when under load

I've created 4096 zfs clones using the same commands that lxd is using and I have observed that the biggest pinch point is when fetching the zfs clone stats as this has to lock using dsl_dataset_hold_obj, fetch the data and then unlock with dsl_dataset_rele. Traversing hundreds of clones is linear and slow, taking ~1 second with ~4000+ clones.

+ 99.22% 0.00% zfs [kernel.kallsyms] [k] entry_SYSCALL_64_fastpath
- 99.14% 0.00% zfs [kernel.kallsyms] [k] zfsdev_ioctl
   - zfsdev_ioctl
      - 99.08% zfs_ioc_objset_stats
         - 98.87% zfs_ioc_objset_stats_impl.part.20
            - 98.81% dmu_objset_stats
               - 98.81% dsl_dataset_stats
                  - 98.76% get_clones_stat
                     - 63.56% fnvlist_add_nvlist
                          nvlist_add_nvlist
                          nvlist_add_common.part.51
                          nvlist_copy_embedded.isra.54
                        + nvlist_copy_pairs.isra.52
                     + 32.74% fnvlist_add_boolean
                     + 1.81% dsl_dataset_hold_obj