Comment 6 for bug 1581334

Revision history for this message
chenwqin (chenwqin) wrote :

Here are 'perf top -p `pgrep qemu` -a` output; I met tcmalloc problem on the osd host and fix it with a larger TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES. The perf top of tcmalloc problem is a little bit different of my problem.
------------------------------------------------------------------
Samples: 15M of event 'cycles', Event count (approx.): 463535858808
  87.68% [kernel] [k] _raw_spin_lock
   1.02% [kernel] [k] futex_wait_setup
   0.66% [kernel] [k] futex_wake
   0.47% libc-2.19.so [.] malloc
   0.42% [kernel] [k] try_to_wake_up
   0.22% [kernel] [k] __unqueue_futex
   0.22% [kernel] [k] _raw_spin_lock_irq
   0.18% libpthread-2.19.so [.] pthread_mutex_lock
   0.18% libc-2.19.so [.] 0x00000000000f4dff
   0.17% libc-2.19.so [.] 0x00000000000ea96f
   0.15% [kernel] [k] idle_cpu
   0.15% libpthread-2.19.so [.] __pthread_mutex_unlock_usercnt
   0.14% [kernel] [k] get_futex_value_locked
   0.13% [kernel] [k] get_futex_key_refs.isra.13
   0.13% libc-2.19.so [.] free
   0.12% [kernel] [k] _raw_spin_lock_irqsave
   0.11% [kernel] [k] update_curr
   0.11% [kernel] [k] select_task_rq_fair
   0.10% [kernel] [k] native_write_msr_safe
   0.10% [kernel] [k] futex_wait_queue_me
   0.10% [kernel] [k] __schedule
   0.09% [kernel] [k] wake_futex
Press '?' for help on key bindings
----------------------------------------------------------------

Tcmalloc problem perf top (log from ceph bug tracer email, same with tcmalloc problem i met)
-------------------------------------------------------------
34.37% libtcmalloc.so.4.1.2 [.] tcmalloc::CentralFreeList::FetchFromSpans
  18.06% libtcmalloc.so.4.1.2 [.] tcmalloc::ThreadCache::ReleaseToCentralCache
  13.76% libtcmalloc.so.4.1.2 [.] tcmalloc::CentralFreeList::ReleaseToSpans
   1.45% libtcmalloc.so.4.1.2 [.] tcmalloc::CentralFreeList::RemoveRange
-------------------------------------------------------------