Nemo very slow when performing copy of very large numbers of files

Bug #1663186 reported by Robert Whitton
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Linux Mint

Bug Description

Nemo version: nemo 2.2.4.

Running under debian jessie release 8.7.

Copying nearly a million (small) files from local disk to a USB2 attached hard disk (not a memory stick). A total copy of 8.7GiB, really not very large. The copy started quickly but become slower and slower. After nearly a day the rate of copying had dropped to a pathetic ~100KiB/s.

"top" shows that one of the CPU cores is ~100% busy.

I took a profile of nemo (using perf) and it shows:

# To display the header info, please use --header/--header-only options.
# Samples: 12K of event 'cycles'
# Event count (approx.): 11388387380
# Overhead Command Shared Object Symbol
# ........ ....... ............................. ................................................
    98.76% pool [.] g_list_last
     0.07% nemo [.] g_slice_alloc
     0.07% nemo [.] _int_malloc
     0.04% nemo [.] pango_find_paragraph_boundary
     0.03% nemo [.] g_hash_table_lookup
     0.03% nemo [.] g_type_value_table_peek
     0.03% pool [.] malloc
     0.03% nemo [.] g_slice_free_chain_with_offset
     0.03% gmain [.] _IO_no_init
     0.02% pool [.] g_slice_alloc
     0.02% nemo [kernel.kallsyms] [k] msecs_to_jiffies
     0.02% nemo [.] strlen
     0.02% nemo [.] 0x00000000000461f3
     0.02% nemo [.] dbus_connection_get_dispatch_status
     0.02% nemo [.] 0x00000000000189cd
     0.02% nemo [.] 0x00000000000065c8
     0.02% nemo [.] g_type_check_instance_is_a
     0.02% nemo [.] 0x000000000005b2b2
     0.02% nemo [.] malloc
     0.02% nemo [.] g_pointer_bit_lock
     0.02% pool [.] g_hash_table_lookup_extended

98.76% of the time we are in g_list_last? That doesn't sound good. If I had to guess I would suggest the we have some computation here that's O(N^X) where X is > 0 and N is the number of files already copied. So it gets slower and slower as the copy progresses.

Let's look at the implementation of libglib... The implementation of g_list_last is truly appalling as the list gets longer...

  if (list)
      while (list->next)
       list = list->next;

  return list;

i.e. it appears to walk the list from the head until it encounters the last element.

I'd strongly suggest that nemo avoids direct or indirect use of g_list_last since this is likely to offer very poor performance when the list becomes long.

Revision history for this message
Robert Whitton (rwhitton) wrote :
description: updated
Changed in linuxmint:
status: New → Incomplete
status: Incomplete → New
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Bug attachments