exaile crashes when rescanning library location mounted using 'sshfs'

[Damien Dixon]

I have been unable to rescan my library using the SVN/BZR releases because it crashes mid way through with no real explanation of the problem. However, I did a scan without my sshfs mount and it ran smoothly. I have deduced that the error is in the fact that SSHFS is mounting a directory but cannot be sure. Ideally I need intervention from someone more skilled to help me get more information on what is doing on during the scan and what actually kills Exaile off.

[Johannes Sasongko (sjohannes)]

Could you attach the console output when this happens?

[Ben Beasley (ben-musicinmybrain)]

I just tried this with the latest BZR version. My steps:

1. Open Exaile (by running ./exaile) with a blank playlist.
2. Add either the mountpoint of the sshfs mount, or a subdirectory to the initially-empty collection. So I would add either /home/ben/remote-music or /home/ben/remote-music/some-directory/, for example.
3. Wait for the automatic rescan.

I couldn’t reproduce the crash scanning /home/ben/remote-music/some-directory/, but it’s possible that’s just because I wasn’t scanning large enough a collection or wasn’t scanning a particular file that triggered the problem when I did that.

I could reproduce this when scanning the root mountpoint (which corresponds as well to a larger collection). At 86% done, the progress ceased. The GUI does not freeze, and the scan can still be canceled by clicking the red X stop button (although it can’t subsequently be rerun without restarting Exaile). However, the following traceback appears in the console:

INFO : Compilation detected: (u'/home/ben/remote-music/music-library/Ben/Uncommon Ritual', u'uncommon ritual')
Exception in thread Thread-9:
Traceback (most recent call last):
  File "/usr/lib/python2.5/threading.py", line 486, in __bootstrap_inner
    self.run()
  File "/home/ben/sources/exaile/xlgui/collection.py", line 62, in run
    self.collection.rescan_libraries()
  File "/home/ben/sources/exaile/xl/collection.py", line 147, in rescan_libraries
    library.rescan(notify_interval=scan_interval)
  File "/home/ben/sources/exaile/xl/collection.py", line 528, in rescan
    items = db.search('basedir="%s" album="%s"' % (basedir, album))
  File "/home/ben/sources/exaile/xl/trackdb.py", line 327, in search
    tracksres = searcher.search(query, tracks)
  File "/home/ben/sources/exaile/xl/trackdb.py", line 578, in search
    tokens = self.tokenize_query(query)
  File "/home/ben/sources/exaile/xl/trackdb.py", line 462, in tokenize_query
    tk += " " + tokens[counter+1]
IndexError: list index out of range

I don't know of anything unusual about the file names or tag contents of Uncommon Ritual or any of the immediately alphabetically succeeding albums.

After capturing this traceback, I restarted exaile and tried rescanning. This time the scan completed very quickly and apparently successfully. I did get an error here:

/home/ben/sources/exaile/xlgui/progress.py:142: GtkWarning: gtk_container_remove: assertion `GTK_IS_TOOLBAR (container) || widget->parent == GTK_WIDGET (container)' failed
  self.box.remove(monitor)

That seems to be a separate issue, though, and is not exclusive to using sshfs. Same with the mangling of certain special characters, where, for example, Evocación became Evocaci鏮, Albéniz became Alb幯iz, and Pärt became P酺t.

I guess the next thing for me to try would be to edit the line:

    tk += " " + tokens[counter+1]

so that when there is an exception, relevant information like the contents of counter and token is printed to the console. This could be a little bit difficult since the crash doesn’t seem to happen every time.

[Ben Beasley (ben-musicinmybrain)]

Actually, the crash does seem to be fully reproducible if I clear user data directories (/home/ben/.local/share/exaile, and /home/ben/.exaile for good measure) between attempts. I used WinPDB to capture some additional debugging information. Here are the locals (with the most relevant ones expanded):

 Name Type Repr
 c unicode u''
 counter int 11
v etokens list [u'basedir=/home/ben/geezers-music/music-library/ben/beach: "gaelic', u'symphony', u'-', u'piano', u'concerto album=beach:', u'gaelic', u'symphony', u'-', u'piano']
  0 unicode u'basedir=/home/ben/geezers-music/music-library/ben/beach: "gaelic'
  1 unicode u'symphony'
  2 unicode u'-'
  3 unicode u'piano'
  4 unicode u'concerto album=beach:'
  5 unicode u'gaelic'
  6 unicode u'symphony'
  7 unicode u'-'
  8 unicode u'piano'
 first int 0
 in_quotes bool False
 last int 7
 n int 145
 newsearch unicode u'basedir="/home/ben/geezers-music/music-library/Ben/Beach: "Gaelic" Symphony - Piano Concerto" album="beach: "gaelic" symphony - ...'
 search unicode u'basedir="/home/ben/geezers-music/music-library/ben/beach: "gaelic" symphony - piano concerto" album="beach: "gaelic" symphony - ...'
> self xl.trackdb.TrackSearcher <xl.trackdb.TrackSearcher object at 0x3d13b10>
 t unicode u''
 tk unicode u'concerto"'
v etokens list [u'basedir="/home/ben/geezers-music/music-library/ben/beach:', u'"gaelic"', u'symphony', u'-', u'piano', u'concerto', u'album=be...'...]
  0 unicode u'basedir=/home/ben/geezers-music/music-library/ben/beach:'
  1 unicode u'"gaelic"'
  2 unicode u'symphony'
  3 unicode u'-'
  4 unicode u'piano'
  5 unicode u'concerto'
  6 unicode u'album="beach:'
  7 unicode u'"gaelic"'
  8 unicode u'symphony'
  9 unicode u'-'
  10 unicode u'piano'
  11 unicode u'concerto"'

[Ben Beasley (ben-musicinmybrain)]

I've marked https://bugs.launchpad.net/exaile/+bug/311141 as a duplicate, although it shows an additional exception as well as the one I've recorded here.

[reacocard (reacocard)]

Ok, the problem appears to be twofold:
- the tokenizer doesn't know how to deal with odd numbers of "s
- the scanner isn't escaping "s properly

I've committed fixes for both of these in r1755, please test and report whether this resolves the issue.

[Ben Beasley (ben-musicinmybrain)]

I checked out the latest BZR version and couldn’t reproduce the bug in three attempts. Unless someone else speaks up, I’d call this one fixed.

[Andy (dr-diesel)]

0.3.0a1 appears to have fixed https://bugs.launchpad.net/exaile/+bug/311141 for me, thanks!