MTP device support

[Adam Olsen]

I recently bought a Creative Zen, an [http://en.wikipedia.org/wiki/Media_Transfer_Protocol MTP] device, so now I need a plugin for my favourite media player.

I've knocked up a plugin derived from plugin.DeviceDriver that fetches the track list from the device and displays it in Exaile. No useful functionality so far, but it's a start.

I actually wrapped libmtp using ctypes to write the plugin (I created a C-like API and a slightly higher-level, more Pythonic API) before finding [https://launchpad.net/pylibmtp/ PylibMTP]. I'll probably switch to PylibMTP but I don't know what it's like yet.

Here's a quick brain dump of what I would ''like'' the plugin to do:

   * Tag albums, artists or tracks for transfer to the Zen, even when the Zen is not connected.

   * Queue transfers and copy in the background. Start transfer immediately (device connected), when device connects (after prompt) or manually after a "Transfer" button click on the device's panel.

   * Remove albums, artists or tracks from device.

   * Transfer album art.

And a list of things I think would be nice:

   * Manage playlists.

   * Display available space on device.

   * Re-sync album art, i.e. after running the "Album Art Collector".

   * Transfer ratings. (I don't personally use ratings but I'm sure some do.)

Ideally the plugin should expect multiple MTP devices to track transfers correctly but handling the first device is probably good enough for the vast majority of people. This implies devices may need registering (devices have a serial number and a nice name) before they can be used.

If multiple devices are supported the UI should simplify itself if Exaile only knows about one device.

I'll post progress to this ticket as and when I get the chance to work on the plugin.

This ticket was migrated from the old trac: re #489

[Dan O'Reilly (oreilldf)]

Hey all,

I've done a little more work on this, using version 0.2 of PyMTP (not the same pymtp already uploaded here, this is a different project with the same name) that Nick Devito released last month. I updated transferring tracks to work with the new version of pymtp, improved the way tracks get tagged on the MTP device, added support for removing tracks, and added/changed a few other things as well. Error handling is still fairly minimal, but should be easier to implement with the new pymtp.

Also, The track display is a little screwy (viewing by Artist shows a bunch of phantom entries that change when depending on how you mouse over them and what parts of the tree are exanded, Album view is empty, and Genre view only shows one Unknown genre), but I'll try to get that straightened out. Obviously there's a whole lot of other stuff that needs to be added/fixed as well, but I thought I'd share what I have so others can comment or make additions themselves. Matt (Goodall, the original creator of this bug and author of the original exaile MTP plugin, I think?), if you're still out there working on it, let me know, we can touch base and pool resources if you're not too much further along in development.

Link for pymtp 0.2: http://nick125.com/projects/pymtp/
mtpdevice.py is attached, put in your ~/.exaile/plugins directory.

[Dan O'Reilly (oreilldf)]

I've figured out the problem with the phantom entries. In devices.py, the self.tree that overwrites the one inited during the ConnectionPanel.__init__() never calls set_row_separator_func(), so when self.model.append(parent, [None, None, None]) gets called, a null entry is getting added to the panel instead of a separator. Copying the set_row_separator_func() call from collection.py to device.py solves the problem. I attached a patch, if I should file a separate bug for it let me know.

[Dan O'Reilly (oreilldf)]

It just occurred to me that someone writing a device plugin can make this call within the plugin itself, but it's not really an obvious addition to make. It takes some digging around in device.py and collection.py to figure out what's going wrong. It seems like the simpler and more intuitive solution is to add it to device.py.

[Dan O'Reilly (oreilldf)]

Here's another updated version, it fixed some problems with removing tracks by making sure the loc variable was set for all the tracks, added searching and sorting support by album/artist/genre, and fixes the problem with separators not displaying properly. Error handling is also improved.

Next on the todo list is playlist support, and adding tracks in a way that matches the hierarchical structure of the users collection. So if your collection is ~/my music/, and you add a track in the ~/my music/Talking Heads/Speaking in Tongues/ directory, the track will get added to the /Talking Heads/Speaking in Tongues/ directory on the MTP player. Right now it just adds all the files to the root directory. I'd also like to add an option to transfer a track from the device to your collection, as well as add album art support.

[Dan O'Reilly (oreilldf)]

Here's a small update, it fixes a few bugs and adds some better error handling. I'll probably start work on playlist support in the next couple of weeks.

[Nic (ntetreau)]

Thanks for your hard work on the mtp plugin. So far it works great. What I found so far is one cannot transfer music from the device to the collection (no feedback given but doesn't seem to happen) and no playback seems possible from the device. Also, Exaile should use hal to discover that you plugged in the device and load it automatically. This way, one wouldn't have to click on Device, then MTPdevice, and then connect (hal detection is enabled in the rhythmbox plugin). This could be configured in the plugin in case you wouldn't want this to be default. Also, there could be a way that transfers to the device are done automatically instead of going through the transfer queue. Thanks again, your work is greatly appreciated!!!

[Nic (ntetreau)]

I forgot to add that the music should normally be transfered to the device into the Music folder. Then, to keep things clean it would be better to create a folder for the artist and then the Album.

[Dan O'Reilly (oreilldf)]

Thanks for the feedback, Nic. Support for transfering from the device to your collection is on its way, I just need to clear out some time in my schedule to start working on it. Same goes for transfering the device to the Music folder (although that might get held back by some limitations in pymtp, I have to test some things). I'll look into HAL detection as well.

I was under the impression from amarok and gnomad2 that it wasn't playback from the device wasn't possible, so I'll be interested to take a peek and Rhythmbox's code and see what approach they take.

[Nic (ntetreau)]

Hi Dan, I'd be happy to test any new development on the mtp plugin. As for hal and playback from the device support, you can take a look at the implementation in rhythmbox: http://bugzilla.gnome.org/attachment.cgi?id=89471

[Dan O'Reilly (oreilldf)]

Time for another update. Attached is a patch that adds support for playing tracks from the device (by dragging them into the playlist, it can be slow if you drag a bunch of tracks at once, so be patient), as well as support for viewing playlists (no creating, editing, deleting, etc. yet). I actually have support for transferring files to Artist/Album directories on the device ready too, but the maintainer of pymtp hasn't implemented a get_folder_list function yet, which is needed to do it. I wrote my own function for it, and talked with the pymtp maintainer about integrating it, but he wanted to hold off and do something different with it (not exactly sure what). So for now I'll just have to hold off on adding it. Next up is replacing the default right-click menu with one that only shows the options that you're actually allowed to use with the plugin, and completing playlist support.

Adam/Johannes: Also included with this is a couple of patches for xl/panels/device.py and collection.py, which make some small changes to how it handles picking a name for a particular node in the TreeModel, and for how Exaile retrieves a track when one is dragged from the panel into the playlist. The patch also assumes you haven't applied the patch I submitted for bug #172680[1].

[1]https://bugs.launchpad.net/exaile/+bug/172680

[Olric86 (djcaco3)]

Hi,
Could someone please post again the pymtp files ? This link <a href="http://nick125.com/projects/pymtp/">http://nick125.com/projects/pymtp/</a> does not work anymore and I'd like to try this plugin :D:D
Thanks !!

[Dan O'Reilly (oreilldf)]

I'll have to track down Nick and see if there's a new link, the one I found on his site is dead. Here's a temporary link for now: http://www2.bc.edu/~oreilldf/pymtp-0.0.2.tar.bz2

[spitfire (mieszkoslusarczyk)]

This is the new link to PyMTP on nick's 125 page.
http://downloads.nick125.com/projects/pymtp/pymtp-latest.tar.bz2

[junior (olavjunior)]

Looking forward to the album art support :) Thanks for a nice plugin!

[Dan O'Reilly (oreilldf)]

Here's a large update to the plugin. The plugin now uses custom right-click menus that only show actions you can actually take, rather than just using the default exaile menus. Support for editing, creating, and deleting playlists is there as well. Finally, there is the added option to choose to store tracks added to the device in either the default music folder, or to store them in the "Artist/Album" directory, based on what those values are in each track's tags.

In order to add some of these features I had to make some changes to pymtp which are not in the version you cant get from nick125.com. For now I'm hosting the custom version on the webspace I get from my school, but it'd be nice to have a more permanent place to keep it. You can get the new pymtp here: http://www2.bc.edu/~oreilldf/pymtp/pymtp.html.

I've been using this version for quite a while and haven't noticed any major bugs, but please let me know if you do. The patch is attached.

[Adam Olsen (arolsen)]

I could give you an account so you can host it on exaile.org if you want. Let me know.

[junior (olavjunior)]

Ehm... how do I apply the patch? (feeling stupid)

[Dan O'Reilly (oreilldf)]

Adam: Yeah, that'd be great.

junior: if you put the .patch file in the directory of mtpdevice.py (should be ~/.exaile/plugins), then run:
patch -i <patchfile>
I think that should work. If you have trouble let me know and I'll just upload the whole file for you.

Also, I noticed I forgot to bump the version number up, so I attached another patch that adds that.

[junior (olavjunior)]

No, sorry, I don't get it. I'm using exail-bzr. If I'm in .exaile/plugins and run the patch, the mtpdevice.py doesn't change. If I apply the patch at exaile-bzr it changes. But when I choose install/uninstall in exaile, it's always version 0.4.0 showing up no matter what :(

[Dan O'Reilly (oreilldf)]

After you apply the patch at exaile-bzr, copy the new mtpdevice.py file into ~/.exaile/plugins. That should make it work.

One last patch, with the updated link to my version of pymtp (Thanks to Adam for giving somewhere to host it).

[huvber (huvber)]

I try to use these patchs when i type patch -i <patch file> it wrote

patching file mtpdevice.py
Hunk #1 FAILED at 17.
Hunk #2 FAILED at 51.
Hunk #3 FAILED at 87.
Hunk #4 FAILED at 115.
Hunk #5 FAILED at 137.
Hunk #6 FAILED at 196.
Hunk #7 FAILED at 449.
Hunk #8 FAILED at 488.
Hunk #9 FAILED at 695.
Hunk #10 FAILED at 705.
10 out of 10 hunks FAILED -- saving rejects to file mtpdevice.py.rej

 ... exaile found my zen but don't download the directories and the songs and i am unable tu upload another songs

[Benjamin Horstman (yodenss)]

First, this plugin is sexy. It's about time my gogear gets supported properly somewhere (WMP broke compatibility a while ago).

However, the plugin seems to have issues transferring odd characters (e.g. Æ); it just stops on the offending file. Not sure where in this ridiculous stack the problem is. Not sure if you can get the usable character set for the device anywhere either...

My workaround is to try to decode the string in latin-1, and then catch the exception for the strings containing funny characters. Leaving metadata.album undefined apparently causes something else to be used as the album; in my case it sends the whole string. The offending character renders as a Box on the device; gnome seems to thing it is a kanji of some sort when it is displayed in the device panel (I have japanese text support enabled...). Note that exaile needs restarted to see this change. At any rate, the transfer doesn't hang.

[Dan O'Reilly (oreilldf)]

Hey Benjamin,

Glad the plugin is working out for you, and thanks for the patch. I'm really busy with finals and stuff right now so I may not have a chance to test it out until sometime next week. I'll also try to figure out a more elegant solution if possible.

Also, I know that the developer of the original pymtp released an update a little while ago that implemented some of the stuff in my custom version, so if I can get it all working I might be able to switch back to that. I briefly tested it a while ago and ran into some problems, but it may have been a libmtp issue.

[junior (olav-ekkje)]

Didn't install bzr in Hardy, and finally the patch works fine :) Love it! It would be nice if it also could connect nokia n95 in media-player mode (on the n95). The plugin is just "loading tracks" forever, and when I click disconnect, exaile crashes. But I don't know if nokia supports the mtp standard?

[spitfire (mieszkoslusarczyk)]

And what about album art support?

[barni (kbarna)]

Hi,
I have a bug with the MTP plugin. While it loads the contents of my MP3 player, Exaile quits.
In the console I have:

PTP: Opening session
Segmentation fault

I'm using Exaile 0.2.14, MTP plugin ver. 0.4.0 under Ubuntu 8.10 64bits. I'm using libmtp8 0.3.0 and python-pymtp 0.0.4
My MP3 player is a Sandisk Sansa Express, and it works perfectly with Rhythmbox.

[Dan O'Reilly (oreilldf)]

This likely because of version incompatibilities between libmtp and pymtp. Libmtp made some backwards incompatible changes to their API, which broke pymtp, which breaks the plugin. I had hacked together a workin version of pymtp, but its not well tested and I don't have it packaged anywhere. The author of pymtp has been working on a new version (http://nick125.com/projects/pymtp.html), so once that is ready I'll update the plugin to work with it.

[Jerome Chabod (jerome-chabod)]

Did you solve this problem of incompatibilities with libmtp/pymtp? If not you can check the patched pymtp I have done for gpodder which should also work for you: http://bugs.gpodder.org/show_bug.cgi?id=307.

[rospo (raffsol)]

no, it doesn't work. Exaile crash...

[Aleksandre Apkhaidze (a1eksandre)]

aaaaaa we still need it, mtp plugin!