does not import RW2 raw-images

[Maui Muc]

Shotwell does not import RW2 raw images from my Lumix LX5. Whereas, Shotwell can deal with this RW2 files by manually selecting them.

Shotwell: 0.11.1
Ubuntu: 11.04

[Lucas Beeler (lucas-yorba)]

When you say "Shotwell does not import RW2 raw images," do you mean that you can see thumbnails for the images in the camera preview page but when you click "Import Selected" or "Import All" the files are not copied into Shotwell? Or do you mean that you cannot even see any thumbnails for the RW2 files in the camera preview page?

[Maui Muc (stefan-mauerberger)]

Sorry for the missing but very important detail.
I cannot see any thumbnails for the RW2 files in the camera preview page?

[Jim Nelson (yorba-jim)]

Although you don't see the thumbnails, does Shotwell show you the photo object itself? That is, do you see the file listed, and you can click on it and import it?

Shotwell can't pull thumbnails off the camera from certain RAW file formats. RW2 might be one of them. Once you import it into Shotwell, it will generate a thumbnail for you.

[Maui Muc (stefan-mauerberger)]

Oh no, it becomes confusing ...

When I plug my camera into the USB port, Shotwell's import dialog is shown. However, this dialog only shows JPG files with its thumbnails. The RW2 raw-files aren't shown at all, neither thumbnails nor file names.

On the other Hand I can show RW2 files whit Shotwell by double clicking them or by manually importing them.

[Jim Nelson (yorba-jim)]

Hmm ... the only other complaint we've had about RW2 is this ticket: http://redmine.yorba.org/issues/3428

It looks like the workaround some of the people used was to import them manually. That is, when you connect your camera to your machine the camera will appear in Nautilus as a mounted drive. You can navigate down and copy the photos onto your hard disk, then drag-and-drop them into Shotwell. Apparently that will allow you to work around this problem.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Maui Muc (stefan-mauerberger)]

Sorry, for the duplicate! I was not aware, that Bugs also tracked at redmine.yorba.org .

For the first test, the workaround seems to work quite well. Thanks for this hint.

If it is useful, I could provide you some of these RW2 files. The simple reason why I didn't post a example file here is its size, nearly 12 MB.

[Maui Muc (stefan-mauerberger)]

> You can navigate down and copy the photos onto your hard disk, then drag-and-drop them into Shotwell.
Unforunatelly, this does not work very well. Indeed, Shotwell imports the JPEGs as well as RW2 files into its database. However, deleting pictures from Shotwell's trash only concerns JPEG files NOT RW2 files. The raw files remain on my hard-disk and re-opening Shotwell causes a file System scan and the pictures are again included in Shotwell.

[Eric Gregory (eric-yorba)]

Maui, have you tried updating to Shotwell 0.11.2? There were some fixes for RAW+JPEG handling that might help you.

[Maui Muc (stefan-mauerberger)]

Hi Eric, thank you very much for Version 11.2! This update fixes the problems with regard to delete RW2+JPEG files.

[Bilal Shahid (s9iper1)]

user said bug is solved so closing it.

[Maui Muc (stefan-mauerberger)]

Wait! As far as I can see the Problem isn't solved yet, it is still present, even in 11.6: RW2 raw files aren't be imported directly from the camera.
The bug which became fixed is that deleting an image now also affects the RW2 file as well as the jpg file.

[Amit Bhor (amit-bhor)]

I have the same problem with both Shotwell and Darktable. Cannot import raw (*.RW2) files directly from Mass storage Camera but if I import a folder by pointing it to the directory mount of the mass storage then it works. Shotwell 0.11.4, Ubuntu 11.10. Help!

[Amit Bhor (amit-bhor)]

This is the open bug at Shotwell referring to this issue. http://redmine.yorba.org/issues/3428

[Romano Giannetti (romano-giannetti)]

Confirm, still happens, seems a bug in libgphoto2. Seems dangerous, if you trust shotwell for the photo download and after that you format your card you can loose (a lot) of data.

Moreover, if you after the fact import the photos from a folder, it seems to download the RW2 files, but they are nowhere to be seen.

Why it is "Invalid"? It can lead to data loss. It's a critical one, and makes shotwell not useful at all.

[Romano Giannetti (romano-giannetti)]

Well, found it --- on import from folder the files are not copied, still stays where they where.

Still dangerous. I was importing form folder and then deleting it. Happy to have couple of backups.

[Romano Giannetti (romano-giannetti)]

Ping? This is a very dangerous bug, in the only preferred app that cannot be changed.

[Romano Giannetti (romano-giannetti)]

Seems a gphoto problem, added to this bug.

[Adam Dingle (adam-yorba)]

Romano,

the gphoto bug has been fixed in the latest libgphoto, version 2.4.12. That version is not yet available in Ubuntu Precise, but should appear there soon. We're still not completely sure whether that will solve the RW2 problem in Shotwell, however; see http://redmine.yorba.org/issues/3428 for details.

What, specifically, do you feel is dangerous? It looks like you imported photos from a folder and expected Shotwell to copy them, but they stayed in place. That has nothing to do with RW2, so if you're concerned about that behavior in Shotwell you should open a separate bug to discuss it.

[Maui Muc (stefan-mauerberger)]

I totally share Romano's concerns!

Isn't the most natural way using Shotwell to -> connect your camera -> import (copy) photos -> and erase memory card? From my point of view, this is exactly the workflow Shotwell suggests with its dialogs popping up.

Now, the very real problem is: JPEGs are copied as expected, whereas, RW2 files are ignored without giving any notice. If someone is not aware of the RW2-bug (most likely not only RW2s are affected), subsequently erasing the memory card deletes all RAW files. This is very annoying and frankly, I do not want to explain this bug/behavior to someone who gave Gnome/gphoto a change.

Just for my two cents, Stefan

[Romano Giannetti (romano-giannetti)]

@Adam: my concern is well expressed by Maui. The problem is that the failure to copy raw files is completely silent. I do not know if it's a problem in gphoto or shotwell, but the problem exists. And yes, the normal behaviour is download photos and then reformat the card, which can lead to a data loss.

I am using Oneiric, I think the bug is sufficiently important that it needs to be addressed here too, not only in the next release.

There are more problems with shotwell when dealing with raw+jpg downloads and management, but this one is a show-stopper.

[Sebastien Bacher (seb128)]

ok, I uploaded libgphoto 2.4.12 then noticed a .13 was rolled yesterday and updated to that one, the libgphoto said should be ok

[Maui Muc (stefan-mauerberger)]

Unfortunately, I have to report that RW2 files are still NOT recognized.

I tried on Debian Testing where I even had to update a whole bunch of libraries before I was able to build current Shotwell 11.92 from trunk. Btw. it is really a mess that you are using all this fancy stuff. Why do you need all these bleeding edge libraries? Is this really necessary? From my point of view it would be much more important to handle at least files correctly instead of referring to most recent libraries.

[Romano Giannetti (romano-giannetti)]

I personally am using now a script I wrote in python to download photos now, because shotwell is quite erratic, but I will help test when a new stable version will be available.

Other things that shotwell does "strange" with raw files:

* For the cameras where it can find raw files, when saving the files to disk the jpg name is mangled, stile "dsc001234_arw.jpg" (?)
* sometime (in the last version) a little-res, low quality "dsc001234_embedded.jpg" is saved too, adding clutter, and I have not found a way to stop it
* when viewing collection, only one miniature is shown, and there is no way to know if you have a raw or not short of opening the file manager
* when importing from a directory the behavior is different from when importing from a card --- files are not copied in the fist case. This is surprising because importing files from a, say, USB external drive gives the impression of having the photos in your collection but they disappear when you give back the key...

Hope this helps to have a better shotwell in the future. Thanks anyway for the work done so far.

[Adam Dingle (adam-yorba)]

Maui: thanks for letting us know that you're still not seeing RW2 files in Shotwell. It's difficult for us to test this since we don't have a camera that shoots RW2 here at Yorba, but we'll see what we can do.

Most GNOME programs (such as Nautilus or gedit or Totem) require the latest versions of all GNOME libraries if you want to build from trunk. I wouldn't say that this is "a mess" or that these libraries are "fancy stuff"; that's completely normal in the GNOME world. In fact, Shotwell is more generous than most GNOME programs in this respect: you can run the latest version of Shotwell even on the previous release of distributions like Ubuntu. For example, the upcoming Shotwell 0.12 will run either on Ubuntu 12.04 (Precise) or on Ubuntu 11.10 (Oneiric). If you want to run the latest Nautilus or Totem, you must run the latest GNOME, found in Ubuntu 12.04.

[Adam Dingle (adam-yorba)]

Romano: thanks for your observations, but we prefer that a bug ticket such as this one be focused on only one issue (RW2 support in this case). If you want to report a number of small problems, you could send a message to the Shotwell mailing list. Or you could file tickets for those problems individually either in Launchpad or in the Shotwell bug database at http://redmine.yorba.org/projects/shotwell .

[Romano Giannetti (romano-giannetti)]

Adam: fair comment. I will see if I can find the time to fill them on yorba. Meanwhile, if you need an image of a SD card with three or four raw and raw+jpg files from my LX5, I can manage to prepare it and share it with UbuntuOne. Just tell me if you need it.

[Adam Dingle (adam-yorba)]

Romano: yes, an SD image like that would be extremely helpful if it lets us reproduce this problem here at Yorba. If you can share an image with us via UbuntuOne (or post a download link) then we can take a look.

[Romano Giannetti (romano-giannetti)]

OK --- I have shot a handful of images, some in jpg, others in raw+jpeg. The link below is a tar.gz of the mounted directory when I insert the card. I can provide a "dd" (image) of the full card, but it's quite bigger, and I do not know if it's needed.

http://ubuntuone.com/1xVll5iwWimC7Iic6iFepk

Hope this helps,

[Adam Dingle (adam-yorba)]

Great. I downloaded the tar.gz and extracted it onto an SD card. For me, Shotwell shows the card as a Mass Storage Camera and correctly displays the RW2 and JPEG images (see attached screenshot). I'm able to import them just fine. I'm using Ubuntu 12.04, Shotwell from git master, libraw 0.14.4, libgphoto 2.4.13.

So it looks to me like this is fixed. Maui, could you try the same experiment and see if it works for you? If not, what versions of libraw and libgphoto do you have?

[Maui Muc (stefan-mauerberger)]

I checked my libraries and both, libgphoto2 and libraw were not up to date. I am even using Wheezy (Debian testing) how can this be?

Well, first of all I forced dpkg to install libgphoto 2.4.13. And this made the difference. After building Shotwell from trunk it shows also raw images. As a second step I tried to update libraw. As with libgphoto apt complained because of a version mismatch. So what, I also forced the installation of '''experimental''' libraw packages. The result is very disappointing:
...
/usr/lib/gcc/i486-linux-gnu/4.6/../../../../lib/libraw_r.so: undefined reference to `omp_get_num_threads'
collect2: ld returned 1 exit status
make: *** [shotwell] Error 1
Now, dependencies are messed up and Shotwell does no longer compile. Anyway, now we know that it is libgphoto which is responsible for our loved RW2 files. What a luck I did this on my test system.

[Adam Dingle (adam-yorba)]

Maui: Debian testing will often not have the latest packages. Even Debian experimental will sometimes have older packages than the alpha/beta releases of commercially supported distributions such as Ubuntu and Fedora. Debian is an awesome community project, but because the commercially supported distros have more resources sometimes they are able to keep packages more up to date.

I see on packages.debian.org that Debian unstable currently has libgphoto2 2.4.13 (the latest release) and libraw 0.14.0 (a few releases old, though that probably won't make a difference for this RW2 issue, I think.)

The build error you're seeing looks like this build error we had on Ubuntu Precise a month ago:

http://redmine.yorba.org/issues/4558

When I investigated that problem, I found that installing liblcms2-dev made Shotwell build without error. Maybe that will help for you too. If not, we can investigate more, but this ticket is not the right place for that discussion; please send email to the public Shotwell mailing list or to the Yorba team at <shotwell at yorba.org>.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Graham White (graham-o)]

This bug still affects Shotwell, and it's a major pain. It's now over 7 years since the bug was confirmed: I am using Shtowell 0.30.2, and it will still not import .RW2 files. There seems
to be no action on this.