Digikam imports images taken with Nikon Cameras extremely slow, takes a large amount of time processing exif data

Bug #596327 reported by Adam Stylinski
112
This bug affects 20 people
Affects Status Importance Assigned to Milestone
KDE Graphics
Confirmed
Undecided
Unassigned
digiKam
Fix Released
Medium
pyexiv2
Invalid
Undecided
Unassigned
exiv2 (Debian)
Fix Released
Unknown
exiv2 (Ubuntu)
Fix Released
Low
Unassigned
Nominated for Lucid by Adam Stylinski
Nominated for Maverick by Adam Stylinski
Nominated for Natty by Joshua McFadden

Bug Description

Binary package hint: exiv2

The upstream bug report was filed and fixed, it can be found here:
http://dev.exiv2.org/issues/show/677
Debian has fixed this as reports were filed for their version of digikam here:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=579835

Simply updating libexiv2 and exiv2 will not fix the problem as I've done this using maverick's packages. I believe that libkexiv needs to be updated as well, as I still had the issue.

Please fix this ASAP, the bug causes unbearably slow imports.

Revision history for this message
In , Dkde (dkde) wrote :

Version: 1.0.0-rc1 (using KDE 4.3.4)
OS: Linux
Installed from: Ubuntu Packages

I had used 1.0.0-beta5 to set the ratings on thousands on images, but
when I checked the JPEG files, the metadata had not been updated in
most of them. In 1.0.0-rc1, if I change the metadata on a single image,
it seems to be written OK, so I don't know if this is a problem that
has been fixed.

Anyway, the issue now is that I have thousands of image files that are
missing their metadata. I got into this situation a year or two ago and
sync'd the image metadata to the database--unaware that the metadata was
missing--and lost thousands of tags. In order to fix the metadata, I'm
going through each album, checking that the metadata (from the database)
looks OK, selecting the images in the album, and then clicking the menu
item "Image" -> "Write Metadata to Selected Images".

This works fine; the metadata is updated correctly in the files. However,
if there are a large number of selected image files, the process becomes
extremely slow. There appears to be a non-linear relationship between the
number of files and the time it takes to write the metadata in RC1:

  39 files -> 2 seconds
  71 files -> 4 seconds
  87 files -> 9 seconds
  120 files -> 43 seconds
  145 files -> 38 seconds
  155 files -> 44 seconds
  276 files -> 132 seconds
  484 files -> 312 seconds
  1,200 files -> I went for lunch.

Throughout this time, digiKam uses about 70-90% CPU.

I have turned on the option to update the file modification time when
the metadata is written. This will make it easier for me to spot when
digiKam is not updating the files properly in the future. Turning off
this option makes a big difference. For example, I updated 120 files in
43 seconds and the same operation took 38 seconds when I simply repeated
it immediately, but when I turned off the file modification time update,
the third run took 15 seconds. Similarly, the album with 484 images took
312 seconds when file modification time stamps were updated and 87 seconds
when the time stamps were not updated.

Yesterday, when I was not really timing things, the end of the 1,200+
file update seemed to hang. However, when I went to the console and
looked at the timestamps on the files, I could see that about five files
would have their time stamps updated and then there would be a long wait
(maybe 30-60 seconds) with no apparent changes and then five more files
would be updated and then nothing for another while, and so on until it
eventually finished.

I can see about 30 images in the thumbnail view at one time. When I
select all images in an album of, say, 100 files and update the metadata,
the progress bar quickly hits about 20-30% and then pauses while all of
the thumbnail images disappear and, one-by-one, slowly appear again.
After that the progress bar slowly moves on to 100%.

Is digiKam re-scanning an album each time it detects a file time stamp
change, even when it is making those changes itself on another thread?
That would explain the kind of behaviour I see. Is this likely to have
a big effect on the performance of other things like batch processing?

Revision history for this message
In , Caulier-gilles-9 (caulier-gilles-9) wrote :

Yes you have right. A process run from KDELibs and use KDirWatch API to check items changed. I think file date time is handle in by it.

I have no idea how to change this behavior for this moment. Anyway your investigation are very interesting for the future. we will take a care about...

Gilles Caulier

Revision history for this message
In , Dkde (dkde) wrote :

It just occurred to me that while I did not time the 1,200+ file
metadata update yesterday, my filesystem did.

There were 1,283 files and the difference between the earliest and
the latest modification time stamp is 30 min 19 sec. So you can
add this to my list of timings:

  1,283 files -> 1,819 seconds

These were from an older 2MP camera with an average files size of
about 800kB.

Would it be possible to stop digiKam from reacting to notifications
from KDirWatch while it has a background operation in progress?
Perhaps digiKam could simply store up these notifications and then
apply the appropriate ones when it has finished the first job. It
does not sound easy to figure out what "appropriate" would be, though.

In the meantime, I'll disable the modification time stamp updates,
write the metadata to the files, run "touch" from the command line
and then turn on the modification time stamp updates again, as I
won't be doing any more large batches of images again for a while.

Revision history for this message
In , Caulier-gilles-9 (caulier-gilles-9) wrote :

stopping KDirWatch events will have side-effect, especially if you use batch tool to sync metadata with database.

This tool is non modal dialog. It run in background and you can switch to main gui and continue to work. Now imagine than folder can non receive events from KDirWatch when you working on with icon-view...

Gilles Caulier

Revision history for this message
In , Dkde (dkde) wrote :

I'm imagining.... It would be much faster, wouldn't it? ;-)

I could always just hit the refresh key when I am interested in
an update.

I have found a slightly easier way around this. If I leave on the
modification time stamp updates (which is what I want), I can avoid
the long waits by selecting an album, clicking "Album" -> "Write
Metadata to Images" and then, once the job starts, selecting a
different album folder and waiting. I did that and it took 4.5 mins
to update 935 files. The file-watching and scanning only seems to
apply to the album currently being viewed, so viewing a different
album from the one where the metadata changes are under way seems to
avoid the issue.

I prefer to wait in the second album, because if I start a second
job to update the metadata, the progress of the first job cannot be
monitored any more. Once the second job completes, the progress bar
is cleared even though I can still hear the disk thrashing away on
the first job (and see the time stamps changing on the files from
the console). Maybe you could integrate the progress with the KDE
notifications widgets that can track several jobs at once (like in
the Dolphin file manager).

Revision history for this message
In , Caulier-gilles-9 (caulier-gilles-9) wrote :

>Maybe you could integrate the progress with the KDE
>notifications widgets that can track several jobs at once (like in
>the Dolphin file manager).

Already done with batch tool to write metadata, not yet with single album action...

Gilles Caulier

Revision history for this message
In , Marcel-wiesweg (marcel-wiesweg) wrote :

Probable explanation for the fact that changing the album helps:
A change of modification date will also trigger recreation of the thumbnail.

Revision history for this message
In , Fathi Boudra (fboudra) wrote :

Debian has a similar report (bug #571811):
Package: digikam
Version: 2:1.1.0-1

at the moment, digikam is very slow on writing metadata to images.

I select round about 20 JPGs between 3 and 7 MB, modify the description and click "apply". The update of the internal DB is fast, but the following write opration to the files is very, very slow, up to 5 minutes.

I'm not sure when this issue occured the first time, a few weeks ago everything run fine.

There's an open Bug at KDE: https://bugs.kde.org/show_bug.cgi?id=218633. I'm unsure if this is the same issue. I've deselected "update modification time" with no result.

iotop doesn't show any unusual, 0 B/s write and read. top reports digikam at 100% CPU.

If you need any additional information, please let me know.

Revision history for this message
In , Michael Holtermann (linux-michael-holtermann) wrote :

Hello,

this issue is still present in 1.2.0 (using in Debian unstable and experimental).

Interesting, in gwenview the same thing happens. Is it possible, that libexiv causes these problems?

Setting and managing of metadata is a key feature of digikam. Is there any chance of getting it back?

Nearly distressed, but with kind regards
Michael

Revision history for this message
In , Caulier-gilles-9 (caulier-gilles-9) wrote :

Yes, this file is fully relevant of Exiv2 library. For Info, I CC Andreas who manage this project.

Q : which Exiv2 version you use exactly ? Go to Help/ Components Info for details.

Gilles Caulier

Revision history for this message
In , Ahuggel (ahuggel) wrote :

Exiv2 0.19 has a known performance issue with images from some Nikon SLRs. It is _very_ slow for these. Images from any other cameras are not affected.

Just from reading the descriptions above, comment #7 might be related to this problem, and it should be easy to determine whether that is really the case. If it is, upgrading to exiv2 from SVN trunk will help.

For any other (new) performance issue, if you think it is related to exiv2, please narrow the test down to exiv2 and report to the exiv2 bugtracker at dev.exiv2.org preferably.

A simple test could be to run a more or less complex update command over a number of images, e.g., something like this:

$ time exiv2 -M'set Exif.Image.ImageDescription Some description' *.jpg

and compare with the time digiKam takes for a similar operation and with the time a simple cp takes to just copy the files within the same directory. Ideally, they should all be pretty close with cp-time < exiv2-time < digiKam-time

Andreas

Revision history for this message
In , Michael Holtermann (linux-michael-holtermann) wrote :

@9 from Gilles:
I'm using Debian unstable's version 0.19.

@10 from Andreas:
I've checked with some Panasonic Lumix images, this works very well. I'm shooting with a Nikon D90 SLR, which fits into your description.
I'm going to open a bug at Debian's libexiv2-6 package for an update to the trunk version of libexiv. If this doesn't help, I'll fill a bug at exiv2.org.

Comment #7 was opened by me at bugs.debian.org and forwarded by Fathi :-)

As suggested, some timings for manipulating 10 images:
Panasonic Lumix:
real 0m0.467s
user 0m0.128s
sys 0m0.204s
Nikon D90:
real 1m41.809s
user 1m35.098s
sys 0m0.516s

Many thanks!
Kind regards, Michael.

Revision history for this message
In , Mark Purcell (msp) wrote :

This looks like a duplicate of Bug 224094

Forwarded upstream:
http://dev.exiv2.org/issues/show/677

Revision history for this message
In , Michael Holtermann (linux-michael-holtermann) wrote :

exiv2 was recently updated to 0.20, which should not contain this bug any more.

Revision history for this message
In , Caulier-gilles-9 (caulier-gilles-9) wrote :

Yes, This file must be fixed using Exiv2 0.20.0.

I close it now. Re-open if necessary

Gilles Caulier

Revision history for this message
In , Caulier-gilles-9 (caulier-gilles-9) wrote :

*** Bug 237104 has been marked as a duplicate of this bug. ***

Revision history for this message
Adam Stylinski (kungfujesus06) wrote :

Actually an update to .20 may be necessary, I'm going to update it by building the library/exiv2 app from source in a minute.

Revision history for this message
Adam Stylinski (kungfujesus06) wrote :

Yeah, an update to .20 is necessary, as well as a rebuild of libkexiv against the newer libexiv2 libraries. Libkexiv has several forward and reverse dependencies so this is a task best left to the package managers, I'm not sure how to handle it on my own for testing other than doing everything by hand from source. It is definitely an important update, though, it can take as long as 20 seconds per image (unacceptable).

Revision history for this message
Adam Stylinski (kungfujesus06) wrote :

This is a duplicate bug report in KDE bugs, but the comments are a little more helpful (explaining the steps to rebuild the dependencies so that libkexiv uses the latest exiv2 libs):
https://bugs.kde.org/show_bug.cgi?id=218633

Changed in digikam:
status: Unknown → Fix Released
Revision history for this message
Adam Stylinski (kungfujesus06) wrote :

Whoops, meant this was the duplicate:
https://bugs.kde.org/show_bug.cgi?id=224094

Changed in exiv2 (Ubuntu):
status: New → Fix Released
Revision history for this message
Adam Stylinski (kungfujesus06) wrote :

While the kdegraphics package (libkexiv, specifically) is inherently fixed with the upstream fix for exiv2 v .20, I'm going to leave the status as "New" so that launchpad does not close this bug. A fix upstream != a fix downstream. Ubuntu Lucid Lynx is still affected by this.

Changed in exiv2 (Ubuntu):
assignee: nobody → Adam Stylinski (kungfujesus06)
assignee: Adam Stylinski (kungfujesus06) → nobody
Changed in exiv2 (Ubuntu):
status: Fix Released → New
Changed in exiv2 (Ubuntu):
status: New → Triaged
importance: Undecided → Low
Revision history for this message
Chris Samuel (chris-csamuel) wrote :

This bug is making it rather hard to use Digikam to manage my Nikon cameras, is there any chance of the 0.20 release been packaged (along with the rebuild of libkexiv) in the Kubuntu beta PPA to allow testing ?

Revision history for this message
Chris Samuel (chris-csamuel) wrote :

Launchpad doesn't seem to parse out the importance of Debian bugs, so it may be useful to know that whilst Ubuntu has marked this as "Low" it is marked in the Debian BTS as "Important".

Revision history for this message
Chris Samuel (chris-csamuel) wrote :

Apologies, can someone remove pyexiv2 as an affected project please ? I was trying to add a link to the upstream bug to the watcher and it added pyexiv2 rather than what I was expecting. :-(

Changed in exiv2 (Debian):
status: Unknown → Confirmed
Revision history for this message
Olivier Tilloy (osomon) wrote :

@Chris: AFAIK there is no way to remove pyexiv2 as an affected project, so I simply marked it as invalid there.

Changed in pyexiv2:
status: New → Invalid
Revision history for this message
Chris Samuel (chris-csamuel) wrote :

To give an indication of the impact of this bug, actions that would previously take about 0.08 seconds now take about 20 seconds, so importing a single photo is *250 times slower* than previous releases. :-(

Figures from https://bugs.kde.org/show_bug.cgi?id=224094#c19

Revision history for this message
p1ngu1n (vincent-barberet) wrote :

Hello,

I have a Nikon camera, and this bug is simply a showstopper. It takes up to one hour to import my photos, and again one hour to geotag the photos.

Is there a workaround ? I've recomplied libexiv2 from source (0.20) , then libkexiv and digikam using apt-source, but it is still linked against old libraries.

could someone post here a way to perform a workaround ?

Revision history for this message
Adam Stylinski (kungfujesus06) wrote :

p1ngu1n, mark this bug as affecting you so that Canonical can see the current impact of it.

Vish (vish)
tags: removed: metadata
Revision history for this message
Michael Holtermann (linux-michael-holtermann) wrote :

Digikam released version 4.4.4-1+b1 of libkexiv2-8 and eviv2-0.20-2, which
fixed the Nikon maker note issue.

Changed in exiv2 (Debian):
status: Confirmed → Fix Released
Revision history for this message
Adam Stylinski (kungfujesus06) wrote :

Is there an ETA on when this package will be brought up to snuff? I know it's not a security issue but it's a huge usability issue and an easy fix.

Revision history for this message
karatsu (pages-michael) wrote :

It's an important bug for everyone using digikam & have a nikon
all metadata can't be update in files because it take too long (couple of minute for 10 photos on AMD64 4000+)

at least is there a way to solve it by building our own deb ?

Revision history for this message
Chris Samuel (chris-csamuel) wrote :

This bug is still present in Maverick (which will be 10.10).

Manipulating an image from a Nikon D90 is over 100 times slower than manipulating the *EXACT* same image with the camera maker set to "NoName" in its metadata - and reverting it back to "NIKON CORPORATION" makes it go over 100 times slower again.

PLEASE PLEASE PLEASE CAN WE GET AN UPDATE ON THIS BUG!

Here's the demonstration of the slowdown:

root@quad:~# apt-cache policy exiv2
exiv2:
  Installed: 0.19-3
  Candidate: 0.19-3
  Version table:
 *** 0.19-3 0
        500 http://mirror.internode.on.net/pub/ubuntu/ubuntu/ maverick/main amd64 Packages
        100 /var/lib/dpkg/status

chris@quad:/tmp$ exiv2 DSC_0147.JPG
File name : DSC_0147.JPG
File size : 4084876 Bytes
MIME type : image/jpeg
Image size : 4288 x 2848
Camera make : NIKON CORPORATION
Camera model : NIKON D90
Image timestamp : 2010:09:18 15:53:45
Image number :
Exposure time : 1/15 s
Aperture : F5
Exposure bias : 0 EV
Flash : No flash
Flash bias :
Focal length : 75.0 mm (35mm equivalent: 112.0 mm)
Subject distance:
ISO speed : 3200
Exposure mode : Auto
Metering mode : Multi-segment
Macro mode :
Image quality : FINE
Exif resolution : 4288 x 2848
White balance : AUTO
Thumbnail : image/jpeg, 9051 Bytes
Copyright :
Exif comment :

chris@quad:/tmp$ time exiv2 -M'set Exif.Image.Make NoName' DSC_0147.JPG

real 0m4.323s
user 0m4.270s
sys 0m0.020s

chris@quad:/tmp$ time exiv2 -M'set Exif.Image.Make NIKON CORPORATION' DSC_0147.JPG

real 0m0.030s
user 0m0.000s
sys 0m0.020s

chris@quad:/tmp$ time exiv2 -M'set Exif.Image.Make NoName' DSC_0147.JPG
real 0m6.201s
user 0m6.170s
sys 0m0.020s

Marco Nolden (marco-n)
description: updated
Revision history for this message
Adam Stylinski (kungfujesus06) wrote :

Yeah, what gives? Debian has had this updated in -stable for a long time now. Don't next releases for Ubuntu follow some derived version of Debian's repos/packages? I would think Mighty Maverick would have this package and the dependent KDE Graphics packages updated.

Revision history for this message
trfonseca@gmail.com (trfonseca) wrote :

Is there anything that can be done to help with getting this defect fixed?
From what I can see this has already been fixed in Debian, and looking at the changelog of the package libexiv2-6 in Maverick, there was also an attempt to get it fixed in Ubuntu.
Unfortunately don't have enough knowledge to apply the fix myself, but if testing or anything else is needed please say so.

Revision history for this message
Adam Stylinski (kungfujesus06) wrote :

Pretty sure it's just Ubuntu's KDE team not compiling a new version of libkexiv against the most recent exiv2, but I can't say for certain. The entire cmake based KDE4 build process is a mess mixed on top of debian's package building process. Let's hope rolling into 10.10 and (K)Ubuntu's updated packages will fix things by release time, but it's hard to say for sure. If it doesn't fix it by then, I am highly considering switching to a different distro or making another visit into #ubuntu to get in touch with a maintainer.

Revision history for this message
Adam Stylinski (kungfujesus06) wrote :

Yep, post-maverick release this bug still exists...

Revision history for this message
Adam Stylinski (kungfujesus06) wrote :

Also Chris your demo of exiv2 shows that the exiv2 package is not the problem at the moment, as when the Make is set to NIKON CORPORATION it works in subsecond time. It's libkexiv that needs rebuilding, somebody should talk to the KDE Graphics team about this.

Revision history for this message
Jeremy Wilkins (wjeremy) wrote :

@Adam Actually, Chis is interpreting his data correctly. It isn't until the make has been set that an operation will be effective under that make, however his example is hard to follow because the set make operation should not be the one being timed. The set operation is only effective on the next command so the commands that are relevant to the Nikon camera slow down are after it is set to Nikon.

Revision history for this message
Joshua McFadden (josh-mindsound) wrote :

This hurts! I asked about it over in the forums but perhaps I was too verbose? http://ubuntuforums.org/showthread.php?p=9984687

If I understand the conversation on this bug correctly, rebuilding libkexiv2 (provided by kdegraphics) against libexiv2 0.20 will fix the problem in digikam. I'm trying to figure out how to delta the source debs from "apt-get source" to make them happy but it's rough going. I haven't touched a compiler in about 5 years and I have no idea how dpkg works. Has anyone tried this path to fix this themselves?

Revision history for this message
Joshua McFadden (josh-mindsound) wrote :

OK, given the upstream bug report (http://dev.exiv2.org/issues/show/677) I:

1) created a patch of Andreas Huggel's fix that went into exiv2 0.20 (svn diff -r2016-2020, the revisions were consecutive).

2) applied it to an exiv2-0.19 source deb, and recompiled that deb and kdegraphics (provides libkexiv2).

3) installed digikam from apt (i'm on amd64).

This fixes the performance problem for me. Here's the exiv2 patch. I will ping Andreas and ask whether he can sanity-check this. I would be extremely grateful if we could get this patch into Maverick!

Revision history for this message
Chris Samuel (chris-csamuel) wrote :

Excellent work Joshua, the only change I had to make to your patch (in order to add it automatically to the build in the debian/patches directory) was to change the references from src/$file to a/src/$file for lines removed and b/src/$file for lines added. Look at exiv2-0.19/debian/patches/00_hyphens_used_as_minus.diff as an example. I also named it 01_nikon_exif.diff and added it to the series file in debian/patches to automate its application.

After rebuilding (and installing) the exiv2 packages and then rebuilding kdegraphics and installing the new libkexiv2-*.deb packages I was able to fire up digikam, shot some random images of my office and downloaded the images from the SD card of my D90 - much much faster!

Final proof that this patch works is redoing my previous tests with the exiv2 command on the same test image:

chris@quad:/tmp$ time exiv2 -M'set Exif.Image.Make NoName' DSC_0147.JPG

real 0m0.027s
user 0m0.000s
sys 0m0.020s

chris@quad:/tmp$ time exiv2 -M'set Exif.Image.Make NIKON CORPORATION' DSC_0147.JPG

real 0m0.016s
user 0m0.000s
sys 0m0.010s

chris@quad:/tmp$ time exiv2 -M'set Exif.Image.Make NoName' DSC_0147.JPG

real 0m0.027s
user 0m0.010s
sys 0m0.010s

I make that between 155x and 229x faster than the unpatched version!

Revision history for this message
Chris Samuel (chris-csamuel) wrote :

Unfortunately having now rebooted (for some other updates) I now find that the patch in question causes Konqueror, Gwenview and Nepomuk to SEGV on some images. For example this HPC-101 poster:

http://mohamedfahmed.files.wordpress.com/2010/10/hpc-poster-56x45-inches.jpg

If I take the packages I built off hold and let aptitude upgrade them back to the PPA versions then it starts working again. :-(

So close Joshua!

Revision history for this message
Adam Stylinski (kungfujesus06) wrote :

Chris, you probably need to recompile the other kde libraries with your updated kexiv and exiv2 sources. Does anybody know any easy way to produce a reverse dependency graph for apt?

Revision history for this message
Adam Stylinski (kungfujesus06) wrote :

Actually instead of fiddling with ldd, install all of the kdegraphics packages under the umbrella that are compiled with the apt-sources. Pretty sure gwenview is included in kdegraphics, see if you still get a segfault with the recompiled version.

tags: added: patch
Revision history for this message
Joshua McFadden (josh-mindsound) wrote :

Chris, Adam is (hopefully) right. I recompiled all of kdegraphics against the patched exiv2 and installed every built deb. I already cleaned up my working dir but it was something like 20 debs. Let us know if you try this and it works for you! (Sorry I can't test more myself, work calls...)

Revision history for this message
Chris Samuel (chris-csamuel) wrote :

I will give that a go though I'm very surprised it may be necessary, I thought the whole point of something like libkexiv was to insulate KDE apps from such changes, and libkexiv itself has not been modified, just recompiled against another shared library.

Revision history for this message
Joshua McFadden (josh-mindsound) wrote :

No clue myself, I'm a lowly Java developer who knows how to use Subversion. :)

Revision history for this message
Adam Stylinski (kungfujesus06) wrote :

My knowledge of C/C++ compilers I wouldn't say is thorough as of yet, but it's possible it's a link time related error where the symbol tables do not line up among the shared libraries. API breakage is quite common among libraries even if the interface is seemingly the same. Normally you intentionally change the API's shared lib name to force other packages to build against the new version of the library whenever you have API changes. In many circumstances you can make changes to the code without worrying about API breakage, but I'm pretty sure there are circumstances that will force a recompile or relink. Pretty sure changing the implementation of functions within a library will cause the API to break if the underlying application operates on data structures utilized by the library. Here's a paper on how APIs can break:
http://syrcose.ispras.ru/2009/files/02_paper.pdf

It's a good read if you have the time.

Revision history for this message
Joshua McFadden (josh-mindsound) wrote :

Comments from Andreas Huggel (who fixed this in exiv2 0.20):

"Well, if my bookkeeping was correct this should work, I don't
immediately see anything wrong with the patch.

"The concern is that this patch is *not* a binary compatible change.
You can't simply re-compile libexiv2 and distribute that instead of
the vanilla 0.19. All applications that depend on libexiv2 will need
to be re-compiled with the patched version of libexiv2. In fact Debian
tested this. They had to rollback the patch."

Revision history for this message
Adam Stylinski (kungfujesus06) wrote :

Yep, sounds like an API incompatibility to me. Either patch and recompile all packages against this guy (after doing a version bump on the shared lib soname) or do the same while bumping up to version .20.

FWIW Fedora 14 will be using .20.1 and have a functioning setup of Digikam, and I'm sure other distros have already or will follow.

Revision history for this message
martinux (mmouze) wrote :

Hello,

Any chances to have this bug fix in released version ?
This bug is present in 10.04 LTS and 10.10 versions and affect Nikon users (at least those having D90 and D5000).
In my case it takes about one hour to upload 100 photos from my D90. I can't live with that as I am doing about 500 shots a weeks.

Revision history for this message
Piotr Kęplicz (keplicz) wrote :

I'd suggest not to wait for an official fix, which is expected in April, 2011, but to fix it yourself.

My solution:
- grab libexiv2-9 and libexiv2-dev from Debian testing (current version is 0.20-2) and install them,
- get source packages for kdegraphics and rebuild them,
- install the rebuild libkexiv2-8 package (no need to update the other ones),
- you might want to mark this package "hold" in order to prevent it from being updated, at least until it gets fixed in Ubuntu.

Changed in kdegraphics:
status: New → Confirmed
Revision history for this message
Adam Stylinski (kungfujesus06) wrote :

Piotr's solution will work, however keep in mind that you may have other packages installed that depend on exiv, in which case you'll have to rebuild those packages with apt-source against the newer library as well in order to be binary compatible.

Revision history for this message
Piotr Kęplicz (keplicz) wrote : Re: [Bug 596327] Re: Digikam imports images taken with Nikon Cameras extremely slow, takes a large amount of time processing exif data

Or, you can leave current libexiv2-6 (0.19) installed alongside the newer
libexiv2-9; of course packages depending on the older version will still be
affected by this bug.

Revision history for this message
martinux (mmouze) wrote :

Thanks guys for the answers.
I don't want to have self complied package on Ubuntu (I use it to have an easy to maintain and up to date distro).
I will stick with my current solution which is to use an other computer that as a debian stable installed on it to upload the photos, then transfer them to my main computer. It's painful but it saves me some hours.

Revision history for this message
Bruno Randolf (randolf-bruno) wrote :

I did what Piotr suggested and rebuild the packages. Here is the libkexic2 package I compiled for amd64:

http://br1.einfach.org/tmp/libkexiv2-8_4.5.1-0ubuntu3_amd64.deb

Here is a copy of the debian testing libexiv2 package, I compiled it against:

http://br1.einfach.org/tmp/libexiv2-9_0.20-2_amd64.deb

Revision history for this message
Adam Stylinski (kungfujesus06) wrote :

Piotr, I'm not sure dpkg will allow you to do that as they have conflicting sonames when the shared libs are installed. I'm sure there's a way to put it under a different prefix if you did it by hand, but I'm not too terribly certain there's a clean way to not replace the old exiv2 with debian's testing .deb. In any case, anything linked against exiv2 (perhaps GNOME related applications) will most probably break by doing this.

 I don't understand why a special PPA hasn't been made yet or this isn't in maverick-proposed at least. Hopefully when Ubuntu puts out the rolling release distribution repo we won't have to deal with this API versioning rules nonsense anymore. I understand why it's there, major versions signify binary incompatibility, but it has to happen between releases anyway in a less frequent manner.

Revision history for this message
Adam Stylinski (kungfujesus06) wrote :

Ahh, it will replace the headers (the libexiv2-dev package), however the libraries seem to be ok, as most of the Ubuntu packages I've looked at are linked against the full so name and not the top level one.
e.g.:

ldd /usr/lib/libgexiv2.so.0.0.0
 linux-gate.so.1 => (0x00137000)
 libgobject-2.0.so.0 => /usr/lib/libgobject-2.0.so.0 (0x00266000)
 libgthread-2.0.so.0 => /usr/lib/libgthread-2.0.so.0 (0x00764000)
 librt.so.1 => /lib/librt.so.1 (0x00aa6000)
 libglib-2.0.so.0 => /lib/libglib-2.0.so.0 (0x00138000)
 libexiv2.so.6 => /usr/lib/libexiv2.so.6 (0x0045e000)
 libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00110000)
 libpthread.so.0 => /lib/libpthread.so.0 (0x00241000)
 libc.so.6 => /lib/libc.so.6 (0x00769000)
 libpcre.so.3 => /lib/libpcre.so.3 (0x00c5a000)
 /lib/ld-linux.so.2 (0x00440000)
 libz.so.1 => /lib/libz.so.1 (0x00bd8000)
 libexpat.so.1 => /lib/libexpat.so.1 (0x00bab000)
 libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00d43000)
 libm.so.6 => /lib/libm.so.6 (0x00207000)

Hopefully the newly built packages against this lib will from this point forward look at the symlink destination for the so.0.0 when they undergo linking.

Revision history for this message
Piotr Kęplicz (keplicz) wrote :

As you see, there's no conflict whatsoever. In fact library package names carry so version number so that they can be installed side by side - unless package contents conflict, of course.

Revision history for this message
Chris Samuel (chris-csamuel) wrote :

FWIW it appears that Natty Narwhal (11.04) will finally have exiv 0.20 according to this:

https://launchpad.net/ubuntu/natty/amd64/exiv2

I've got to say I'm *sorely* tempted to upgrade to Natty because of this, though I suspect I'll put it off until the beta's appear..

Revision history for this message
In , Dkde (dkde) wrote :

I think my bug report was hijacked by comment #7. My original problem has nothing to do with Nikon images and Exiv2 problems, I shoot Canon. The issue is that enabling updates to the file modification times causes problems when DigiKam's own changes to images causes DigiKam to rescan the images and generates new thumbnails even though it is unnecessary.

Can we wind this bug back to comment #6? Is the problem as described originally and in comments #1-#6 fixed, or was this bug marked as "FIXED" because the unrelated issue with Nikon images and Exiv2 was fixed?

Changed in digikam:
importance: Unknown → Medium
Revision history for this message
Chris Samuel (chris-csamuel) wrote :

OK, upgraded to 11.04 (Natty) beta of Kubuntu, the issue with exiv2 does indeed appear to be solved, though I need the chance to recuperate from hospital before having new photos to import into Digikam to confirm it's OK there.

Here's what a repeat of the test I reported in comment #16 https://bugs.launchpad.net/ubuntu/+source/exiv2/+bug/596327/comments/16 shows now:

chris@quad:/tmp$ time exiv2 -M'set Exif.Image.Make NoName' DSC_0147.JPG

real 0m0.020s
user 0m0.000s
sys 0m0.010s
chris@quad:/tmp$ time exiv2 -M'set Exif.Image.Make NIKON CORPORATION' DSC_0147.JPG

real 0m0.020s
user 0m0.000s
sys 0m0.010s
chris@quad:/tmp$ time exiv2 -M'set Exif.Image.Make NoName' DSC_0147.JPG

real 0m0.020s
user 0m0.000s
sys 0m0.010s

*Much* better...

Revision history for this message
Chris Samuel (chris-csamuel) wrote :

Was well enough today to go out and take photos and I can confirm that the import into Digikam of images from the SD card of my Nikon D90 is lightning fast compared to before! Even just loading the first preview of all the photos is noticeably faster.

Importing the 63 photos took only a minute or so, compared with tens of minutes previously.

Changed in exiv2 (Ubuntu):
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.