UnicodeDecodeErro reloading metadatas

[LCG]

Hi,

I create a collection with a photo folder named 2011 (which contains every photos taken in 2011).

All images seem to be imported but the metadatas don't. I have no tags in the list on the right.
So I tried to select all files en reloading metadatas but in a terminal I have an error. Looks like there is a problem with Gimp files (I don't put all errors as they are all the same) :

-----------------------------------------------------------------
####adding /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Junior/Junior-01.xcf
verify loading metadata /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Junior/junior-2.xcf
****REMOVING /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Junior/junior-2.xcf
Error reading metadata for /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Junior/junior-2.xcf
Traceback (most recent call last):
  File "/home/nico/UTILITAIRES/phraymd/phraymd/modules/phraymd/metadata2.py", line 65, in load_metadata
    rawmeta.read()
  File "/usr/lib/python2.7/dist-packages/pyexiv2/metadata.py", line 107, in read
    self.__image = self._instantiate_image(self.filename)
  File "/usr/lib/python2.7/dist-packages/pyexiv2/metadata.py", line 79, in _instantiate_image
    return libexiv2python._Image(filename)
IOError: /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Junior/junior-2.xcf: The file contains data of an unknown image type

####adding /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Junior/junior-2.xcf
verify loading metadata /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Mark/Mark-01.xcf
****REMOVING /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Mark/Mark-01.xcf
Error reading metadata for /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Mark/Mark-01.xcf
Traceback (most recent call last):
  File "/home/nico/UTILITAIRES/phraymd/phraymd/modules/phraymd/metadata2.py", line 65, in load_metadata
    rawmeta.read()
  File "/usr/lib/python2.7/dist-packages/pyexiv2/metadata.py", line 107, in read
    self.__image = self._instantiate_image(self.filename)
  File "/usr/lib/python2.7/dist-packages/pyexiv2/metadata.py", line 79, in _instantiate_image
    return libexiv2python._Image(filename)
IOError: /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Mark/Mark-01.xcf: The file contains data of an unknown image type

####adding /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Mark/Mark-01.xcf
verify loading metadata /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Micha/Micha-01.xcf
****REMOVING /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Micha/Micha-01.xcf
Error reading metadata for /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Micha/Micha-01.xcf
Traceback (most recent call last):
  File "/home/nico/UTILITAIRES/phraymd/phraymd/modules/phraymd/metadata2.py", line 65, in load_metadata
    rawmeta.read()
  File "/usr/lib/python2.7/dist-packages/pyexiv2/metadata.py", line 107, in read
    self.__image = self._instantiate_image(self.filename)
  File "/usr/lib/python2.7/dist-packages/pyexiv2/metadata.py", line 79, in _instantiate_image
    return libexiv2python._Image(filename)
IOError: /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Micha/Micha-01.xcf: The file contains data of an unknown image type

####adding /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Micha/Micha-01.xcf
verify loading metadata /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Nathan/Nathan-01.xcf
****REMOVING /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Nathan/Nathan-01.xcf
Error reading metadata for /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Nathan/Nathan-01.xcf
Traceback (most recent call last):
  File "/home/nico/UTILITAIRES/phraymd/phraymd/modules/phraymd/metadata2.py", line 65, in load_metadata
    rawmeta.read()
  File "/usr/lib/python2.7/dist-packages/pyexiv2/metadata.py", line 107, in read
    self.__image = self._instantiate_image(self.filename)
  File "/usr/lib/python2.7/dist-packages/pyexiv2/metadata.py", line 79, in _instantiate_image
    return libexiv2python._Image(filename)
IOError: /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Nathan/Nathan-01.xcf: The file contains data of an unknown image type

####adding /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Nathan/Nathan-01.xcf
verify loading metadata /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Quentin/Quentin-01.xcf
****REMOVING /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Quentin/Quentin-01.xcf
Error reading metadata for /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Quentin/Quentin-01.xcf
Traceback (most recent call last):
  File "/home/nico/UTILITAIRES/phraymd/phraymd/modules/phraymd/metadata2.py", line 65, in load_metadata
    rawmeta.read()
  File "/usr/lib/python2.7/dist-packages/pyexiv2/metadata.py", line 107, in read
    self.__image = self._instantiate_image(self.filename)
  File "/usr/lib/python2.7/dist-packages/pyexiv2/metadata.py", line 79, in _instantiate_image
    return libexiv2python._Image(filename)
IOError: /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Quentin/Quentin-01.xcf: The file contains data of an unknown image type

####adding /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Quentin/Quentin-01.xcf
verify loading metadata /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Remy/Remy-01.xcf
****REMOVING /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Remy/Remy-01.xcf
Error reading metadata for /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Remy/Remy-01.xcf
Traceback (most recent call last):
  File "/home/nico/UTILITAIRES/phraymd/phraymd/modules/phraymd/metadata2.py", line 65, in load_metadata
    rawmeta.read()
  File "/usr/lib/python2.7/dist-packages/pyexiv2/metadata.py", line 107, in read
    self.__image = self._instantiate_image(self.filename)
  File "/usr/lib/python2.7/dist-packages/pyexiv2/metadata.py", line 79, in _instantiate_image
    return libexiv2python._Image(filename)
IOError: /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Remy/Remy-01.xcf: The file contains data of an unknown image type

####adding /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Remy/Remy-01.xcf
verify loading metadata /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Ryan/Ryan-01.xcf
****REMOVING /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Ryan/Ryan-01.xcf
Error reading metadata for /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Ryan/Ryan-01.xcf
Traceback (most recent call last):
  File "/home/nico/UTILITAIRES/phraymd/phraymd/modules/phraymd/metadata2.py", line 65, in load_metadata
    rawmeta.read()
  File "/usr/lib/python2.7/dist-packages/pyexiv2/metadata.py", line 107, in read
    self.__image = self._instantiate_image(self.filename)
  File "/usr/lib/python2.7/dist-packages/pyexiv2/metadata.py", line 79, in _instantiate_image
    return libexiv2python._Image(filename)
IOError: /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Ryan/Ryan-01.xcf: The file contains data of an unknown image type

####adding /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Ryan/Ryan-01.xcf
verify loading metadata /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Thib-Jul/Thib-Jul-01.xcf
****REMOVING /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Thib-Jul/Thib-Jul-01.xcf
Error reading metadata for /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Thib-Jul/Thib-Jul-01.xcf
Traceback (most recent call last):
  File "/home/nico/UTILITAIRES/phraymd/phraymd/modules/phraymd/metadata2.py", line 65, in load_metadata
    rawmeta.read()
  File "/usr/lib/python2.7/dist-packages/pyexiv2/metadata.py", line 107, in read
    self.__image = self._instantiate_image(self.filename)
  File "/usr/lib/python2.7/dist-packages/pyexiv2/metadata.py", line 79, in _instantiate_image
    return libexiv2python._Image(filename)
IOError: /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Thib-Jul/Thib-Jul-01.xcf: The file contains data of an unknown image type

####adding /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Thib-Jul/Thib-Jul-01.xcf
verify loading metadata /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Yohan/Yohan-01.xcf
****REMOVING /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Yohan/Yohan-01.xcf
Error reading metadata for /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Yohan/Yohan-01.xcf
Traceback (most recent call last):
  File "/home/nico/UTILITAIRES/phraymd/phraymd/modules/phraymd/metadata2.py", line 65, in load_metadata
    rawmeta.read()
  File "/usr/lib/python2.7/dist-packages/pyexiv2/metadata.py", line 107, in read
    self.__image = self._instantiate_image(self.filename)
  File "/usr/lib/python2.7/dist-packages/pyexiv2/metadata.py", line 79, in _instantiate_image
    return libexiv2python._Image(filename)
IOError: /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Yohan/Yohan-01.xcf: The file contains data of an unknown image type

####adding /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Yohan/Yohan-01.xcf
INFO:phraymd core:Image verification complete
Exception in thread Thread-602:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 755, in run
    self.function(*self.args, **self.kwargs)
  File "/home/nico/UTILITAIRES/phraymd/phraymd/modules/phraymd/plugins/tagui.py", line 805, in refresh
    tag_cloud_list.sort()
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe0 in position 5: invalid continuation byte
----------------------------------------------------------------------------------

The loading metadata process, then, doesn't go to the end.

[dmoore (damien-moore)]

phraymd currently only supports metadata that the Exiv2 library supports. I
don't think gimp's XCF is in the list of supported file types. If there is
a simple work around to add XCF support I am happy to add it.

Does phraymd hang or do you just get the messages in the terminal? If it
doesn't hang, this is expected behavior until a work around for reading XCF
metadata is found.

Do thumbnails display correctly?

Thanks for the bug report.
On May 31, 2012 6:35 PM, "LCG" <email address hidden> wrote:

> Public bug reported:
>
> Hi,
>
> I create a collection with a photo folder named 2011 (which contains
> every photos taken in 2011).
>
> All images seem to be imported but the metadatas don't. I have no tags in
> the list on the right.
> So I tried to select all files en reloading metadatas but in a terminal I
> have an error. Looks like there is a problem with Gimp files (I don't put
> all errors as they are all the same) :
>
> -----------------------------------------------------------------
> ####adding
> /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Junior/Junior-01.xcf
> verify loading metadata
> /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Junior/junior-2.xcf
> ****REMOVING
> /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Junior/junior-2.xcf
> Error reading metadata for
> /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Junior/junior-2.xcf
> Traceback (most recent call last):
> File
> "/home/nico/UTILITAIRES/phraymd/phraymd/modules/phraymd/metadata2.py", line
> 65, in load_metadata
> rawmeta.read()
> File "/usr/lib/python2.7/dist-packages/pyexiv2/metadata.py", line 107, in
> read
> self.__image = self._instantiate_image(self.filename)
> File "/usr/lib/python2.7/dist-packages/pyexiv2/metadata.py", line 79, in
> _instantiate_image
> return libexiv2python._Image(filename)
> IOError:
> /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Junior/junior-2.xcf:
> The file contains data of an unknown image type
>
> ####adding
> /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Junior/junior-2.xcf
> verify loading metadata
> /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Mark/Mark-01.xcf
> ****REMOVING
> /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Mark/Mark-01.xcf
> Error reading metadata for
> /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Mark/Mark-01.xcf
> Traceback (most recent call last):
> File
> "/home/nico/UTILITAIRES/phraymd/phraymd/modules/phraymd/metadata2.py", line
> 65, in load_metadata
> rawmeta.read()
> File "/usr/lib/python2.7/dist-packages/pyexiv2/metadata.py", line 107, in
> read
> self.__image = self._instantiate_image(self.filename)
> File "/usr/lib/python2.7/dist-packages/pyexiv2/metadata.py", line 79, in
> _instantiate_image
> return libexiv2python._Image(filename)
> IOError:
> /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Mark/Mark-01.xcf: The
> file contains data of an unknown image type
>
> ####adding
> /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Mark/Mark-01.xcf
> verify loading metadata
> /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Micha/Micha-01.xcf
> ****REMOVING
> /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Micha/Micha-...

[LCG (le-coude-gauche)]

Ok, it's not really a bug. Sorry to have post it as it.

Well,you're right,. xcf files are not in the supported list of Exiv2. I don't know how to make them work.
But for me it's not a real problem. I don't need .xcf's metadatas to be read and add to the database (I juste have a few xcf files in my collection). The "best solution" will probably to ask phraymd to not read metadata in XCF's files.

The problem is that, due to these errors, phraymd doesn't import all others metadatas (from jpg and .nef files). So I can't import my 2011 collection which represents more than 33000 photos. So it make the soft unusuable (for me).

"Does phraymd hang or do you just get the messages in the terminal? If it
 doesn't hang, this is expected behavior until a work around for reading XCF
 metadata is found."
No, phraymd doesn't hang. The message just appear in the terminal. In the main window it's just like everything is fine (except tags are not imported)

"Do thumbnails display correctly?"
Yes, phraymd import all photos and their thumbnails correctly.

Thanks for your quick reply.

[dmoore (damien-moore)]

> The problem is that, due to these errors, phraymd doesn't import all
> others metadatas (from jpg and .nef files). So I can't import my 2011
> collection which represents more than 33000 photos. So it make the soft
> unusuable (for me).

The errors reading metadata on XCF shouldn't have any effect on reading the
metadata of the other files. I have a few XCF's in my collection that
haven't caused me any problems.

Looking more closely at the console output, it looks like the culprit is in
the last traceback error message. phraymd is attempting to sort the list of
tags, but fails due to an encoding issue. One or more of your NEF or JPG
files may be encoded in something other than utf8 and exiv2 fails to
convert it, or the metadata in one of your files is corrupt. Try to isolate
which file or files are causing this and if possible send me a sample. I
can try to come up with a work around to make the tag sort more robust
without a sample, but a sample would make it easier.

[dmoore (damien-moore)]

I committed an attempt to fix this bug in revision 556. Please test
and let me know if it solves the problem.

[LCG (le-coude-gauche)]

Hello and sorry for my late reply.

Well, I'm on revision 558 since thuis morning. I still have error message about .xcf files.
I noticed that the "tag cloud" is not created for my 2011 collection but jpg files have tags when I point the mouse on them.

Isolate one or few files in a 33000 photos collection seems difficult to me. Do you have an idea on how I can procede?

I have another problem with tags. Maybe it is related?
In my 2012 collection, my jpg's tagged files are not listed (they are not in the thumbnails grid).

Since 2011, I use the script created on PhotoLinux to tag my pictures. (you can see it here : http://www.photolinux.fr/2011/09/tagguez-vos-images-rapidement-et-en-lot/).

Maybe the source of the problem?

[LCG (le-coude-gauche)]

I forget to say that I have no problem with my 2012 .nef files.

[dmoore (damien-moore)]

> Well, I'm on revision 558 since thuis morning. I still have error message
about .xcf files.

Ignore those (I will eventually clean up those messages)

> I noticed that the "tag cloud" is not created for my 2011 collection but
jpg files have tags when I point the mouse on them.

Yes. This bug only affects the listing of tags in the tag pane. I think the
tags are being read correctly in most but not all cases.

>
> Isolate one or few files in a 33000 photos collection seems difficult to
> me. Do you have an idea on how I can procede?

You could try creating a new collection on subfolders of your 2011 files.
But don't worry about doing that yet. I will check in another change later
that should help us diagnose the problem. It will sort the tag list in a
more robust way.

>
> I have another problem with tags. Maybe it is related?
> In my 2012 collection, my jpg's tagged files are not listed (they are
not in the thumbnails grid).

The images are missing altogether or just thumbnails?

Sounds odd. post some of the output when you create the collection on that
folder.

>
> Since 2011, I use the script created on PhotoLinux to tag my pictures.
> (you can see it here : http://www.photolinux.fr/2011/09/tagguez-vos-
> images-rapidement-et-en-lot/).
>
> Maybe the source of the problem?

Could be. Maybe some sort of encoding issue.

[dmoore (damien-moore)]

> Well, I'm on revision 558 since thuis morning. I still have error message about .xcf files.
> I noticed that the "tag cloud" is not created for my 2011 collection but jpg files have tags when I point the mouse on them.

One more thing. Can you post last error message when running rev 558
(should be the one related to the tag_cloud)

[dmoore (damien-moore)]

Now at rev 561. Rev 560 added more robust tag handling (hopefully).
Please test and report the relevant console output.

[LCG (le-coude-gauche)]

Hello,

>> I have another problem with tags. Maybe it is related?
>> In my 2012 collection, my jpg's tagged files are not listed (they are not in the thumbnails grid).
>The images are missing altogether or just thumbnails?
>Sounds odd. post some of the output when you create the collection on that
 folder.

Well, it looks like the images are missing in the collection. But only the JPG files. NEF files still presents.
Here is the output when I open this collection (there are a lot of mimetype error before the one posted) :

-------------------------------------------
DEBUG:phraymd core:Directory walk found invalid mimetype text/plain for /media/HDIMAGES/2012/2012-06/2012-06-02-anniversaire-calie-6-ans/_MAY0912.NEF.xmp
DEBUG:phraymd core:Directory walk found invalid mimetype text/plain for /media/HDIMAGES/2012/2012-06/2012-06-05/_MAY1046.NEF.xmp
DEBUG:phraymd core:Directory walk found invalid mimetype text/plain for /media/HDIMAGES/2012/2012-06/2012-06-05/_MAY1047.NEF.xmp
DEBUG:phraymd core:Directory walk found invalid mimetype text/plain for /media/HDIMAGES/2012/2012-06/2012-06-05/_MAY1051.NEF.xmp
DEBUG:phraymd core:Directory walk complete for /media/HDIMAGES/2012
running verify job
verify loading metadata /media/HDIMAGES/2012/2012-05/2012-05-13-calie-emilie-provence-coquelicot/a__MAY9801.xcf
****REMOVING /media/HDIMAGES/2012/2012-05/2012-05-13-calie-emilie-provence-coquelicot/a__MAY9801.xcf
Error reading metadata for /media/HDIMAGES/2012/2012-05/2012-05-13-calie-emilie-provence-coquelicot/a__MAY9801.xcf
Traceback (most recent call last):
  File "/home/nico/UTILITAIRES/phraymd/phraymd/modules/phraymd/metadata2.py", line 65, in load_metadata
    rawmeta.read()
  File "/usr/lib/python2.7/dist-packages/pyexiv2/metadata.py", line 107, in read
    self.__image = self._instantiate_image(self.filename)
  File "/usr/lib/python2.7/dist-packages/pyexiv2/metadata.py", line 79, in _instantiate_image
    return libexiv2python._Image(filename)
IOError: /media/HDIMAGES/2012/2012-05/2012-05-13-calie-emilie-provence-coquelicot/a__MAY9801.xcf: The file contains data of an unknown image type

####adding /media/HDIMAGES/2012/2012-05/2012-05-13-calie-emilie-provence-coquelicot/a__MAY9801.xcf
INFO:phraymd core:Image verification complete
----------------------------------------------------------------------------
Let me know if this output is good for you or if you need more informations.

>One more thing. Can you post last error message when running rev 558 (should be the one related to the tag_cloud)

I'm now on rev 563. The "tag cloud" message error seems to have disapear.
And my 2011 collection (which was the initial problem) load perfectly. The errors about .xcf still appearing in the terminal

----------------------
####adding /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Ryan/Ryan-01.xcf
verify loading metadata /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Thib-Jul/Thib-Jul-01.xcf
****REMOVING /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Thib-Jul/Thib-Jul-01.xcf
Error reading metadata for /media/HDIMAGES/2011/2011-11/2011-11-calendrier-AVB/Thib-Jul/Thib-Jul-01.xcf
Traceback (most recent...

[dmoore (damien-moore)]

> Well, it looks like the images are missing in the collection. But only
the JPG files. NEF files still presents.
> Here is the output when I open this collection (there are a lot of
mimetype error before the one posted) :
[...SNIP...]
> IOError:
/media/HDIMAGES/2012/2012-05/2012-05-13-calie-emilie-provence-coquelicot/a__MAY9801.xcf:
The file contains data of an unknown image type
>
> ####adding
/media/HDIMAGES/2012/2012-05/2012-05-13-calie-emilie-provence-coquelicot/a__MAY9801.xcf
> INFO:phraymd core:Image verification complete
>
----------------------------------------------------------------------------
> Let me know if this output is good for you or if you need more
informations.

Is there any output related to the JPG files?

>>One more thing. Can you post last error message when running rev 558
> (should be the one related to the tag_cloud)
>
> I'm now on rev 563. The "tag cloud" message error seems to have disapear.
> And my 2011 collection (which was the initial problem) load perfectly.
The errors about .xcf still appearing in the terminal
> And there is a problem with the tag "mise à mort"
> --------
[...SNIP...]
> Warning: bad decode of tag mise ? mort
>
> (phraymd:26923): Gtk-WARNING **: Failed to set text from markup due to
error parsing markup: Erreur à la ligne 1, caractère 28 : Codage UTF-8 non
valide dans le nom - « mise \xe0 mort (56/56) » n'est pas valide

This is the only tag that is bad?

> This tag still present selectable in the tag cloud.

That's a good thing, but I'd like to come up with a better solution than
what I have in the code now. Having badly formed unicode floating around
has potential to cause all sorts of problems. (e.g. what happens when you
modify the metadata and try to write changes)

> Don't hesitate to ask me more informations or tests. I still making tests
with Phraymd as I think it could be the soft I was looking for.

Thanks. I do use it to manage my own growing collection, but don't have the
time to make it as clean, fully featured and bug free as I would like. I'm
still ironing out bugs from changes I made 6 months ago. (So keep
updating!!) If there are specific features you need I suggest filing bugs
for them. They help me prioritize. Stuff that is easy gets done quickly.

[LCG (le-coude-gauche)]

> Is there any output related to the JPG files?

No I can't see any. But error related to mimetype for .xmp files are numerousous. So I can't see the first output when I open a collection. (this messages were not there when I open this bug.

> This is the only tag that is bad?

This is the only one with an output. But I never used accent in my tags. This one is probably the only one with accent.

> That's a good thing, but I'd like to come up with a better solution than
> what I have in the code now. Having badly formed unicode floating around
> has potential to cause all sorts of problems. (e.g. what happens when you
> modify the metadata and try to write changes)

Adding tag on a .NEF file = the tag is added. in the file, in the tag cloud ans is readable by other software like Geeqie.
Here is the output
-------------------------------------------------------------------
(phraymd:19701): Gtk-WARNING **: Failed to set text from markup due to error parsing markup: Erreur à la ligne 1, caractère 28 : Codage UTF-8 non valide dans le nom - « mise \xe0 mort (56/56) » n'est pas valide
app_meta {'SensingMethod': 2, 'FNumber': (28L, 5L), 'Copyright': '(c)Tous Droits Reserves ', 'Artist': 'Nicolas Mayer ', 'FocalLength': (70L, 1L), 'Make': 'NIKON CORPORATION', 'MeteringMode': 2, 'IsoSpeed': 3200, 'Model': 'NIKON D700', 'ExposureBiasValue': (0L, 1L), 'Software': 'Ver.1.01 ', 'ExposureTime': (1L, 400L), 'Orientation': 1, 'ExposureProgram': 3, 'Sharpness': 2, 'DateTaken': datetime.datetime(2011, 3, 17, 22, 0, 21), 'GainControl': '2', 'Keywords': ['cirque', 'groupe'], 'DigitalZoomRatio': '1'}
Exiv2 set data failure Artist
Traceback (most recent call last):
  File "/home/nico/UTILITAIRES/phraymd/phraymd/modules/phraymd/metadata2.py", line 506, in set_exiv2_meta
    data[2](exiv2_meta,data[6],app_meta[appkey]) ##todo: only set values that have actually changed?? e.g. could check data[1], but for e.g. orientation should be savable but data[1]=false
  File "/home/nico/UTILITAIRES/phraymd/phraymd/modules/phraymd/metadata2.py", line 255, in conv_str
    metaobject[keys[0]]=value
  File "/home/nico/UTILITAIRES/phraymd/phraymd/modules/phraymd/metadata2.py", line 48, in __setitem__
    value=pyexiv2.XmpTag(key,value)
  File "/usr/lib/python2.7/dist-packages/pyexiv2/xmp.py", line 114, in __init__
    self._set_value(value)
  File "/usr/lib/python2.7/dist-packages/pyexiv2/xmp.py", line 222, in _set_value
    raise TypeError('Expecting a list of values')
TypeError: Expecting a list of values

Exiv2 set data failure GainControl
Traceback (most recent call last):
  File "/home/nico/UTILITAIRES/phraymd/phraymd/modules/phraymd/metadata2.py", line 506, in set_exiv2_meta
    data[2](exiv2_meta,data[6],app_meta[appkey]) ##todo: only set values that have actually changed?? e.g. could check data[1], but for e.g. orientation should be savable but data[1]=false
  File "/home/nico/UTILITAIRES/phraymd/phraymd/modules/phraymd/metadata2.py", line 255, in conv_str
    metaobject[keys[0]]=value
  File "/home/nico/UTILITAIRES/phraymd/phraymd/modules/phraymd/metadata2.py", line 41, in __setitem...

[dmoore (damien-moore)]

Three follow-ups:

#1 In your error messages above I noticed that phraymd was trying to
write the metadata for two fields incorrectly:
* Artist information (iptc Credit / xmp Creator -- phraymd tries to
write to both fields)
* The Exif field GainControl (phraymd was trying to write a text value
instead of a number, but perhaps phraymd shouldn't be writing anything
at all if nothing has changed)
I have committed some fixes to these bugs.

#2 On the original issue caused by problems with unicode, have you
tried to add a tag with an accent in phraymd? Try saving the change,
then right click on the thumbnail and click "Reload metadata" and
report any output from all of these steps

#3 Do you still have the issue with your 2012 JPEGs? Perhaps we could
open a new bug for that one.

[LCG (le-coude-gauche)]

Hello,
sorry for my late response.

#1 OK.

#2 I tried to add the tag "filé" to a jpg file. Apprently no problem. Here is the output :
------
app_meta {'DateTaken': datetime.datetime(2010, 12, 19, 12, 32, 8), 'Keywords': [u'volley', u'Couleur', u'Sport', u'Avignon', u'gymnase', u'avb', u'interieur', u'Reportage', 'fil\xc3\xa9'], 'Processing Software': 'digiKam-1.2.0', 'Software': 'digiKam-1.2.0'}
DEBUG:phraymd core:file event MODIFY on /media/HDIMAGES/2011/2011-05/2011-05-dvd-avb/2010-12-18-AVB-Saint-Brieuc/AVB-Saint-Brieuc-011.jpg5719
DEBUG:phraymd core:file event MODIFY on /media/HDIMAGES/2011/2011-05/2011-05-dvd-avb/2010-12-18-AVB-Saint-Brieuc/AVB-Saint-Brieuc-011.jpg5719
DEBUG:phraymd core:file event MODIFY on /media/HDIMAGES/2011/2011-05/2011-05-dvd-avb/2010-12-18-AVB-Saint-Brieuc/AVB-Saint-Brieuc-011.jpg5719
DEBUG:phraymd core:file event MODIFY on /media/HDIMAGES/2011/2011-05/2011-05-dvd-avb/2010-12-18-AVB-Saint-Brieuc/AVB-Saint-Brieuc-011.jpg5719
DEBUG:phraymd core:file event MODIFY on /media/HDIMAGES/2011/2011-05/2011-05-dvd-avb/2010-12-18-AVB-Saint-Brieuc/AVB-Saint-Brieuc-011.jpg5719
DEBUG:phraymd core:file event MODIFY on /media/HDIMAGES/2011/2011-05/2011-05-dvd-avb/2010-12-18-AVB-Saint-Brieuc/AVB-Saint-Brieuc-011.jpg5719
DEBUG:phraymd core:file event MODIFY on /media/HDIMAGES/2011/2011-05/2011-05-dvd-avb/2010-12-18-AVB-Saint-Brieuc/AVB-Saint-Brieuc-011.jpg5719
DEBUG:phraymd core:file event MODIFY on /media/HDIMAGES/2011/2011-05/2011-05-dvd-avb/2010-12-18-AVB-Saint-Brieuc/AVB-Saint-Brieuc-011.jpg5719
DEBUG:phraymd core:file event MODIFY on /media/HDIMAGES/2011/2011-05/2011-05-dvd-avb/2010-12-18-AVB-Saint-Brieuc/AVB-Saint-Brieuc-011.jpg5719
DEBUG:phraymd core:file event MODIFY on /media/HDIMAGES/2011/2011-05/2011-05-dvd-avb/2010-12-18-AVB-Saint-Brieuc/AVB-Saint-Brieuc-011.jpg5719
DEBUG:phraymd core:file event MODIFY on /media/HDIMAGES/2011/2011-05/2011-05-dvd-avb/2010-12-18-AVB-Saint-Brieuc/AVB-Saint-Brieuc-011.jpg5719
DEBUG:phraymd core:file event MODIFY on /media/HDIMAGES/2011/2011-05/2011-05-dvd-avb/2010-12-18-AVB-Saint-Brieuc/AVB-Saint-Brieuc-011.jpg5719 (THIS LINE APPEARS A LOT OF TIME)
[...]
DEBUG:phraymd core:file event MODIFY on /media/HDIMAGES/2011/2011-05/2011-05-dvd-avb/2010-12-18-AVB-Saint-Brieuc/AVB-Saint-
INFO:phraymd core:Deferred directory monitor event
INFO:phraymd core:deleting /media/HDIMAGES/2011/2011-05/2011-05-dvd-avb/2010-12-18-AVB-Saint-Brieuc/AVB-Saint-Brieuc-011.jpg
INFO:phraymd core:deleting /media/HDIMAGES/2011/2011-05/2011-05-dvd-avb/2010-12-18-AVB-Saint-Brieuc/AVB-Saint-Brieuc-011.jpg5719
Warning: bad decode of tag mise � mort

(phraymd:5719): Gtk-WARNING **: Failed to set text from markup due to error parsing markup: Erreur à la ligne 1, caractère 28 : Codage UTF-8 non valide dans le nom - « mise \xe0 mort (56/56) » n'est pas valide
Traceback (most recent call last):
----

#3 Everything seems to be OK.
If the problem persists/reappears I will create a new bug.