Kopete does not import pidgin history

[Maddev]

Binary package hint: kdenetwork

Pidgin package is 2.5.5-1ubuntu8.
The attachment contains screenshot of the result of import which contains nicknames and date but not contains messages.

ProblemType: Bug
Architecture: amd64
DistroRelease: Ubuntu 9.04
NonfreeKernelModules: fglrx
Package: kopete 4:4.2.85-0ubuntu1~jaunty1~ppa2
ProcEnviron:
 LANGUAGE=
 PATH=(custom, no user)
 LANG=ru_UA.UTF-8
 SHELL=/bin/bash
SourcePackage: kdenetwork
Uname: Linux 2.6.28-12-generic x86_64
UnreportableReason: Это не оригинальный пакет Ubuntu

[Björn Schöpe (bjoern-schoepe)]

Version: (using KDE KDE 3.4.1)
Installed from: Gentoo Packages
Compiler: GCC 3.4.4
OS: Linux

I think a useful feature of the history plugin might be an import- and export-function as it offers Psi with a simple "export..." button in the history window. this way one could import p.e. from gaim or psi histories or export them from kopete (when switching from one client to another or when using kopete on different boxes). Editing directly the log file is quite a lot of work just to have a complete history.

[W-kde (w-kde)]

i would really appreciate importing logs especially from centericq, sim and trillian (because i have about 1,5 years of logs in that formats) :)

exporting has a much lower priority IMHO :)

[VF (vfiend)]

The ability to import Gaim logs in particular would be terrific, especially since it's used by so many people, even lots of KDE users.

[VF (vfiend)]

*** This bug has been confirmed by popular vote. ***

[Philip Rodrigues (phil-kde)]

*** Bug 108305 has been marked as a duplicate of this bug. ***

[Kde-bugz (kde-bugz)]

I've just moved from Gaim to Kopete, and a function to import my chat history from Gaim would be fantastic. Even if it was just a script to drag the data in that would be sufficient.

The dupe mentions use of xsltproc to do a conversion manually, but it appears likely to be non-trivial to come up with a solution using that (unless someone else has a clue at hand).

[chefe (hullemulle)]

in bug report 108305, it reads at the beginning: The program you have to use to convert the logs is called "xsltproc" and comes with your libxsl package.

can anyone please elaborate on how to do this? any help would be greatly appreciated!
best regards

[Philip Rodrigues (phil-kde)]

kopeteUSER, see "A simple Kopete History->HTML convertor" on this page: http://people.fruitsalad.org/phil/kde/

[chefe (hullemulle)]

thank you for your reply philip. but is there a way "the other way round", that is, to convert gaim logs (html) into kopete-format-logs?

[Philip Rodrigues (phil-kde)]

It's certainly possible, and probably quite easy, but I don't have any gaim logs to work on, so I can't do it myself.

[chefe (hullemulle)]

hi philip. unfortunately i do not know anything about programing/coding/scripting, so i believe you that it should be quite easy. if you are interested in helping out, not only me of course ;-) afaik there are many many more who wish for such an option, i would be happy to email you some logs to practice on.

btw: this is my first try to seek help the "direct" way, that is through bug reports/mailing lists. and i am very impressed about how fast there are competent an willing users raising there hands in comparison to numerous "more popular" (e.g. ubuntuforums.org, ubuntuusers.de) message boards.

have a nice day!

marcel

[Philip Rodrigues (phil-kde)]

If you can send me some gaim logs as an example, I might have some time to produce something, although I can't promise :-)

[chefe (hullemulle)]

cool. email is out. thank you for everything you can do.
marcel

[thoand (michael-schoenbeck)]

Philip, are there some advances?

[Philip Rodrigues (phil-kde)]

I've taken a look at the gaim log format, and it doesn't seem great as far as converting to kopete log format is concerned - at least, I don't think it can be done easily with XSLT. It'll probably need some Perl or something, which is beyond my abilities, I'm afraid.

[thoand (michael-schoenbeck)]

would it be an idea to write a plugin for kopete to save logs like pidgin does, or the other way round? I have no idea how complicate it is. Is there a way to donate for special feature requests?

[Chain (chain)]

I think a plugin storing the messages in the pidgin HTML-format would be quite nice. But I don't think anyone is going to do that^^;

[Philip Rodrigues (phil-kde)]

You mean like this?: http://bram85.blogspot.com/2007/07/export-your-kopete-history-to-text-or.html

[thoand (michael-schoenbeck)]

no, not just an export function, I thought of a bidirectional way.

[Dennis Schridde (devurandom)]

I would like to have at least the export function, too. Possibly with a little bit nicer stylesheet than Brams's (more Kopete-style).

[jonathan (jonathan-schluessler)]

I'm using Pidgin at the moment but I'm thinking about changing to Kopete. When I've checked Kopete 3.5 years ago and compared it to Pidgin (back then still named gaim), my decision was for Pidgin.
But in the last years Kopete has really improved so I would like to use Kopete as my main chat-program.
My Problem is that I have 3.5 years of important chat logs which I definitively don't want to loose. So a way to import my Pidgin logs to Kopete is a must have for me.
Are there any news or some kind of workaround?

[Timo (timo-schluessler)]

I wrote an extension to the kopete history plugin so that you can import the history from pidgin/purple. Until now my code wasn't checked in, but you can try the patches and the code from my email to the kopete-devel mailinglist: http://lists.kde.org/?l=kopete-devel&m=122832025716577&w=2.

Even when nobody told me if my extension works for him, it does a very nice job for me. use it with care and tell me about bugs :)

regards

[Matt Rogers (mattr-kde)]

SVN commit 909147 by mattr:

Add the ability to import Pidgin logs into Kopete.

Patch by Timo Schlüßler. Thanks!

This will be in KDE 4.3. Please note that this is not a generic history
import feature. It only handles Pidgin logs.
CCBUG: 109942
CCMAIL:<email address hidden>

 M +2 -1 CMakeLists.txt
 M +8 -0 historydialog.cpp
 M +2 -0 historydialog.h
 A historyimport.cpp [License: GPL (v2+)]
 A historyimport.h [License: GPL (v2+)]
 M +6 -4 historylogger.cpp
 M +8 -1 historyviewer.ui

WebSVN link: http://websvn.kde.org/?view=rev&revision=909147

[A-null (a-null)]

Thanks for adding the pidgin import feature. I was thinking of using Kopete, eagerly looking forward to KDE 4.3 !!

[Maddev (khulap)]

This patch to the history plugin helps me to solve this problem.

[Yuriy Kozlov (yuriy-kozlov)]

https://bugs.launchpad.net/kdenetwork/+bug/378428 reports that this doesn't quite work right in the beta and provides a patch

[Timo (timo-schluessler)]

Thank you for reporting this bug and attaching the patch ...

Anyway I can't import my pidgin logs when using your patch. There are two problems (for .html log import):
1. My messages are surrounded by the </b></font> tags on the left and by the <br/> tag on the right. Regarding your code, it seems to me that your message starts _after_ the <br/> tag. Therefore the string beginMessage isn't found in line (historyimport.cpp::513) and so the resulting messageText is only rubbish.
2. I've got some cases where there are special formats inside the message. E.g. something like this:
<font color="#A82F2F"><font size="2">(22:12:33)</font> <b>Nick:</b></font> <font sml="AIM/ICQ">[MessageText]</font><br/>
In such cases your way of handling the logs doesn't filter out the <font ...> tag.

Don't you think it's better to use the QXmlStreamReader and adjust the code so that it supports your kind of pidgin logs? I was also asking myself how fast the use of QRegExp is in contrast to the xml-stream reader. (sorry for my poor english).

Can you tell me the structure of your logs? Or maybe supply me with some example logs? What a pity that pidgin's logs differ so much ....

Regards,
Timo

[Maddev (khulap)]

I attach the example of my html log. I hide all private information in it but the structure of log is unchanged.
I think that using QRegExp is better because the result parser is much simpler and clear to understand. I hope you have one message per line format too. The speed is not important because import of logs is operation that user performs only one time.
The RegExp parser is simple to adapt to your message format. For example, in your log <font color="#A82F2F"><font size="2">(22:12:33)</font> <b>Nick:</b></font> <font sml="AIM/ICQ">[MessageText]</font><br/> you need to correct beginMessage to "</b></font> " and replace text <font sml="AIM/ICQ"> and </font><br/> with "".
There is a problem with html links in the text body such as <a href='mailto:<email address hidden>'><email address hidden></a> but it can be solved with a regexp too.

[Timo (timo-schluessler)]

Then I would suggest to first replace <br/> by newlines and then filter out all lasting tags with a regexp like "<.*>" (the <font sml="AIM/ICQ"> tag is not in all of my pidgin-logs and may be different in logs from other people ...). Or do you have another idea?
It seems to me that kopete doesn't store something like <a>-tags in its logs. Links are detected on the fly, so removing link-tags is no problem.

[Timo (timo-schluessler)]

It works now fine for both our logs.
But I have no idea how to parse named entities like &quot; without using a XML parser. Do you know how to do this easily?

[Timo (timo-schluessler)]

Think I've got it now. Would you please like to check it?

Maybe you also would like to run something like this
   grep -R "&.*;" . | grep -v "&amp;" | grep -v "&gt;" | grep -v "&apos;" | grep -v"&quot;" | grep -v "&lt;" | grep -v "Binary file"
in your pidgin log directory to check if there are other named entities, which are so far not regarded.

Regards,
Timo

[Jithin Emmanuel (jithin1987)]

I use kubuntu jaunty kde 4.3 beta 2 . How do I use this function? I cannot find any options in kopete.

[Timo (timo-schluessler)]

(In reply to comment #25)
> I use kubuntu jaunty kde 4.3 beta 2 . How do I use this function? I cannot find
> any options in kopete.

Go to the history viewer dialog (e.g. right-click on a buddy -> view history) and then click on the button in the top which says "import history".

Good luck,
Timo

[Jonathan Thomas (echidnaman)]

Incomplete pending feedback.

[Groszdanireg (groszdanireg)]

A more general import/export feature would be nice too for backuping history or transfering it from an installation to another.

[Timo (timo-schluessler)]

(In reply to comment #27)
> A more general import/export feature would be nice too for backuping history or
> transfering it from an installation to another.

You could achieve a backup or transfer of different histories very easily through saving, moving or copying kopete's log folder (HOME/.kde/share/apps/kopete/logs).
I don't think that it would be necessary to have such a feature as part of the history plugin. Do you?

Greetings,
Timo

[Groszdanireg (groszdanireg)]

I do because you can only expect the most advanced users to find and copy files from Kopete's hidden configuration area by hand while this is a feature which even average users might want.

[GokulChittaranjan (gokulchittaranjan)]

  grep -R "&.*;" . | grep -v "&amp;" | grep -v "&gt;" | grep -v "&apos;" | grep -v"&quot;" | grep -v "&lt;" | grep -v "Binary file"
does not work.
I also have output that looks like
<font color="#16569E"><font size="2">(05:40:05 EST)</font> <b>Me:</b></font> yes<br/>
The date itself is encoded in the log file name.
Thanks.

[Timo (timo-schluessler)]

@Gokie: The grep command was only to check if there are entities in logs generated by pidgin that were not recognized by the import of kopete (in the state of the attached patch above).

Since yesterday the original problem is fixed in official kde sources and QXmlStreamReader is still used there, so we don't have any problems with named entities.
@Maddev and everybody else: Would you mind test it? (svn co svn://anonsvn.kde.org/home/kde/trunk/KDE/kdenetwork/kopete)

Regards, Timo

[Timo (timo-schluessler)]

@bug supervisor: this bug can be closed because it is fixed.

[Timo (timo-schluessler)]

(In reply to comment #24)
> https://bugs.launchpad.net/kdenetwork/+bug/378428 reports that this doesn't
> quite work right in the beta and provides a patch

That problem is now fixed in kopete-svn too.

[Psychonaut (psychonaut)]

The link in Comment #17 no longer works. The new URL for the Kopete history to HTML/text converter is http://www.bramschoenmakers.nl/en/node/136

[Juancho (juan-baptiste-0)]

This isn't working with kopete from KDE 4.5.2. When trying to import the history for one of my contacts I get an error window with title "Cannot Map Nickname to Account", and asking me which was my nickname on that history, but it starts to show random contacts from my list. For example, trying with a MSN contact, on the details text I see the following error:

WARNING: Cannot find matching account for <email address hidden> (msn).

Doing some testing, it seems the problem is with the name of the protocol directory on pidgin logs. For example, for MSN protocol, on ~/.purple/logs the directory is named "msn", but kopete directory on ~/.kde4/share/apps/kopete/logs is "WlmProtocol". If you rename pidgin's msn directory to "WlmProtocol" and then try to import the logs on kopete, the import process won't fail and it will start asking you which was your nickname on the conversations (which is very annoying btw, but it's a different issue), and apparently, it correctly imports your contacts.

I have other observations about this feature but probably they deserve their own bug report. If so then please tell me and I'll open them:

1. accents and other characters (á, é, í, ó, ñ, etc) appear screwed, there's some encoding issue on the conversion.
2. The import history feature is badly placed: you access it when looking at a contact's history, but when you click on the "import History" it will import all history for all contacts of all accounts (not sure of this last one but it'll surely will do it for all contacts of the contact's protocol). If I'm looking at a contact's history, I expect that that "Import History..." button imports _that_ user history, not the history from all contacts in all accounts. Thsi feature should be moved to kopete's configuration dialog or to the File dialog, bellow the "Export contacts" entry.
3. It's REALLY annoying that the import feature asks hundreds of times which was my nickname on the history logs. I suppose this is because it isn't easy to detect on a pidgin's conversation which contact is from the user importing the history and which one is from the person(s) he was talking to ?

[Timo (timo-schluessler)]

(In reply to comment #32)
> This isn't working with kopete from KDE 4.5.2. When trying to import the
> history for one of my contacts I get an error window with title "Cannot Map
> Nickname to Account", and asking me which was my nickname on that history, but
> it starts to show random contacts from my list. For example, trying with a MSN
> contact, on the details text I see the following error:
>
> WARNING: Cannot find matching account for <email address hidden> (msn).
>
> Doing some testing, it seems the problem is with the name of the protocol
> directory on pidgin logs. For example, for MSN protocol, on ~/.purple/logs the
> directory is named "msn", but kopete directory on
> ~/.kde4/share/apps/kopete/logs is "WlmProtocol". If you rename pidgin's msn
> directory to "WlmProtocol" and then try to import the logs on kopete, the
> import process won't fail and it will start asking you which was your nickname
> on the conversations (which is very annoying btw, but it's a different issue),
> and apparently, it correctly imports your contacts.

I just submitted a patch to the kopete people that takes different protocol-names into consideration.

> I have other observations about this feature but probably they deserve their
> own bug report. If so then please tell me and I'll open them:
>
> 1. accents and other characters (á, é, í, ó, ñ, etc) appear screwed, there's
> some encoding issue on the conversion.

Utf-8 is now set as codec for .txt file imports, which should be pidgins default (please alert me if it is not so!).

> 2. The import history feature is badly placed: you access it when looking at a
> contact's history, but when you click on the "import History" it will import
> all history for all contacts of all accounts (not sure of this last one but
> it'll surely will do it for all contacts of the contact's protocol). If I'm
> looking at a contact's history, I expect that that "Import History..." button
> imports _that_ user history, not the history from all contacts in all accounts.
> Thsi feature should be moved to kopete's configuration dialog or to the File
> dialog, bellow the "Export contacts" entry.

I agree. Let's see what the kopete people suggest ...

> 3. It's REALLY annoying that the import feature asks hundreds of times which
> was my nickname on the history logs. I suppose this is because it isn't easy to
> detect on a pidgin's conversation which contact is from the user importing the
> history and which one is from the person(s) he was talking to ?

That's right, only the .html pidgin-log-files contain valuable information about which nicknames are the importers ones.
The patch also contains a new algorithm that should fire much less queries about the used nicknames. Maybe you could test the import once again as soon as the patch made it into the trunk?

Timo

[enedene (enedene)]

Kubuntu 11.10, I can't import history from pidgin.

[Timo (timo-schluessler)]

Could you send an example of your pidgin logs? (Just dot out the real messages and names..)

[Pali (pali)]

*** Bug 227349 has been marked as a duplicate of this bug. ***