Adding Books rules for tags adding incorrect tags

[[e-mail]]

In Preferences - Adding Books - Create rules to filter/transform tags.

I have set up some rules:

An Example - 'Replace the tag, if it matches pattern: biog*
with the tag: Biography

When I run 'download metadata in bulk' the above tag ends up biographyphyphyphpyphyp, and variants. Some of the residual original tag info is added to the end of the long line, no spaces are copied or added.

I have deleted the rules and replaced them, ensuring I was not including any accidentally hidden characters. I have 11 rules, all of the Replace if it matches pattern type.

I am sorry, but as I don't know what the original tags were, I cannot be certain this happens with every rule.

I have restarted the PC, restarted the app, and run the process more than once. There was no change in the behavior.

I am using 2.50.1, Windows 7 64 bit

 Plug ins are Library Codes, MultiColumnSearch and Search the INternet, View Manager

Metadata source plugins are Fantastic Fiction, Fiction DB and Goodreads.

All are up to date

[[e-mail] (shirley-elrick)]

I am the reporter of this bug. I did some further 'tests'

1 - I removed all rules and I still see the same errors.

2 - I deleted as metadata sources all except Goodreads and Fantastic Fiction and I still see the same errors

I have configured the goodreads source (Preferences - Plugins - Metadata Source Plugins Goodreads (1.1.10) by Grant Drake - customize Plugin)

all fields are ticked, and I have 56 mappings, but none appear to be the cause of the incorrect tags.

As a test, using just the one book to download individual metadata I did the following

restarted Calibre
checked to make sure there were no mapping rules in 'Create rules to filter/transform tags.'
Enabled JUST Goodreads in the source of metadata preferences
NO CHANGE

I repeated the above with JUST Fantastic Fiction enabled
NO CHANGE

I have attached a screen print showing the suggestions window.

[[e-mail] (shirley-elrick)]

This is the reporter of the bug - again.

Just to reiterate, the error I am seeing HAS TO BE related to some rules I recently added. The first and only time I have ever used the tag "Crime Mystery Thriller" was a rule I set up recently.

The errors occurred the first time I added books in bulk, right after setting up the above rule. One of about a dozen, most of which cause errors in generated tags

I deleted all rules, and downloaded more metadata on individual books, AND I STILL SAW THE SAME ERROR.

I restarted Calibre, no change.

This does not make sense to me. The errors HAVE TO BE related to the rules I set up, but the error is still occurring after I have deleted all those rules.

I also have mappings set up in the Goodreads metadata source, BUT I MAKE NO REFERENCE TO or MENTION of the tag 'Crime Mystery Thriller'

I rebooted the PC, then ran the get metadata from source on the same book (I did not previously save the downloaded metadata for this one 'test' book). The same error occurred.

This last test was run with Calibre in debug mode. The log file -

calibre Debug log
calibre 2.50.1 [64bit] isfrozen: True is64bit: True
Windows-7-6.1.7601-SP1 Windows ('64bit', 'WindowsPE')
('Windows', '7', '6.1.7601')
Python 2.7.9
Windows: ('7', '6.1.7601', 'SP1', 'Multiprocessor Free')
Successfully initialized third party plugins: Fantastic Fiction && Quality Check && Find Duplicates && Search The Internet && Consolidate All Library Metadata && View Manager && Goodreads && MultiColumnSearch && FictionDB && Fantastic Fiction Adults && Library Codes && Reading List
Starting up...
Started up in 14.72 seconds with 3647 books

This is NOT a show stopper, as the resulting incorrect tags are easily edited. My multiple posts are in the hope of assisting you with your diagnosis, and are NOT an indication of the degree of impact of the 'bug'.

SHIRLEY

[Kovid Goyal (kovid)]

Umm as far as I can tell, this appears to be a bug in the Goodreads plugin. I cannot replicate it using just the builtin metadata sources (Amazon, google) for the book The Killing Circle by Chris Wiltz.

I dont provide support for third party plugins, you should post in the plugins thread of the mobileread.com calibre forum, where hopefully the plugin author will be able to help you.

Also note that regarding your first post. Your rule expression is incorrect. The rules engine uses regular expressions. See http://manual.calibre-ebook.com/regexp.html

I'm not sure what you were trying to express, you want words that start with biog to be replaced by Biology? If so the correct match expression is biog.* -- note the period before the asterisk.

When you create tag mapping rules, it is useful to use the Test Rules button to see what the rules you have created will do to a particular tag.

[[e-mail] (shirley-elrick)]

Thank you for replying. I will follow your suggestion

SHIRLEY

On 3 February 2016 at 14:43, Kovid Goyal <email address hidden> wrote:

> Umm as far as I can tell, this appears to be a bug in the Goodreads
> plugin. I cannot replicate it using just the builtin metadata sources
> (Amazon, google) for the book The Killing Circle by Chris Wiltz.
>
> I dont provide support for third party plugins, you should post in the
> plugins thread of the mobileread.com calibre forum, where hopefully the
> plugin author will be able to help you.
>
> Also note that regarding your first post. Your rule expression is
> incorrect. The rules engine uses regular expressions. See http://manual
> .calibre-ebook.com/regexp.html
>
> I'm not sure what you were trying to express, you want words that start
> with biog to be replaced by Biology? If so the correct match expression
> is biog.* -- note the period before the asterisk.
>
> When you create tag mapping rules, it is useful to use the Test Rules
> button to see what the rules you have created will do to a particular
> tag.
>
> ** Changed in: calibre
> Status: New => Invalid
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1540848
>
> Title:
> Adding Books rules for tags adding incorrect tags
>
> Status in calibre:
> Invalid
>
> Bug description:
> In Preferences - Adding Books - Create rules to filter/transform tags.
>
> I have set up some rules:
>
> An Example - 'Replace the tag, if it matches pattern: biog*
> with the tag: Biography
>
> When I run 'download metadata in bulk' the above tag ends up
> biographyphyphyphpyphyp, and variants. Some of the residual original
> tag info is added to the end of the long line, no spaces are copied or
> added.
>
> I have deleted the rules and replaced them, ensuring I was not
> including any accidentally hidden characters. I have 11 rules, all of
> the Replace if it matches pattern type.
>
> I am sorry, but as I don't know what the original tags were, I cannot
> be certain this happens with every rule.
>
> I have restarted the PC, restarted the app, and run the process more
> than once. There was no change in the behavior.
>
> I am using 2.50.1, Windows 7 64 bit
>
> Plug ins are Library Codes, MultiColumnSearch and Search the
> INternet, View Manager
>
> Metadata source plugins are Fantastic Fiction, Fiction DB and
> Goodreads.
>
> All are up to date
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/calibre/+bug/1540848/+subscriptions
>

--
Happy Hunting - Shirley - <email address hidden> -
http://www.shirley-elrick.com

[David Forrester (davidfor)]

Kovid,

I think this is in the way the tag mapper rules are applied. Looking at the code, the rules are reapplied to the result of the mapping up 20 times. If the result matches one of the rules, then we get the result that Shirley is reporting.

My testing was done using the tester in the tag mapper dialog, so it isn't just Goodreads plugin.

[Kovid Goyal (kovid)]

Note that the correct expression, as I posted before is biog.* not biog*
and that results in biography->biography

[[e-mail] (shirley-elrick)]

Kovid - I am the reporter of the original bug.

If I may just restate what my 'current' issue is, it may assist.

In preferences, plugins, metadata source, Goodreads 1.1.11, customise - I have set up some mappings to Calibre Tag(s)

One example is Biography > Biographies Autobiographies & Memoirs.

There are no wild cards used.

When I download metadata from Goodreads for Poe: A Life Cut Short I get Biographygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphies AutoBiographygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphies & Memoirs, Non-Fiction, History.

The above is just an example - it happens consistently on all metadata downloads for all the effected mappings.

Thanks Shirley

[[e-mail] (shirley-elrick)]

David - I notice that the Bug Report has not been updated since I made my
entry. In your opinion, Has this been overlooked for a technical reason
(ie marked as invalid)?

If yes, is there something I should be doing to get it re-assessed?

I am happy to wait, is the 'delay' is normal and not an oversight

SHIRLEY

On 16 February 2016 at 23:04, David Forrester <email address hidden>
wrote:

> Kovid,
>
> I think this is in the way the tag mapper rules are applied. Looking at
> the code, the rules are reapplied to the result of the mapping up 20
> times. If the result matches one of the rules, then we get the result
> that Shirley is reporting.
>
> My testing was done using the tester in the tag mapper dialog, so it
> isn't just Goodreads plugin.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1540848
>
> Title:
> Adding Books rules for tags adding incorrect tags
>
> Status in calibre:
> Invalid
>
> Bug description:
> In Preferences - Adding Books - Create rules to filter/transform tags.
>
> I have set up some rules:
>
> An Example - 'Replace the tag, if it matches pattern: biog*
> with the tag: Biography
>
> When I run 'download metadata in bulk' the above tag ends up
> biographyphyphyphpyphyp, and variants. Some of the residual original
> tag info is added to the end of the long line, no spaces are copied or
> added.
>
> I have deleted the rules and replaced them, ensuring I was not
> including any accidentally hidden characters. I have 11 rules, all of
> the Replace if it matches pattern type.
>
> I am sorry, but as I don't know what the original tags were, I cannot
> be certain this happens with every rule.
>
> I have restarted the PC, restarted the app, and run the process more
> than once. There was no change in the behavior.
>
> I am using 2.50.1, Windows 7 64 bit
>
> Plug ins are Library Codes, MultiColumnSearch and Search the
> INternet, View Manager
>
> Metadata source plugins are Fantastic Fiction, Fiction DB and
> Goodreads.
>
> All are up to date
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/calibre/+bug/1540848/+subscriptions
>

--
Happy Hunting - Shirley - <email address hidden> -
http://www.shirley-elrick.com

[Kovid Goyal (kovid)]

There is no bug. You need to use the pattern biog.* not biog*

[[e-mail] (shirley-elrick)]

Kovid - I am really sorry that my original bug report is misleading you.

If I may just quote my update on the 17th - In preferences, plugins, metadata source, Goodreads 1.1.11, customise - I have set up some mappings to Calibre Tag(s)

One example is Biography > Biographies Autobiographies & Memoirs.

There are no wild cards used.

When I download metadata from Goodreads for Poe: A Life Cut Short I get Biographygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphies AutoBiographygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphygraphies & Memoirs, Non-Fiction, History.

The above is just an example - it happens consistently on all metadata downloads for all the effected mappings.

END QUOTE

There are no rules set up in the Preferences/Adding Books/create rules to transform....

I will of course gladly create a new bug report omitting all reference to the Adding Book process in preferences. Just tell me if this would be better option for you.

Again - I am really sorry this is being such a saga....I fully understand it is being caused by my poor reporting in the first instance.

SHIRLEY

[Kovid Goyal (kovid)]

Mappings you setup in the goodreads plugin are nothing to do with
calibre, you need to follow up on those with the goodreads plugin
developer.

I can only address problems caused by mappings created in
Preferences->Adding Books->Create rules to filter tags

[[e-mail] (shirley-elrick)]

OK - consider this closed from my point of view as well as yours.

SHIRLEY