OpenLP importer can't handle new song songbook DB structure (AttributeError: 'OldSong' object has no attribute 'book')

[Azaziah]

Importing songs that are using the new database format via import menu > OpenLP 2 or First Time Wizard causes traceback.

Database structure has been changed from 2.2.1
2.2.1 songs.sqlite does not have a table named "songs_songbooks"
Songbook details are stored in “songs” table with fields:
“song_book_id” and “song_number”

The problem seems to be that the importer tries to look for the
songbook data from the "songs" table.

This traceback appears on 2.4.4 when trying to import a 2.4 >
database into OpenLP 2.4 >

Traceback (most recent call last):
  File "openlp\core\ui\wizard.py", line 216, in on_current_id_changed
  File "openlp\plugins\songs\forms\songimportform.py", line 351, in perform_wizard
  File "openlp\plugins\songs\lib\importers\openlp.py", line 224, in do_import
AttributeError: 'OldSong' object has no attribute 'book'

This also means 2.4> databases are not compatible with 2.2.1. <

Is there a benefit in using the new db structure or should this be reverted?

If 2.4> database is imported to 2.2.1 this traceback appears
and OpenLP crashes with "Fatal error! Openlp returned -1"

Traceback (most recent call last):
File "c:\Python33\lib\site-packages\sqlalchemy\orm\relationships.py", line 1932, in _determine_joins
File "<string>", line 2, in join_condition
File "c:\Python33\lib\site-packages\sqlalchemy\sql\selectable.py", line 762, in _join_condition
sqlalchemy.exc.NoForeignKeysError: Can't find any foreign key relationships between 'songs' and 'song_books'.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "<string>", line 44, in <module>
File "D:\OpenLP_Development\OpenLP_Code\2.2\openlp\core\__init__.py", line 388, in main
File "D:\OpenLP_Development\OpenLP_Code\2.2\openlp\core\__init__.py", line 155, in run
File "D:\OpenLP_Development\OpenLP_Code\2.2\openlp\core\ui\mainwindow.py", line 668, in first_time
File "D:\bzr\Portable_builds\2_2_1\App\OpenLP\plugins\songs\songsplugin.py", line 317, in first_time
importer.do_import(progress)
File "D:\OpenLP_Development\OpenLP_Code\2.2\openlp\plugins\songs\lib\importers\openlp.py", line 154, in do_import
File "c:\Python33\lib\site-packages\sqlalchemy\orm\scoping.py", line 149, in do
File "c:\Python33\lib\site-packages\sqlalchemy\orm\session.py", line 1151, in query
File "c:\Python33\lib\site-packages\sqlalchemy\orm\query.py", line 106, in __init__
File "c:\Python33\lib\site-packages\sqlalchemy\orm\query.py", line 116, in _set_entities
File "c:\Python33\lib\site-packages\sqlalchemy\orm\query.py", line 149, in _set_entity_selectables
File "c:\Python33\lib\site-packages\sqlalchemy\orm\query.py", line 2987, in setup_entity
File "c:\Python33\lib\site-packages\sqlalchemy\util\langhelpers.py", line 712, in __get__
File "c:\Python33\lib\site-packages\sqlalchemy\orm\mapper.py", line 1856, in _with_polymorphic_mappers
File "c:\Python33\lib\site-packages\sqlalchemy\orm\mapper.py", line 2560, in configure_mappers
File "c:\Python33\lib\site-packages\sqlalchemy\orm\mapper.py", line 1673, in _post_configure_properties
File "c:\Python33\lib\site-packages\sqlalchemy\orm\interfaces.py", line 143, in init
File "c:\Python33\lib\site-packages\sqlalchemy\orm\relationships.py", line 1510, in do_init
File "c:\Python33\lib\site-packages\sqlalchemy\orm\relationships.py", line 1586, in _setup_join_conditions
File "c:\Python33\lib\site-packages\sqlalchemy\orm\relationships.py", line 1849, in __init__
File "c:\Python33\lib\site-packages\sqlalchemy\orm\relationships.py", line 1953, in _determine_joins
sqlalchemy.exc.NoForeignKeysError: Could not determine join condition between parent/child tables on relationship OldSong.book - there are no foreign keys linking these tables. Ensure that referencing columns are associated with a ForeignKey or ForeignKeyConstraint, or specify a 'primaryjoin' expression.

[Tomas Groth (tomasgroth)]

Which version of OpenLP was used to create the database?

[Laura Ekstrand (ldeks)]

I had the same problem. The database was made with trunk (17 Nov 2016) on Fedora 23. When I upgraded to Fedora 25 today, trunk did not contain the song database, and importing it with OpenLP 2 (.sqlite) format did not work (I've attached the error log).

I have a patch to fix this, but I'm new to developing for OpenLP, so I haven't sent it yet:

=== modified file 'openlp/plugins/songs/lib/importers/openlp.py'
--- openlp/plugins/songs/lib/importers/openlp.py 2016-05-27 08:13:14 +0000
+++ openlp/plugins/songs/lib/importers/openlp.py 2016-12-10 02:56:07 +0000
@@ -221,11 +221,14 @@
                     if not existing_book:
                         existing_book = Book.populate(name=entry.songbook.name, publisher=entry.songbook.publisher)
                     new_song.add_songbook_entry(existing_book, entry.entry)
- elif song.book:
- existing_book = self.manager.get_object_filtered(Book, Book.name == song.book.name)
- if not existing_book:
- existing_book = Book.populate(name=song.book.name, publisher=song.book.publisher)
- new_song.add_songbook_entry(existing_book, '')
+ else:
+ try:
+ existing_book = self.manager.get_object_filtered(Book, Book.name == song.book.name)
+ if not existing_book:
+ existing_book = Book.populate(name=song.book.name, publisher=song.book.publisher)
+ new_song.add_songbook_entry(existing_book, '')
+ except(AttributeError):
+ pass
             # Find or create all the media files and add them to the new song object
             if has_media_files and song.media_files:
                 for media_file in song.media_files:

The problem, as I see it, is that "elif song.book:" is not the proper way to check if song has a book attribute because if song doesn't have book, the call will throw AttributeError. Simply catching and ignoring the error worked for me, but that might not be the correct solution.

[Laura Ekstrand (ldeks)]

Sorry about the lack of code formatting, I didn't realize it pasted like that. I've attached the patch for you.

[Phill (phill-ridout)]

It would appear Raoul fixed this in: http://bazaar.launchpad.net/~openlp-core/openlp/trunk/revision/2728

[Raoul Snyman (raoul-snyman)]

It was fixed in 2.4 too: https://code.launchpad.net/~raoul-snyman/openlp/fix-importers-2.4/+merge/318838