event_view changes requires schema upgrade

[Mikkel Kamstrup Erlandsen]

The recently merged changes to use journal_mode=WAL and adding some columns to event_view requires schema upgrades.

 1) Setting WAL journalling *permanently* makes the DB unreadable with sqlite 3.6 which is still widely deployed. Notably if I boot back into Ubuntu Lucid Lynx Zeitgeist will no longer work as lucid only ships sqlite 3.6.

 2) Contrary to what is stated in the merge request for the event_view changes the event_view VIEW is *not* regenerated on each startup. That is one of the whole points of doing schema versioning. This change makes Zeitgeist trunk unable to return any results if run on an older db. Strangely enough it doesn't raise any errors...

Guys - if you are making low level changes to our DB structure like both of these changes are, please make sure you understand all the ramifications before merging.

[Siegfried Gevatter (rainct)]

2010/9/19 Mikkel Kamstrup Erlandsen <email address hidden>:
>  2) Contrary to what is stated in the merge request for the event_view
> changes the event_view VIEW is *not* regenerated on each startup.

Actually they do, create_db() is executed every time at startup and it
has a DROP VIEW.

--
Siegfried-Angel Gevatter Pujals (RainCT)
Free Software Developer       363DEAE3

[Markus Korn (thekorn)]

Hi Mikkel,
I'm sorry, number two on your list is definitely my bad. I did not know that this "DROP VIEW [...]" line has no effect, and I also did not test my changes on an already existing database. My changes are indeed not working as I expect them to work.
I'll work on fixing the issue in bug 642803.

Can't say anything to number one on your list, because I was not involved into the process of deciding about using different options, but I raised the question about using our db versioning mechanisms here.

Markus

[Siegfried Gevatter (rainct)]

2010/9/19 Siegfried Gevatter <email address hidden>:
> Actually they do, create_db() is executed every time at startup and it
> has a DROP VIEW.
Err, scratch that.

[Seif Lotfy (seif)]

Hmmmmmmm I can't really figure out a good fix for this:
1) We revert the WAL change :(
2) We create a translation script that copies the DB content 1-to-1 in a new
db file in case something below sqlite 3.7 is used

On Sun, Sep 19, 2010 at 3:48 PM, Markus Korn <email address hidden> wrote:

> Hi Mikkel,
> I'm sorry, number two on your list is definitely my bad. I did not know
> that this "DROP VIEW [...]" line has no effect, and I also did not test my
> changes on an already existing database. My changes are indeed not working
> as I expect them to work.
> I'll work on fixing the issue in bug 642803.
>
> Can't say anything to number one on your list, because I was not
> involved into the process of deciding about using different options, but
> I raised the question about using our db versioning mechanisms here.
>
> Markus
>
> --
> WAL and event_view changes requires schema upgrade
> https://bugs.launchpad.net/bugs/642772
> You received this bug notification because you are subscribed to The
> Zeitgeist Project.
>
> Status in Zeitgeist Framework: New
>
> Bug description:
> The recently merged changes to use journal_mode=WAL and adding some columns
> to event_view requires schema upgrades.
>
> 1) Setting WAL journalling *permanently* makes the DB unreadable with
> sqlite 3.6 which is still widely deployed. Notably if I boot back into
> Ubuntu Lucid Lynx Zeitgeist will no longer work as lucid only ships sqlite
> 3.6.
>
> 2) Contrary to what is stated in the merge request for the event_view
> changes the event_view VIEW is *not* regenerated on each startup. That is
> one of the whole points of doing schema versioning. This change makes
> Zeitgeist trunk unable to return any results if run on an older db.
> Strangely enough it doesn't raise any errors...
>
> Guys - if you are making low level changes to our DB structure like both of
> these changes are, please make sure you understand all the ramifications
> before merging.
>
>
>

--
This is me doing some advertisement for my blog http://seilo.geekyogre.com

[Seif Lotfy (seif)]

I found the following statement in http://www.sqlite.org/wal.html under
Backwards Compatibility

One can explicitly change out of WAL mode using a pragma such as this:

PRAGMA journal_mode=DELETE;

Deliberately changing out of WAL mode changes the database file format
version numbers back to 1 so that older versions of SQLite can once again
access the database file.

I am uploading a test fix for this... Please tell me what you think

lp:~zeitgeist/zeitgeist/fix-642772

On Sun, Sep 19, 2010 at 4:05 PM, Siegfried Gevatter <email address hidden>wrote:

> 2010/9/19 Siegfried Gevatter <email address hidden>:
> > Actually they do, create_db() is executed every time at startup and it
> > has a DROP VIEW.
> Err, scratch that.
>
> --
> WAL and event_view changes requires schema upgrade
> https://bugs.launchpad.net/bugs/642772
> You received this bug notification because you are subscribed to The
> Zeitgeist Project.
>
> Status in Zeitgeist Framework: New
>
> Bug description:
> The recently merged changes to use journal_mode=WAL and adding some columns
> to event_view requires schema upgrades.
>
> 1) Setting WAL journalling *permanently* makes the DB unreadable with
> sqlite 3.6 which is still widely deployed. Notably if I boot back into
> Ubuntu Lucid Lynx Zeitgeist will no longer work as lucid only ships sqlite
> 3.6.
>
> 2) Contrary to what is stated in the merge request for the event_view
> changes the event_view VIEW is *not* regenerated on each startup. That is
> one of the whole points of doing schema versioning. This change makes
> Zeitgeist trunk unable to return any results if run on an older db.
> Strangely enough it doesn't raise any errors...
>
> Guys - if you are making low level changes to our DB structure like both of
> these changes are, please make sure you understand all the ramifications
> before merging.
>
>
>

--
This is me doing some advertisement for my blog http://seilo.geekyogre.com

[Seif Lotfy (seif)]

scratch that

On Sun, Sep 19, 2010 at 4:22 PM, Seif Lotfy <email address hidden> wrote:

> I found the following statement in http://www.sqlite.org/wal.html under
> Backwards Compatibility
>
> One can explicitly change out of WAL mode using a pragma such as this:
>
> PRAGMA journal_mode=DELETE;
>
> Deliberately changing out of WAL mode changes the database file format
> version numbers back to 1 so that older versions of SQLite can once again
> access the database file.
>
> I am uploading a test fix for this... Please tell me what you think
>
> lp:~zeitgeist/zeitgeist/fix-642772
>
> On Sun, Sep 19, 2010 at 4:05 PM, Siegfried Gevatter <email address hidden>wrote:
>
>> 2010/9/19 Siegfried Gevatter <email address hidden>:
>> > Actually they do, create_db() is executed every time at startup and it
>> > has a DROP VIEW.
>> Err, scratch that.
>>
>> --
>> WAL and event_view changes requires schema upgrade
>> https://bugs.launchpad.net/bugs/642772
>> You received this bug notification because you are subscribed to The
>> Zeitgeist Project.
>>
>> Status in Zeitgeist Framework: New
>>
>> Bug description:
>> The recently merged changes to use journal_mode=WAL and adding some
>> columns to event_view requires schema upgrades.
>>
>> 1) Setting WAL journalling *permanently* makes the DB unreadable with
>> sqlite 3.6 which is still widely deployed. Notably if I boot back into
>> Ubuntu Lucid Lynx Zeitgeist will no longer work as lucid only ships sqlite
>> 3.6.
>>
>> 2) Contrary to what is stated in the merge request for the event_view
>> changes the event_view VIEW is *not* regenerated on each startup. That is
>> one of the whole points of doing schema versioning. This change makes
>> Zeitgeist trunk unable to return any results if run on an older db.
>> Strangely enough it doesn't raise any errors...
>>
>> Guys - if you are making low level changes to our DB structure like both
>> of these changes are, please make sure you understand all the ramifications
>> before merging.
>>
>>
>>
>
>
> --
> This is me doing some advertisement for my blog http://seilo.geekyogre.com
>

--
This is me doing some advertisement for my blog http://seilo.geekyogre.com

[Seif Lotfy (seif)]

from what I understand people with who dual boot are confronted with this
issue right?

On Sun, Sep 19, 2010 at 4:22 PM, Launchpad Bug Tracker <
<email address hidden>> wrote:

> ** Branch linked: lp:~zeitgeist/zeitgeist/fix-642772
>
> --
> WAL and event_view changes requires schema upgrade
> https://bugs.launchpad.net/bugs/642772
> You received this bug notification because you are subscribed to The
> Zeitgeist Project.
>
> Status in Zeitgeist Framework: New
>
> Bug description:
> The recently merged changes to use journal_mode=WAL and adding some columns
> to event_view requires schema upgrades.
>
> 1) Setting WAL journalling *permanently* makes the DB unreadable with
> sqlite 3.6 which is still widely deployed. Notably if I boot back into
> Ubuntu Lucid Lynx Zeitgeist will no longer work as lucid only ships sqlite
> 3.6.
>
> 2) Contrary to what is stated in the merge request for the event_view
> changes the event_view VIEW is *not* regenerated on each startup. That is
> one of the whole points of doing schema versioning. This change makes
> Zeitgeist trunk unable to return any results if run on an older db.
> Strangely enough it doesn't raise any errors...
>
> Guys - if you are making low level changes to our DB structure like both of
> these changes are, please make sure you understand all the ramifications
> before merging.
>
>
>

--
This is me doing some advertisement for my blog http://seilo.geekyogre.com

[Seif Lotfy (seif)]

I reverted the WAL changes so this way the part one of the bug is not either fixed or invalid

[Seif Lotfy (seif)]

@thekorn what is your take on this issue. Since WAL stuff has been taken out we still need your take on 2) so we can close this bug

[Seif Lotfy (seif)]

WAL is not there anymore and won't be introduced until we figure a good way to support switching between sqlite <3.7 and >=3.7
The scheme version has been updated to 2 for adding subj_uri_id and subj_origin_id columns to the event_view. Thus bug is fixed