Implement Versioning for Files

[Paul Everitt]

Very similar to Versioning for Wikipages (and attachments):

- FILES tool gets a submenu which has "Trash", move the actions up into the submenu like in WIKI
- Individual files get "History"
- Evolve script which bulk-loads existing files into the repo to have their initial state in the history

[Paul Everitt (paul-agendaless)]

Doesn't look like this will be done in time for review this week. Let's plan to work on it this week and deliver next week. Tres, please reply with what your schedule on this looks like, or if you're booked elsewhere.

[Tres Seaver (tseaver)]

I've got the first bit done -- need push access to the repository.

[Tres Seaver (tseaver)]

The 'file_versions' branch on github now contains the changes implementing
this feature. It should be ready for testing on a staging server.

[Chris Rossi (chris-archimedeanco)]

Available for testing on staging branch3:

https://karlstaging.gocept.com/branch3/osf

At time of writing, it is in maintenance mode while an evolve script is run. It should be up shortly.

[Chris Rossi (chris-archimedeanco)]

The evolve script for this one failed on staging due to running out of RAM. I also noticed that the script would have unnecessarily created versions for objects already in the repository, so I've tweaked that and am rerunning. Unfortunately the evolve takes hours to run, so I won't know for a bit whether we're going to run out of memory again. If we run out of memory again it may be necessary to refactor to do the evolve in batches.

[Paul Everitt (paul-agendaless)]

I'm surprised on this, I thought everything was in the repo already and didn't need an evolve.

--Paul

On Aug 30, 2011, at 9:55 AM, Chris Rossi wrote:

> The evolve script for this one failed on staging due to running out of
> RAM. I also noticed that the script would have unnecessarily created
> versions for objects already in the repository, so I've tweaked that and
> am rerunning. Unfortunately the evolve takes hours to run, so I won't
> know for a bit whether we're going to run out of memory again. If we
> run out of memory again it may be necessary to refactor to do the evolve
> in batches.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/821462
>
> Title:
> Implement Versioning for Files
>
> Status in KARL3:
> Fix Committed
>
> Bug description:
> Very similar to Versioning for Wikipages (and attachments):
>
> - FILES tool gets a submenu which has "Trash", move the actions up into the submenu like in WIKI
> - Individual files get "History"
> - Evolve script which bulk-loads existing files into the repo to have their initial state in the history
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/karl3/+bug/821462/+subscriptions

[Tres Seaver (tseaver)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 08/30/2011 10:41 AM, Paul Everitt wrote:
> I'm surprised on this, I thought everything was in the repo already
> and didn't need an evolve.

Folders needed to be addeed to the repository: they didn't have the
right adapters registered the last time.

If we still have RAM problems, then I can tweak the script to deactivate
the objects after adding them.

Tres.
- --
===================================================================
Tres Seaver [phone] <email address hidden>
Palladion Software "Excellence by Design" http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5dJrsACgkQ+gerLs4ltQ45QACfYcXfqs81BVBUpS0yCamGLQKU
tl8AoLh0pPQBi959n/HrSfqzTJ9xmCud
=8phV
-----END PGP SIGNATURE-----

[Chris Rossi (chris-archimedeanco)]

It failed again with a MemoryError. The script does contain a call to context._p_deactivate() for each object already, so I think we're probably going to have to go for a batching solution.

[Tres Seaver (tseaver)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 08/30/2011 03:12 PM, Chris Rossi wrote:
> It failed again with a MemoryError. The script does contain a call
> to context._p_deactivate() for each object already, so I think we're
> probably going to have to go for a batching solution.

One option would be to commit the transaction as each community is
completed, and then call 'obj._p_jar.cacheGC()'.

Tres.
- --
===================================================================
Tres Seaver [phone] <email address hidden>
Palladion Software "Excellence by Design" http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5dOZEACgkQ+gerLs4ltQ48AACgykpO+gR4kgwWZoVX4h8e0I8B
FgUAnifNxpIestKrUNuzNFuCpUShrWF6
=Q+Uw
-----END PGP SIGNATURE-----

[Shane Hathaway (shane-hathawaymix)]

Does the evolve script commit the transaction on a regular basis? SQLAlchemy seems unfortunately naive about large blobs; I think the only way to free the RAM consumed by a large file is to commit the transaction.

[JimPGlenn (jpglenn09)]

fixed