Clean up KARL2 data before migration

[Paul Everitt]

This issue contains a list of content tasks that need to be done before the final migration. Comments on this issue will provide more detail, and the body will be used for updating the actual list.

1) Fix object IDs that are too long for filenames on disk during export.

fiscal-transparency-and-accountability-group
international-justice-forum
kenya

2) Put in a new "Feature".

[Paul Everitt (paul-agendaless)]

Chris Rossi said, re: "Fix object IDs":

"""
The following 3 communities failed to *export* due to having blog entries with ids that were too long for filenames:

fiscal-transparency-and-accountability-group
international-justice-forum
kenya

Grepping the export output for "error" and "exception" reveals only these three problems. Removing these partially exported communities from the export data fixes the import.

The problem causes the export to fail loudly (if you bother to read the output) so I'd be inclined to either grep the error in the export log and manually delete the partially exported community or, even better, fix up the exporter to delete any partial output when it chokes on something.
"""

[Paul Everitt (paul-agendaless)]

I need to assign this to someone at OSI. Anthony, if there is a better candidate who serves as "content wrangler", feel free to re-assign.

[Anthony (agalietti)]

Paul,

The 'Feature' issue has been referred to our Communications Dept. Chipp Winston will handle this, but I will make sure we are available to them for support so that they can make the update.

Regarding the long community blog entries, how do you suggest we fix this?

Thanks Paul.

Anthony

[Paul Everitt (paul-agendaless)]

Fixing the content with the long IDs will involve going into the Plone
UI and using Plone's "rename" to change the identifier (versus the
title).

If you need a little demo on this, let me know.

I added Chris Rossi as a subscriber on this issue. He'll be the one
that works with you on getting the actual list of content that needs
scrubbing. Since I will be at the conference next week, it is good
that the two of you work directly.

--Paul

On May 18, 2009, at 11:22 AM, Anthony wrote:

> Paul,
>
> The 'Feature' issue has been referred to our Communications Dept.
> Chipp
> Winston will handle this, but I will make sure we are available to
> them
> for support so that they can make the update.
>
> Regarding the long community blog entries, how do you suggest we fix
> this?
>
> Thanks Paul.
>
> Anthony
>
> --
> Clean up KARL2 data before migration
> https://bugs.launchpad.net/bugs/372714
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in Porting KARL to a new architecture: New
>
> Bug description:
> This issue contains a list of content tasks that need to be done
> before the final migration. Comments on this issue will provide
> more detail, and the body will be used for updating the actual list.
>
> 1) Fix object IDs that are too long for filenames on disk during
> export.
>
> fiscal-transparency-and-accountability-group
> international-justice-forum
> kenya
>
> 2) Put in a new "Feature".

[Paul Everitt (paul-agendaless)]

This is actually for the last part of the equation

[Chris Rossi (chris-archimedeanco)]

There are three communities which we are unable to export due to very long ids which are longer than the filesystem can handle in file paths. In all three communities, the offending content items are blog entries. After some poking around in the Web UI for Karl2 on karldev, it's not evident to me that there is any easy way to edit the titles of these blog entries. If I am missing something, let me know. The frequency of this error is low enough that it's conceivable to it could be cleaned up by hand, except that I'm not seeing 'Edit' buttons on blog entries when looking at the site.

Failing that, I'll probably just need to write a script for Karl2 for cleaning these up prior to exporting. Does that sound reasonable?

Chris

[Chris Rossi (chris-archimedeanco)]

Ah, of course, after another cup of coffee Chris remembers that Karl2 is built on Plone, which in turn is built on Zope2, so there are other UI's that should be able to handle this.

Just for reference, the locations where this problem occurs are:

http://karldev.soros.org/communities/fiscal-transparency-and-accountability-group-1/blog
http://karldev.soros.org/communities/kenya/blog?start:int=60&sort_on=created&sort_order=desc
http://karldev.soros.org/communities/international-justice-forum/blog

The problematic blog posts should be fairly obvious.

Nat, Anthony, if you decide to go ahead and try to clean these up on your end, I would like to go ahead and retest export with the cleaned up id's to make sure we've gotten them all. Since we're doing exports on Karldev and we also need these changes made to the production data, I would suggest, actually, making the edits in both places and letting me know so I can retest the export.

Do you guys think you could get to that today?

If you prefer, alternatively, I can spend some time writing a script on Karl2 to try to do this cleanup automatically before export. Let me know if you prefer to go this route.

Thanks,
Chris

[Paul Everitt (paul-agendaless)]

Re-assigning to Nat, as Anthony is out next week.

[Nat Katin-Borland (nborland)]

Hi All,

I've cleaned up the posts in KARL 2.0 for the 3 communities listed in the ticket. All of the entries were weird code triggered by out of office auto-replies.

Thanks,
Nat

[Chris Rossi (chris-archimedeanco)]

Hi Nat,

Did you do this in production karl or in karldev?

Thanks,
Chris

On Tue, May 26, 2009 at 10:42 AM, Nat Katin-Borland <email address hidden>wrote:

> Hi All,
>
> I've cleaned up the posts in KARL 2.0 for the 3 communities listed in
> the ticket. All of the entries were weird code triggered by out of
> office auto-replies.
>
> Thanks,
> Nat
>
> ** Changed in: karl3
> Status: New => Fix Committed
>
> --
> Clean up KARL2 data before migration
> https://bugs.launchpad.net/bugs/372714
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in Porting KARL to a new architecture: Fix Committed
>
> Bug description:
> This issue contains a list of content tasks that need to be done before the
> final migration. Comments on this issue will provide more detail, and the
> body will be used for updating the actual list.
>
> 1) Fix object IDs that are too long for filenames on disk during export.
>
> fiscal-transparency-and-accountability-group
> international-justice-forum
> kenya
>
> 2) Put in a new "Feature".
>

[Nat Katin-Borland (nborland)]

In production. Jason said to do it in the live version, then we can
re-sink with KARL-Dev so you have the most recent version of the
content.

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of
Chris Rossi
Sent: Tuesday, May 26, 2009 10:53 AM
To: Nathaniel Katin-Borland
Subject: Re: [Bug 372714] Re: Clean up KARL2 data before migration

Hi Nat,

Did you do this in production karl or in karldev?

Thanks,
Chris

On Tue, May 26, 2009 at 10:42 AM, Nat Katin-Borland
<email address hidden>wrote:

> Hi All,
>
> I've cleaned up the posts in KARL 2.0 for the 3 communities listed in
> the ticket. All of the entries were weird code triggered by out of
> office auto-replies.
>
> Thanks,
> Nat
>
> ** Changed in: karl3
> Status: New => Fix Committed
>
> --
> Clean up KARL2 data before migration
> https://bugs.launchpad.net/bugs/372714
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in Porting KARL to a new architecture: Fix Committed
>
> Bug description:
> This issue contains a list of content tasks that need to be done
before the
> final migration. Comments on this issue will provide more detail, and
the
> body will be used for updating the actual list.
>
> 1) Fix object IDs that are too long for filenames on disk during
export.
>
> fiscal-transparency-and-accountability-group
> international-justice-forum
> kenya
>
> 2) Put in a new "Feature".
>

--
Clean up KARL2 data before migration
https://bugs.launchpad.net/bugs/372714
You received this bug notification because you are a direct subscriber
of the bug.

Status in Porting KARL to a new architecture: Fix Committed

Bug description:
This issue contains a list of content tasks that need to be done before
the final migration. Comments on this issue will provide more detail,
and the body will be used for updating the actual list.

1) Fix object IDs that are too long for filenames on disk during export.

fiscal-transparency-and-accountability-group
international-justice-forum
kenya

2) Put in a new "Feature".