New approach to big sampledata

[Paul Everitt]

At the beginning of KARL3, we wanted to get an idea of the performance when you had some data in the site. "Performance" was somewhat amorphous: reads vs. writes on various screens, searches, memory footprint, using ab to pound the site, whatever. I wrote a console script and checked in some sample data that would bulk-load the site.

This console script had gotten bit rotted and it was somewhat dumb to have large XML files checked into the src/osi customization package for testing purposes. So I removed this data. I haven't yet removed the console script.

Spec
=======

- Console script that uses ZEO to create small, medium, and large example sites

- Use the fake-view method from src/osi/osi/run.py's populate function to stay closer in sync with changes in code

- Wire up unit tests to see if the basics of the sample data loader get out of sync. Design the script with this testability in mind.

- Find some way (lorem ipsum module, ispell dict) to get large amounts of sample data that contains unique words

- Use some small .doc and .pdf files for files

- Try to get a representative sample of content types into communities (blog entries, files, wiki pages, calendar events)

Tasks
=====

- Read the src/osi/osi/scripts/bulkloader.py and get an idea what it was trying to do

- Then, svn remove it and remove the entry point in src/osi/setup.py

- Create in src/karl somewhere (or karlsample) a new bulksample.py or some other named script

- Document in the main README.txt

[Paul Everitt (paul-agendaless)]

Shane, I know you're out today and tomorrow. When you get a chance, could you give an update on this?

[Shane Hathaway (shane-hathawaymix)]

I created a script called "samplegen". It produces communities containing blog entries, wiki pages, calendar events, and files. It pulls random words and text from Alice in Wonderland, a public domain book. I added a bit about the new script to README, then I removed bulkloader.py and its entry point.

[Shane Hathaway (shane-hathawaymix)]

I also added tests of samplegen, as specified.

[Paul Everitt (paul-agendaless)]

On Oct 9, 2009, at 3:20 AM, Shane Hathaway wrote:

> I created a script called "samplegen". It produces communities
> containing blog entries, wiki pages, calendar events, and files. It
> pulls random words and text from Alice in Wonderland, a public domain
> book. I added a bit about the new script to README, then I removed
> bulkloader.py and its entry point.

Hi Shane. Do you think you could extend this to create the sample
users affliliate1-9 and staff1-9 if they don't exist?

--Paul

[Shane Hathaway (shane-hathawaymix)]

Paul Everitt wrote:
> Hi Shane. Do you think you could extend this to create the sample
> users affliliate1-9 and staff1-9 if they don't exist?

Yes. I will later.

Shane

[Shane Hathaway (shane-hathawaymix)]

samplegen now creates the sample users you listed.