Solidify the pack, backup, restore plan with repozo

Bug #377047 reported by Paul Everitt
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
KARL3
Fix Released
Low
Shane Hathaway

Bug Description

Get with Calvin and the other folks and decide how the bin/zeopack and repozo will interplay in a cron job with the backup system. Once implemented, *test drive it*. We need to make sure it plays nice with the BLOBs.

Tags: operations
Revision history for this message
LarsN (lars-sixfeetup) wrote :

Once kdiaa is moved into production, and kdiab is moved to continued development, I will use the development server to work on this ticket.

Revision history for this message
Paul Everitt (paul-agendaless) wrote :

To help increase focus on this week, moving this task to next week.

Changed in karl3:
milestone: m15 → m16
Revision history for this message
Paul Everitt (paul-agendaless) wrote :

Lars, is this still on the radar?

Revision history for this message
LarsN (lars-sixfeetup) wrote : Re: [Bug 377047] Re: Solidify the pack, backup, restore plan with repozo

Yes, it sure is. You said to push it off 1 week to get through the
other more important tasks. Now that I have kdi-dev to bang on I
intend to get this squared away.

Thanks, Lars
--
S i x F e e t U p , I n c . | http://www.sixfeetup.com
Phone: +1 (317) 861-5948 x609
<email address hidden>
ANNOUNCING the first Plone Immersive Training Experience | Sept.
10-11-12, 2009
http://sixfeetup.com/immerse

On May 24, 2009, at 9:22 AM, Paul Everitt wrote:

> Lars, is this still on the radar?
>
> --
> Solidify the pack, backup, restore plan with repozo
> https://bugs.launchpad.net/bugs/377047
> You received this bug notification because you are a bug assignee.
>
> Status in Porting KARL to a new architecture: New
>
> Bug description:
> Get with Calvin and the other folks and decide how the bin/zeopack
> and repozo will interplay in a cron job with the backup system.
> Once implemented, *test drive it*. We need to make sure it plays
> nice with the BLOBs.

Revision history for this message
Paul Everitt (paul-agendaless) wrote :

Re-assigning to Shane, as I ran into problems when running it. Symptoms:

1) I run:

 bin/zeopack -h localhost -p 8884

2) Minutes go by with nothing happening (particularly, no var/osi.db.pack appears and starts getting larger.)

3) I kill it.

4) A minute later, osi.db.pack appears and gets bigger, finishing with osi.db getting replaced and osi.db.old getting created.

5) However, the blobs are broken in the site.

Changed in karl3:
assignee: LarsN (lars-sixfeetup) → Shane Hathaway (shane-hathawaymix)
Revision history for this message
Paul Everitt (paul-agendaless) wrote :

Since Lars is doing full backups, this is a lower priority.

Changed in karl3:
importance: Medium → Low
Changed in karl3:
milestone: m16 → m18
Revision history for this message
Shane Hathaway (shane-hathawaymix) wrote :

I tested it on kdi-dev. Packing appears to work fine. It took about a half hour, I think. I did not see any problems with blobs.

There is a long wait period at the beginning of a pack while ZODB scans the whole database. This is normal.

While the "repozo" utility works, it is ignorant of blobs, so keep that in mind when designing the backup plan. One safe plan would be to run repozo before backing up the blob directory. That way you might end up with extra blobs if someone happens to upload files during the backup, but the system will still be transactionally consistent.

BTW, regular packing improves the disk cache hit rate, leading to better ZEO performance, leading to a faster site. Let's pack daily if we can.

Revision history for this message
Shane Hathaway (shane-hathawaymix) wrote :

BTW, the earlier blob problem may have occurred while developers were moving databases around without restarting ZEO, leading to undefined behavior.

Changed in karl3:
assignee: Shane Hathaway (shane-hathawaymix) → LarsN (lars-sixfeetup)
Revision history for this message
Paul Everitt (paul-agendaless) wrote :

Shuffle off to next week.

Changed in karl3:
milestone: m18 → m19
Revision history for this message
Paul Everitt (paul-agendaless) wrote :

Move to next week.

Changed in karl3:
milestone: m19 → m21
Revision history for this message
Chris Rossi (chris-archimedeanco) wrote :

Lars, is this something you think we can get in place this week?

Changed in karl3:
milestone: m21 → m22
Revision history for this message
Paul Everitt (paul-agendaless) wrote :

Hi Shane. I think Lars has gotten backed up a bit. Re-assigning it to you, with the idea that:

- Six Feet Up does not do repozo backups. I haven't heard much about this, so let's pretend they don't.

- You'll get a cron job that runs in the night to do an automatic pack on the production database.

- Once you confirm that this works ok, you close the ticket.

Changed in karl3:
assignee: LarsN (lars-sixfeetup) → Shane Hathaway (shane-hathawaymix)
Changed in karl3:
milestone: m22 → m23
Revision history for this message
Shane Hathaway (shane-hathawaymix) wrote :

Added a pack script to the crontab for 'zope' in production. It runs nightly at 1:45 AM.

Changed in karl3:
status: New → Fix Committed
Changed in karl3:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.