OpenStack Compute (Nova)

Instance files should be tombstoned and cleaned up later instead of immediately deleted

Reported by Vish Ishaya on 2011-03-21
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Wishlist
Johannes Erdfelt

Bug Description

This is a huge deployment concern. Recovering from accidental deletions by users is impossible if we immediately delete the instance files.

justinsb (justin-fathomdb) wrote :

Agree this is important.

Do you think there should be a administrative command that e.g. gets run in cron job or do you think we should schedule it ourselves somehow? I like the former, because it feels simpler and easier for everyone, but it's really just pushing more work onto the packagers / deployers.

Anyone from Rackspace want to weigh in with a recommended policy? Do we just leave the disk image lying around for - say - 24 hours? Do we archive it off to Swift? What happens if people launch and shut down a lot of machines quickly - could that DoS the cloud?

I like the manage command + cron job. Perhaps enabling tombstoning with a flag, because it is pretty useless in dev mode.

On Mar 22, 2011, at 11:18 PM, justinsb wrote:

> Agree this is important.
>
> Do you think there should be a administrative command that e.g. gets run
> in cron job or do you think we should schedule it ourselves somehow? I
> like the former, because it feels simpler and easier for everyone, but
> it's really just pushing more work onto the packagers / deployers.
>
> Anyone from Rackspace want to weigh in with a recommended policy? Do we
> just leave the disk image lying around for - say - 24 hours? Do we
> archive it off to Swift? What happens if people launch and shut down a
> lot of machines quickly - could that DoS the cloud?
>
> --
> You received this bug notification because you are a direct subscriber
> of the bug.
> https://bugs.launchpad.net/bugs/739601
>
> Title:
> Instance files should be tombstoned and cleaned up later instead of
> immediately deleted
>
> Status in OpenStack Compute (Nova):
> New
>
> Bug description:
> This is a huge deployment concern. Recovering from accidental
> deletions by users is impossible if we immediately delete the instance
> files.
>
> To unsubscribe from this bug, go to:
> https://bugs.launchpad.net/nova/+bug/739601/+subscribe

justinsb (justin-fathomdb) wrote :

I've realized that of course we don't actually delete things in the database, we just set the deleted_at flag, so this might not even be particularly hard...

There might be some helpful stuff which comes out of my work on making sure instances don't disappear every time the host is restarted which is a WIP here:
lp:~justin-fathomdb/nova/restart-instance

justinsb (justin-fathomdb) wrote :

Particularly now a lot of the hard work has been done for us...
https://code.launchpad.net/~jk0/nova/xs-rescue-periodic-tasks/+merge/54597

justinsb (justin-fathomdb) wrote :

lp:~justin-fathomdb/nova/restart-instance now includes a tombstone function, which we could use instead of actually deleting data.

I'd like to hear from people with experience in running Rackspace CloudServers on what we should be doing here.

Thierry Carrez (ttx) on 2011-03-30
Changed in nova:
importance: Undecided → Wishlist
status: New → Confirmed

This is controlled by reclaim_instance_interval, which got merged back in September 2011. Closing this out since it's been implemented for a while.

Changed in nova:
status: Confirmed → Fix Released
status: Fix Released → Fix Committed
Thierry Carrez (ttx) on 2012-02-29
Changed in nova:
milestone: none → essex-4
status: Fix Committed → Fix Released
Mark McLoughlin (markmc) on 2012-04-03
Changed in nova:
assignee: nobody → Johannes Erdfelt (johannes.erdfelt)
Thierry Carrez (ttx) on 2012-04-05
Changed in nova:
milestone: essex-4 → 2012.1
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers