Comment 21 for bug 1253980

Revision history for this message
Phil Day (philip-day) wrote : RE: [Bug 1253980] Re: DoS attack via setting os_type in snapshots.

> @Phil: agreed... waiting for confirmation from other VMT members to go
> ahead. Would the ephemeral backing files all land on the same server
> (reaching a state of partial DoS quickly) or be spread across all compute
> nodes (reaching a state of total DoS but slowly) ?

That depends entirely on the scheduling policy configured in Nova - a stacking model would hit the issue on specific nodes more quickly that a spreading model (so a quicker DoS on specific servers) - but really it’s the total DoS that is the issue which is kind of the same in either case, its just a question of if you fill from the side or from the bottom.

> -----Original Message-----
> From: <email address hidden> [mailto:<email address hidden>] On Behalf
> Of Thierry Carrez
> Sent: 10 December 2013 09:49
> To: Day, Phil
> Subject: [Bug 1253980] Re: DoS attack via setting os_type in snapshots.
>
> New proposed impact desc:
>
> --------------------------------------------
> Title: Nova compute DoS through ephemeral disk backing files
> Reporter: Phil Day (HP)
> Products: Nova
> Affects: All supported versions
>
> Description:
> Phil Day from HP reported a vulnerability in the handling of ephemeral disk
> backing files on Nova compute nodes. By repeatedly creating snapshots,
> changing the os_type to a new random value, and spawning new instances
> from the snapshot (and quickly deleting those instances), an authenticated
> user could generate lots of different ephemeral disk backing files and fill up
> compute node disks, potentially resulting in a Denial of Service against a Nova
> setup.
> --------------------------------------------
>
> @Phil: agreed... waiting for confirmation from other VMT members to go
> ahead. Would the ephemeral backing files all land on the same server
> (reaching a state of partial DoS quickly) or be spread across all compute
> nodes (reaching a state of total DoS but slowly) ?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1253980
>
> Title:
> DoS attack via setting os_type in snapshots.
>
> Status in OpenStack Compute (Nova):
> In Progress
> Status in OpenStack Compute (nova) grizzly series:
> New
> Status in OpenStack Compute (nova) havana series:
> New
> Status in OpenStack Security Advisories:
> Triaged
>
> Bug description:
> If the os_type metadata is set of an image, the ephemeral disk backing file
> for that image will be named ephemeral_[size]_[os_type].
> Because the user can change os_type they can use this to create new
> ephemeral backing files.
> Nova image cache management does not include deleting ephemeral
> backing files (presumably because they are expected to be a small, stable
> set.
>
> Hence a user can fill the disk with ephemeral backing files via the
> following:
>
> 1) Spawn a instance
> 2) Create a snapshot from it, delete the original instance
> 3) In a loop:
> generate a random os_type
> set os_type to the snapshot
> spawn and instance from it, and then delete the instance
>
> Every iteration will generate an ephemeral backing file on a compute
> host. With a stacking scheduling policy there is a good chance of
> hitting the same host repeatedly until its disk is full.
>
> Suggested mitigation
>
> Only use “os_type” in the ephemeral file name if there is a specific
> mkfs command defined, otherwise use “default” (Currently for
> undefined os-types it will use the default mkfs command, but still
> uses os_type in the name.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/nova/+bug/1253980/+subscriptions