Cinder

Bug #1408763
Comment #1

Comment 1 for bug 1408763

Revision history for this message

John Griffith (john-griffith) wrote on 2015-01-13:

After spending a good deal of time looking at ways to fix this I've come to the conclusion that our current strategy is flawed and flat out broken. There are a number of issues involved here because of taskflow retries that will skew the counter, and also cause things to become out of sync on delete.

After a good deal of thought I see only one "right" way to fix this, and that is to push the reporting back to the driver. The challenge here is that everybody disagrees on how "numbers" should be reported. So I would like to just set one standard for the purpose of scheduler capacity filtering and move on.

I'd propose the addition of a "get_capacity_inf" method to each driver. Each driver would then be responsible for returning 3 items as defined by Cinder (not by the backends). Those 3 items would be:

    provisioned_gb: <number of GB provisioned on the backend>
    allocated_gb: <number of GB actually allocated on the backend>
    available_gb: <number of GB the backend will still allow to be provisioned>

Frankly the only one I care about is provisioned and available, but if and when we get things like over-subscription options we'll need allocated as well. The reason I chose the definitions that I have here is mostly because those are pretty standard definitions in the storage industry, but more importantly they most closely match the internal calculations we're doing in the scheduler that I would like to remove/replace.

I realize I'll get the argument from those that say they do "infinite capacity" because they just add disk shelves as they go, that's fine... but most of the cases where that's been argued they don't actually have a driver in Cinder, and in their internal deployment they don't have more than one backend. So there's no reason they can't just report "actual" numbers at the time and grow (or lie, honestly I don't care). Drivers that ARE in Cinder's source-tree however I would expect to report real and accurate numbers.

This is slightly different that the update_stats call, the reason is because I would like to have it such that the manager makes this call to the driver after every "driver.volume_create" to keep the view of the world in sync and NOT rely on the periodic like we do today. By keeping the calls to a minimum here it will hopefully avoid any crushing overload for devices that aren't very good at handling API requests.

The base driver for now should raise "not implemented" and fall back to the existing manual allocation increment/decrement routine in the manager.

After spending a good deal of time looking at ways to fix this I've come to the conclusion that our current strategy is flawed and flat out broken.  There are a number of issues involved here because of taskflow retries that will skew the counter, and also cause things to become out of sync on delete.

After a good deal of thought I see only one "right" way to fix this, and that is to push the reporting back to the driver.  The challenge here is that everybody disagrees on how "numbers" should be reported.  So I would like to just set one standard for the purpose of scheduler capacity filtering and move on.

I'd propose the addition of a "get_capacity_inf" method to each driver.  Each driver would then be responsible for returning 3 items as defined by Cinder (not by the backends).  Those 3 items would be:

provisioned_gb:  <number of GB provisioned on the backend>
    allocated_gb:       <number of GB actually allocated on the backend>
    available_gb:       <number of GB the backend will still allow to be provisioned>

Frankly the only one I care about is provisioned and available, but if and when we get things like over-subscription options we'll need allocated as well.  The reason I chose the definitions that I have here is mostly because those are pretty standard definitions in the storage industry, but more importantly they most closely match the internal calculations we're doing in the scheduler that I would like to remove/replace.

I realize I'll get the argument from those that say they do "infinite capacity" because they just add disk shelves as they go, that's fine... but most of the cases where that's been argued they don't actually have a driver in Cinder, and in their internal deployment they don't have more than one backend.  So there's no reason they can't just report "actual" numbers at the time and grow (or lie, honestly I don't care).  Drivers that ARE in Cinder's source-tree however I would expect to report real and accurate numbers.

This is slightly different that the update_stats call, the reason is because I would like to have it such that the manager makes this call to the driver after every "driver.volume_create" to keep the view of the world in sync and NOT rely on the periodic like we do today.  By keeping the calls to a minimum here it will hopefully avoid any crushing overload for devices that aren't very good at handling API requests.

The base driver for now should raise "not implemented" and fall back to the existing manual allocation increment/decrement routine in the manager.