OSD weight should be the TB capacity of the drive

[Chris Holcombe]

I got a test cluster up and running today and it shows the following OSD tree:

chris@chris-ThinkPad-T410s:~$ ceph osd tree
# id weight type name up/down reweight
-1 0.9899 root default
-2 0.33 host chris-local-machine-22
0 0.11 osd.0 up 1
1 0.11 osd.1 up 1
4 0.11 osd.4 up 1
-3 0.33 host chris-local-machine-21
3 0.11 osd.3 up 1
5 0.11 osd.5 up 1
8 0.11 osd.8 up 1
-4 0.33 host chris-local-machine-20
2 0.11 osd.2 up 1
6 0.11 osd.6 up 1
7 0.11 osd.7 up 1

Everything looks to be evenly weighed. That's fine for a cluster that has the same capacity in every server. However we could enhance this by assigning a weight point of 1 for every TB of disk capacity on the server. With that in place Ceph will balance the servers even if the capacity is not even.

I'm not entirely sure but I think the total weight of all the ceph osd servers needs to add up to 1.00. I'll double check on that.

We could add this in with a statvfs call to the device.

[James Page (james-page)]

I'm not entirely sure this is a charm bug; I think the weight of each OSD is determines in the upstart/systemd startup when its started.

[James Page (james-page)]

http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/wily/ceph/wily/view/head:/src/ceph-osd-prestart.sh#L29

The osd-prestart script should be setting the weight automatically on start.

[James Page (james-page)]

Chris - can you confirm that this is actually a problem in a unbalanced server configuration? - I think things should be working OK already.

[Chris Holcombe (xfactor973)]

James - Yeah I ran that prestart command on my local box. It's showing the .001 per GB capacity. That's more accurate than I was going for which is great.