Comment 1 for bug 1749544

Root Cause:

The pools were not created and ansible [1] returned the following message from ceph:

"Error ERANGE: pg_num 128 size 3 would mean 768 total pgs, which exceeds max 600 (mon_max_pg_per_osd 200 * num_in_osds 3)"

The workaround is to change any of the above three variables to satisfy the following function when we create, for OpenStack by default, seven pools:

 https://github.com/ceph/ceph/blob/e59258943bcfe3e52d40a59ff30df55e1e6a3865/src/mon/OSDMonitor.cc#L5670-L5698

This is new to queens because it's using lumionus which has the above feature. The problem is that EVERY queens deployment that doesn't override the defaults will have this problem.

Here's one workaround which satisfies the function above:

parameter_defaults:
  CephPoolDefaultSize: 3
  CephPoolDefaultPgNum: 128
  CephConfigOverrides:
    mon_max_pg_per_osd: 3072

In the above case I increased mon_max_pg_per_osd based on the closest power of 2 greater than (* 128 3 7).

[1] grep Error /var/log/mistral/ceph-install-workflow.log | grep 128