Enable allow_ec_overwrites on erasure coded pools

[Gustavo Sanchez]

Deploying a cinder-ceph charm with pool-type erasure-coded seems missing one step to be able to use pool for RBD.

After deploying the charm [1] and running some post-configuration [2]
Creating cinder volumes will fail [3].

Manually trying to create an RBD volume directly from ceph will also fail unless manually enabling allow_ec_overwrites on erasure coded pool [4][5]

This can be enabled by the charm to avoid confusion and automate creation of a working EC pool for RBD.

[1] https://pastebin.ubuntu.com/p/8HDd3nY8Nq/
[2] https://pastebin.ubuntu.com/p/RTbF7kVbfd/
[3] https://pastebin.ubuntu.com/p/6bBCvGdS5D/
[4] https://pastebin.ubuntu.com/p/tJHdCjMqRb/
[5] https://ceph.io/en/news/blog/2017/new-luminous-erasure-coding-rbd-cephfs/

[Nobuto Murata (nobuto)]

Looks like the charm is trying to do it?
https://github.com/openstack/charm-cinder-ceph/blob/709b184259c05adabb5d8ee284fc9bcd486348c0/hooks/cinder_hooks.py#L191