Ceph should bind both cluster network and public network to Fuel Storage by default

[Andrey Grebennikov]

By default when deploying with Ceph, Ceph Public network is assigned to Fuel Management network. In order to avoid OpenStack management traffic interference with Ceph data traffic, the last one should be always isolated.
Suggesting to associate both ceph/public and ceph/replication with Fuel Storage network. These networks can be later separated using network templates.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/284467

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/284467
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=f4d4546610b9159b17a9009e8222074c363f5e36
Submitter: Jenkins
Branch: master

commit f4d4546610b9159b17a9009e8222074c363f5e36
Author: Andrey <email address hidden>
Date: Wed Feb 24 15:22:47 2016 -0800

    Assign ceph/public to Fuel Storage by default

    In order to isolate Ceph data traffic from OpenStack
    management traffic, move ceph/public from Management
    to Storage network.

    Change-Id: If08aa1d2f92e6cf05d5da60b1bbfeaa957b67f13
    Closes-Bug: #1549522

[Alexey Stupnikov (astupnikov)]

Setting this bug's status to 'Opinion' for MOS 8.0 and MOS 7.0 due to the following reasons:
 - it is a documented MOS behaviour;
 - proposed patch will not work for existing installations, since its enablement will destroy existing environments;
 - changes from openstack.yaml should be manually injected to fuel DB.I think that deployment engineer or customer's engineer could manually change openstack.yaml file and upload changes to fuel DB with the same impact. I can write how-to article for salesforce knoledge DB if it is needed there.

[Ilya Kharin (akscram)]

Guys, such decisions was obviously right to assign the ceph/public role on the storage network and there is only one question like "why this was not done sooner?". But this change cardinally complicated the upgrade procedure to the 9.x release and introduced more challenges in it. Such assignment assumes that addresses for ceph-mons are allocated from the storage network and not from the management network how it was in the previous releases. Ceph-OSD nodes and Computes nodes will lost their connections with Ceph-Monitors during the re-deployment of Controller nodes on the 9.x release. This problem can be solved at leat in two ways:

* configure two sets of ceph-mons on each controller on two IP addresses (to serve old clients and new clients) until all old nodes will be upgraded
* mandatory use a network template for the destination cloud to not change the ceph/public role on the storage network and preserve it on the management network

[0] https://bugs.launchpad.net/fuel/+bug/1549522
[1] http://docs.ceph.com/docs/master/rados/operations/add-or-rm-mons/#changing-a-monitor-s-ip-address