[virtualbox] Network verification does not work after deployment on CentOS

[Anastasia Palkina]

"build_id": "2015-05-21_04-04-09",
"build_number": "446",
"release_versions": {"2014.2.2-6.1": {"VERSION": {"build_id": "2015-05-21_04-04-09", "build_number": "446", "api": "1.0", "fuel-library_sha": "a03efb582b06bfe8d9776dce244d4a2f2e2ba886", "nailgun_sha": "403c6b7ea3c62bb4fda27eb9cedee37f7144558c", "feature_groups": ["mirantis"], "openstack_version": "2014.2.2-6.1", "production": "docker", "python-fuelclient_sha": "e19f1b65792f84c4a18b5a9473f85ef3ba172fce", "astute_sha": "795f8a045400fe82ccc30ae018e85324b3fa1de5", "fuel-ostf_sha": "3dd25a018f2a5c47ec6c885436b3ba69690ef1b9", "release": "6.1", "fuelmain_sha": "5c8ebddf64ea93000af2de3ccdb4aa8bb766ce93"}}},
"auth_required": true,
"api": "1.0",
"fuel-library_sha": "a03efb582b06bfe8d9776dce244d4a2f2e2ba886",
"nailgun_sha": "403c6b7ea3c62bb4fda27eb9cedee37f7144558c",
"feature_groups": ["mirantis"],
"openstack_version": "2014.2.2-6.1",
"production": "docker",
"python-fuelclient_sha": "e19f1b65792f84c4a18b5a9473f85ef3ba172fce",
"astute_sha": "795f8a045400fe82ccc30ae018e85324b3fa1de5",
"fuel-ostf_sha": "3dd25a018f2a5c47ec6c885436b3ba69690ef1b9",
"release": "6.1",
"fuelmain_sha": "5c8ebddf64ea93000af2de3ccdb4aa8bb766ce93"

1. Create new environment (CentOS)
2. Choose Neutron, VLAN
3. Choose Ceph for volumes and Ceph for images
4. Choose Ceph RadosGW for objects
5. Choose Murano
6. Choose Ceilometer
7. Add 3 controller, 2 compute, 2 ceph, 3 mongo
8. Move Storage network to eth1
9. Move Management network to eth2
10. Start deployment. It was successful
11. Start network verification. It has failed (see screen)

Logs are here: https://drive.google.com/a/mirantis.com/file/d/0B6SjzarTGFxaTWFSWXk3THI0dkk/view?usp=sharing

[Aleksey Kasatkin (alekseyk-ru)]

According to net-checker logs traffic with private tags (1000-1030) doesn't run between nodes.

There is a problem on controller nodes also: "Listerner: tcpdump: can't create rx ring on packet socket: Cannot allocate memory" - same as in https://bugs.launchpad.net/fuel/+bug/1425090 .

But listeners on other nodes are fine so the main problem is not here.

[Stanislav Makar (smakar)]

I would like to know in which way did you
8. Move Storage network to eth1
9. Move Management network to eth2 and untag it
?

[Stanislav Makar (smakar)]

astute.log shows above
Iface eth0.1000 does not exist.Iface eth0.1001 does not exist.Iface eth0.1002 does not exist.Iface eth0.1003 does no
t exist.Iface eth0.1004 does not exist.Iface eth0.1005 does not exist.Iface eth0.1006 does not exist.Iface eth0.1007 does not exist.Iface eth0.
1008 does not exist.Iface eth0.1009 does not exist.Iface eth0.1010 does not exist.Iface eth0.1011 does not exist

That means that network checker did not manage to create these interfaces before verifying

[Dima Shulyak (dshulyak)]

Stanislav, sorry for confusing log. But please take a look at Aleksey comment - it is pretty accuretely described the problem.
We wasnt able to create listener for interface eth0 due to low memory availability.

And in fact it is the same problem as in this bug https://bugs.launchpad.net/fuel/+bug/1425090, but for virtualbox environments.
My proposal is next:
- If this issue reproducable often enough - increase memory for virtualbox environments
- In case if it is one-time failure - then we can set it wont fix, or atleast decrease priority and move to next release

[Dima Shulyak (dshulyak)]

Looks like it is not a network checker problem.

[root@node-3 ~]# nova list
+--------------------------------------+------+--------+------------+-------------+---------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------+--------+------------+-------------+---------------------+
| cf3b1ea5-836a-464e-86f0-b5defda4b2b0 | test | ACTIVE | - | Running | net04=192.168.111.4 |
+--------------------------------------+------+--------+------------+-------------+---------------------+

Then tried to ping this VM from qrouter namespace on this node:

[root@node-3 ~]# ip netns exec qrouter-43c62888-a1a4-4434-b5f6-782381ca8835 ping 192.168.111.4
PING 192.168.111.4 (192.168.111.4) 56(84) bytes of data.
From 192.168.111.1 icmp_seq=1 Destination Host Unreachable

During this ping procedure i am able to see traffic on br-int, with local significant vlan (vlan=1):

[root@node-3 ~]# tcpdump -nnei br-int

09:10:53.955756 fa:16:3e:79:5d:cf > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 1, p 0, ethertype ARP, Request who-has 192.168.111.4 tell 192.168.111.1, length 28

And then on br-prv with vlan 1030:

09:11:41.013032 fa:16:3e:79:5d:cf > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 1030, p 0, ethertype ARP, Request who-has 192.168.111.4 tell 192.168.111.1, length 28

But on the compute node where this VM is running i am not able to see any traffic on br-prv.

Sergei Ovsianikov will help with reproducing

[Dima Shulyak (dshulyak)]

Guys from fuel-library, please take a look, maybe you already saw this problem.
Also one more observation - it if will create vlan tagged interface like eth0.1001 - traffic will be seen on a node.

[Stanislav Makar (smakar)]

To ping between VMs inside tenant you should allow icmp traffic in sec group
by default it is disabled

[Dima Shulyak (dshulyak)]

Yes. But i even wasnt able to see arp response on controller, and arp query on compute node.
And i am also unable to ping from qrouter on node-3 to qdhcp on node-2.

[Dima Shulyak (dshulyak)]

If anyone will want to try generate tagged traffic - here is script:

http://paste.openstack.org/show/237554/

Save it to sender.py, and use like this

>> python sender.py eth0 1001

[Serhii Ovsianikov (sovsianikov)]

VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "6.1"
  openstack_version: "2014.2.2-6.1"
  api: "1.0"
  build_number: "466"
  build_id: "2015-05-25_20-55-26"
  nailgun_sha: "61ef0edfbfe0c457265a62f0eab05af634ec3b91"
  python-fuelclient_sha: "e19f1b65792f84c4a18b5a9473f85ef3ba172fce"
  astute_sha: "0bd72c72369e743376864e8e8dabfe873d40450a"
  fuel-library_sha: "d7128c27a1b76f4813f3697609f82875c68e85ed"
  fuel-ostf_sha: "87819878bc0ca572900e1f6933d9b99e666d6f62"
  fuelmain_sha: "5c8ebddf64ea93000af2de3ccdb4aa8bb766ce93"

I installed environment on the virtual box and the test of "Verify Networks" passed OK

[Serhii Ovsianikov (sovsianikov)]

My mistake - this is Ubuntu

[Serhii Ovsianikov (sovsianikov)]

Under CentOS the test "Verify Networks" doesn't work.

The version of VirtualBox
# VBoxManage -v
4.3.28r100309

[Stanislav Makar (smakar)]

Could be connected with https://bugs.launchpad.net/fuel/+bug/1458764
We have to verify with new kmod-openvswitch package

[Stanislav Makar (smakar)]

it was KVM using new ISO with correct kmod-openvswitch package
We should check it on VirtualBox

[Anastasia Palkina (apalkina)]

Stanislav Makar, please, attach request related to correct kmod-openvswitch package and change bug status to 'Fix Committed' if you think that issue fixed.

[Stanislav Makar (smakar)]

looks like issue has gone after this https://review.fuel-infra.org/#/c/6992/

[Anastasia Palkina (apalkina)]

Reproduced on ISO #471

"build_id": "2015-05-26_20-59-56", "build_number": "471", "release_versions": {"2014.2.2-6.1": {"VERSION": {"build_id": "2015-05-26_20-59-56", "build_number": "471", "api": "1.0", "fuel-library_sha": "8cfeca1a86179ebed1e4e03b2133b49c27350f6f", "nailgun_sha": "f737675091bd1903aace0e36812e855ce47dfec7", "feature_groups": ["mirantis"], "openstack_version": "2014.2.2-6.1", "production": "docker", "python-fuelclient_sha": "e19f1b65792f84c4a18b5a9473f85ef3ba172fce", "astute_sha": "0bd72c72369e743376864e8e8dabfe873d40450a", "fuel-ostf_sha": "87819878bc0ca572900e1f6933d9b99e666d6f62", "release": "6.1", "fuelmain_sha": "13b3e9cf074ba1cf1ae06509c55fbab613c73f4e"}}}, "auth_required": true, "api": "1.0", "fuel-library_sha": "8cfeca1a86179ebed1e4e03b2133b49c27350f6f", "nailgun_sha": "f737675091bd1903aace0e36812e855ce47dfec7", "feature_groups": ["mirantis"], "openstack_version": "2014.2.2-6.1", "production": "docker", "python-fuelclient_sha": "e19f1b65792f84c4a18b5a9473f85ef3ba172fce", "astute_sha": "0bd72c72369e743376864e8e8dabfe873d40450a", "fuel-ostf_sha": "87819878bc0ca572900e1f6933d9b99e666d6f62", "release": "6.1", "fuelmain_sha": "13b3e9cf074ba1cf1ae06509c55fbab613c73f4e"

Verification after deployment failed again

[Anastasia Palkina (apalkina)]

Logs are here https://drive.google.com/a/mirantis.com/file/d/0B6SjzarTGFxaalp3WThYWXFkX3c/view?usp=sharing

[Dima Shulyak (dshulyak)]

So, when environmnet deployment with next network allocation:

eth0: pxe, private(vlan 1000-1030)
eth1: public, storage 102
eth2: management: 101

Tagged traffic from private network is not passing from one node to another.

On provided environment we tried to generate traffic from qdhcp on node-17, and listen for it in qrouter on node-15.
But it wasnt received by latter host (node-15) neither on eth0 or br-prv. It is seen with tag on br-prv on node-17, and vboxnet0 (corresponding bridge on host system).

[Stanislav Makar (smakar)]

Verification succeeded. Your network is configured correctly.

The same network configuration on KVM works without problems. It is Virtualbox problem only

[Stanislav Makar (smakar)]

On VirtualBox

[root@node-17 ~]# tcpdump -pnevvi eth0 vlan
tcpdump: WARNING: eth0: no IPv4 address assigned
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
12:58:03.237708 fa:16:3e:b8:f7:86 > Broadcast, ethertype 802.1Q (0x8100), length 46: vlan 1030, p 0, ethertype ARP,
 Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.111.1 tell 192.168.111.2, length 28
12:58:04.237069 fa:16:3e:b8:f7:86 > Broadcast, ethertype 802.1Q (0x8100), length 46: vlan 1030, p 0, ethertype ARP,
 Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.111.1 tell 192.168.111.2, length 28

[root@node-17 ~]# tcpdump -pnevvi eth0 vlan
tcpdump: WARNING: eth0: no IPv4 address assigned
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
12:58:03.237708 fa:16:3e:b8:f7:86 > Broadcast, ethertype 802.1Q (0x8100), length 46: vlan 1030, p 0, ethertype ARP,
 Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.111.1 tell 192.168.111.2, length 28
12:58:04.237069 fa:16:3e:b8:f7:86 > Broadcast, ethertype 802.1Q (0x8100), length 46: vlan 1030, p 0, ethertype ARP,
 Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.111.1 tell 192.168.111.2, length 28
12:58:05.252938 fa:16:3e:b8:f7:86 > Broadcast, ethertype 802.1Q (0x8100), length 46: vlan 1030, p 0, ethertype ARP,
 Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.111.1 tell 192.168.111.2, length 28

on node-16 where is 192.168.111.2

[root@node-15 ~]# ip netns exec qdhcp-87c14191-4621-44bc-9290-4243c1457b36 ip a
39: tap63298197-85: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
    link/ether fa:16:3e:b8:f7:86 brd ff:ff:ff:ff:ff:ff
    inet 192.168.111.2/24 brd 192.168.111.255 scope global tap63298197-85
    inet6 fe80::f816:3eff:feb8:f786/64 scope link
       valid_lft forever preferred_lft forever
40: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
[root@node-15 ~]# tcpdump -pnevvvi eth0 vlan
tcpdump: WARNING: eth0: no IPv4 address assigned
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes

On host machine we see that all is ok with traffic from node-17

root@anastasia-X9SRE-X9SRE-3F-X9SRi-X9SRi-3F:~# tcpdump -pnevvvvi vboxnet0 host 192.168.111.1
tcpdump: listening on vboxnet0, link-type EN10MB (Ethernet), capture size 65535 bytes
16:00:49.122187 fa:16:3e:b8:f7:86 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 1030, p 0, ethertype ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.111.1 tell 192.168.111.2, length 28
16:00:50.123088 fa:16:3e:b8:f7:86 > ff:ff:ff:ff:ff:ff, ethertype...

[Stanislav Makar (smakar)]

I tested by myself below configuration
All - Neutron + VLAN
All on the same host:
  anastasia@anastasia-X9SRE-X9SRE-3F-X9SRi-X9SRi-3F:~$ uname -r
  3.13.0-53-generic
  anastasia@anastasia-X9SRE-X9SRE-3F-X9SRi-X9SRi-3F:~$ lsb_release -a
  No LSB modules are available.
  Distributor ID: Ubuntu
  Description: Ubuntu 14.04.2 LTS
  Release: 14.04
 C odename: trusty

Oracle VM VirtualBox Manager 4.3.28
All used the same virtualbox networks
interface driver - e1000
OpenVswitch version: 2.3.1

All env:
Admin and Public - untagged
Management - tagged 101
Storage - tagged 102

Fuel ISO
  release: "6.1"
  openstack_version: "2014.2.2-6.1"
  api: "1.0"
  build_number: "471"
  build_id: "2015-05-26_20-59-56"

1. Centos, admin and private on eth0 -- network verification failed and private traffic dosn't go
2. Ubuntu, admin and private on eth0 - all is ok
3. Centos with fedora kernel (3.10), admin and private on eth0 -- all is ok!!!
4. Centos all network by default, public on eth1, all other networks on eth0 -- all is ok
5. Centos, public and private on eth1, all other on eth0

As we see we have just problem with default kernel and if private network is on interface which has untagged networks
tests 1,5
If interface except private network has any tagged network (test 4) -- all is ok

So we can say that we have problem only with default Centos kernel and only on VirtualBox

[Stanislav Makar (smakar)]

Sorry forgot to add test results
5. Centos, public and private on eth1, all other on eth0 - network verification failed and private traffic doesn't go

[Nastya Urlapova (aurlapova)]

@Stanislav, default means "work from box", it must work with 2.6 kernel.

[Vladimir Kuklin (vkuklin)]

@mos-linux folks, please investigate why OVS may not work with Virtual Box here

[Pavel Boldin (pboldin)]

Seems to be related http://openvswitch.org/pipermail/discuss/2015-March/016760.html

[Ihor Kalnytskyi (ikalnytskyi)]

I'm not sure it's related to openvswitch. Neither kmod-openvswitch nor openvswitch are installed on base-os node. There should be another issue. Haven't we update CentOS default kernel?

[Pavel Boldin (pboldin)]

@Igor, well, try to update it manually to the latest CentOS 6.6 version.

[Pavel Boldin (pboldin)]

Unable to reproduce at ISO #489.

VirtualBox version:
$ VBoxManage --version
4.3.10_Ubuntur93012

[Anastasia Palkina (apalkina)]

Reproduced on ISO #490

"build_id": "2015-05-31_20-55-26", "build_number": "490", "release_versions": {"2014.2.2-6.1": {"VERSION": {"build_id": "2015-05-31_20-55-26", "build_number": "490", "api": "1.0", "fuel-library_sha": "c9a86ac0e6da95d36e328ce5130715792a2eb177", "nailgun_sha": "3830bdcb28ec050eed399fe782cc3dd5fbf31bde", "feature_groups": ["mirantis"], "openstack_version": "2014.2.2-6.1", "production": "docker", "python-fuelclient_sha": "4fc55db0265bbf39c369df398b9dc7d6469ba13b", "astute_sha": "5d570ae5e03909182db8e284fbe6e4468c0a4e3e", "fuel-ostf_sha": "7413186490e8d651b8837b9eee75efa53f5e230b", "release": "6.1", "fuelmain_sha": "6b5712a7197672d588801a1816f56f321cbceebd"}}}, "auth_required": true, "api": "1.0", "fuel-library_sha": "c9a86ac0e6da95d36e328ce5130715792a2eb177", "nailgun_sha": "3830bdcb28ec050eed399fe782cc3dd5fbf31bde", "feature_groups": ["mirantis"], "openstack_version": "2014.2.2-6.1", "production": "docker", "python-fuelclient_sha": "4fc55db0265bbf39c369df398b9dc7d6469ba13b", "astute_sha": "5d570ae5e03909182db8e284fbe6e4468c0a4e3e", "fuel-ostf_sha": "7413186490e8d651b8837b9eee75efa53f5e230b", "release": "6.1", "fuelmain_sha": "6b5712a7197672d588801a1816f56f321cbceebd"

Test case:
1. Create new environment (CentOS)
2. Choose Neutron, GRE
3. Choose Ceilometer
4. Add 1 controller, 1 compute, 1 cinder, 2 mongo
5. Move Storage network to eth1
6. Move Management network to eth2 and untag it
7. Verify networks. It was successful
8. Deploy the environment. It was successful
9. Verify networks. It has failed with the same error

[Pavel Boldin (pboldin)]

The original report is about Neutron, VLAN.

Can you please aid me in implementing steps 5 and 6?

[Anastasia Palkina (apalkina)]

Pavel, you can see needed configuration of interfaces for all nodes on attached screenshot. Management network is untagged.

[Pavel Boldin (pboldin)]

STATUS UPDATE

The followings are the investigation results:
1. The bug is confirmed to be connected with leaving PRIVATE network on the interface where it is the only VLAN-tagged network.
2. The packets generated by the CentOS kernel in broken and working configurations are seems to be exactly the same.

These show that the bug is either in the VLAN driver or in the CentOS/VirtualBox interconnection.

The steps to try to fix the bug (w/o digging into the code):
1. Update CentOS kernel to the latest version and check if bug remains.
2. Enable openvswitch's workaround for a broken VLAN drivers and see if that helps the problem.

[Pavel Boldin (pboldin)]

The bug is the way `e1000' driver behaves by default in the CentOS + VirtualBox couple.

If there is *no* VLANs assigned to `eth0' then VLAN mode is *disabled* and all the VLAN-tagged eth packages are dropped.

This is not affecting production environments but only the `net_probe.py' from `fuel-web/network_checker'. Therefore, I'm moving that bug to a "Medium".

I'm looking for a way to enable VLANs by default at the moment and will provide this as a workaround till better solution is implemented by `net_probe.py'.

[Pavel Boldin (pboldin)]

The `net_probe.py' assumes that the VLAN mode is always *enabled* on all the Ethernet devices. This is wrong in that case making that assumption unreasonable.

The `e1000' driver disables VLAN mode when there is no VLANs active on the device and *does not* enables it even in promiscuous mode. While this can look like a bug it is only a way driver behaves. Making assumptions on the driver behaviour is a bad design pattern.

The solutions are either:
1. Fix `net_probe' so it adds VLANs it going to `listen' traffic on. This should be done using e.g. `vconfig' utility.
2. Fix `net_probe' so it uses `ethtool' to enable the VLAN modes on devices it is going to `listen' to.

[Dima Shulyak (dshulyak)]

Thanks Pavel, it makes sense, but this behaviour affects not only net_probe.py, but also private traffic between neutron endpoints (vm interfaces, qdhcp, qrouter). Some time ago we used "hard" splinters workaround, which is implementation of your 1st solution.

Please take a look at this comment https://bugs.launchpad.net/fuel/+bug/1457478/comments/26

[Pavel Boldin (pboldin)]

Dima, if this affects VM traffic then this is a bug in Neutron.

But I think it is not affecting them. If someone has a running environment please check that via healthcheck.

[Dima Shulyak (dshulyak)]

Exactly this was observed by me and Stas Makar. If you will run healthcheck atleast one test (that verifies network connectivity) will fail.
But obviously your private network should not be on the same interface as any tagged network. This is an example of valid configuration where bug can be reproduced:

eth0: pxe, private(vlan 1000-1030)
eth1: public, storage 102
eth2: management: 101

[Pavel Boldin (pboldin)]

Well, I'm unable to find that info in the thread. If you have really found out that network is not working beyond the network verification with `net_probe.py' why don't you make a note about it?

This makes the problem far more serious.

[Pavel Boldin (pboldin)]

Well, there is a workaround for that problem.

That is: one should enable at least one VLAN (even unused) at the interface during nodes start-up.

To do so I have created a configuration for a fake VLAN used on `eth0':
[root@node-47 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0.50
VLAN=yes
DEVICE=eth0.50
ONBOOT=yes
BOOTPROTO=none

And uploaded that file to all the nodes rebooting them after:
for n in 48 49 50 51; do scp ifcfg-eth0.50 node-$f:/etc/sysconfig/network-scripts; ssh node-$f reboot; done

After the cluster reboot both `network verification' and `Check network connectivity from instance via floating IP' do pass.

I think this information must be included into the release notes.

[Aleksey Kasatkin (alekseyk-ru)]

It is virtualbox-only and has a workaround ( https://bugs.launchpad.net/fuel/+bug/1457478/comments/44 ) so moving to 7.0 with 'medium' priority.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/6.1
Review: https://review.openstack.org/194961

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/194961
Committed: https://git.openstack.org/cgit/stackforge/fuel-docs/commit/?id=0e26e7d7cc153d179ec34985645dd23cdd239ddb
Submitter: Jenkins
Branch: stable/6.1

commit 5cc5f0c643aebecaf3bf4580535a3ea7c3334a6c
Author: Mike Scherbakov <email address hidden>
Date: Tue Jun 23 13:43:35 2015 -0700

    Removed streamlined patching backend pieces

    Change-Id: I955e76ccdbd12a9145f4e9b689f80bdf9fcaf929

commit 563c4b5c78ebfcb1f4f91047c2919f6270f9a1d4
Author: Mike Scherbakov <email address hidden>
Date: Tue Jun 23 13:30:30 2015 -0700

    Removed outdated patching guide

    Change-Id: I76180c277789ade9c5ebedd19fe2092847c0b7d9

commit 8d120c14bec1ab41d448683ad146a3053a57c4ee
Author: Irina Povolotskaya <email address hidden>
Date: Tue Jun 23 19:59:11 2015 +0300

    Add dual hypervisor ref arch into 6.1 docs

    Change-Id: I900c24c9de878eafadbfc995aa879b7f55737fac

commit feebd1592d3305b64bbdfd0bc5fe108190aef120
Author: OlgaGusarenko <email address hidden>
Date: Tue Jun 23 18:38:17 2015 +0300

    [OPs guide] Running Ceilometer section edits

    1. conf file extract is updated
    2. note is updated

    Closes-bug: 1467817
    Change-Id: I0217e164108e0ba6c1397045a5e57d13ff429223

commit 44a93f9dead7511a3461ec35248dbb689c81eafd
Author: OlgaGusarenko <email address hidden>
Date: Tue Jun 23 18:04:40 2015 +0300

    [RN6_1] Final changes

    1. capitalization
    2. 2014.2 to 2014.2.2
    3. general improvements

    Change-Id: I45057e90c90550559f66bc67ccdf97a559fd9000

commit bb41389cae58084285688853281516b659686422
Author: evkonstantinov <email address hidden>
Date: Tue Jun 23 16:45:35 2015 +0300

    Update patching decription

    Update patching description with
    the standard Linux commands.

    Change-Id: Ia1a8346639c468fdfce15a11d2430bf3a4731244

commit bf3018fae3f2e564413d33aba6cdebf8868f0b4e
Author: OlgaGusarenko <email address hidden>
Date: Tue Jun 23 15:55:49 2015 +0300

    [RN6_1] Clean up

    1. Rearranges sections
    2. Improves RST
    3. Changes titles order

    Change-Id: I6110bf515667d3d6ba08ad35ff5d593dbc96641e

commit 1c7e4457808e8f2d6c56fdf31252170972e444b9
Author: Maria Zlatkova <email address hidden>
Date: Tue Jun 23 15:26:28 2015 +0300

    Replaces VBOX screenshots

    This patch:
    - replaces VBOX screenshots
    - changes the link for Download Mirantis VirtualBox scripts
     to https://docs.mirantis.com/openstack/fuel/fuel-master/#downloads

    Change-Id: I58dede960c5c3355d39b07ff44b757403f6af02c
    Closes-Bug: #1467872

commit 0a568bf53fc0e25d1d692d5d74b4a7b4d983bbcc
Author: evkonstantinov <email address hidden>
Date: Tue Jun 23 14:01:55 2015 +0300

    6.1 --separate repos

    change wording and add links to the
    separate repos feature.

    Change-Id: Ib5d0778a0d8f1534f79ed2f553574cb69a3150b0

commit 95a188b21cbdd064d92696b7920e6a0105fe0c56
Author: Maria Zlatkova <email address hidden>
Date: Tue Jun 23 12:07:28 2015 +0300

    Corrects the output 'pcs status'

    Changes the example outputs to appropriate ones.

    Change-Id: Ib6d83...

[Vladimir Sharshov (vsharshov)]

Do not support CentOs on this release. Moving to 8.0. Also we have workaround: https://bugs.launchpad.net/fuel/+bug/1457478/comments/44

[Dmitry Pyzhov (dpyzhov)]

We still don't support CentOS and we've made a lot of changes in related code. Closing bug as Won't Fix. Let's create new bug if we get similar issue in future.