Fuel should support multiple floating ranges

[Dmitry Tyzhnenko]

If user setup several floating ranges in network settings in CLI cluster has only first range.

Scenario:
  1. Create cluster with Neutron
  2. Add addional ranges in CLI mode
  3. Add 3 Controllers
  4. Add 1 Compute
  5. Add 1 Cinder
  6. Verify network
  7. Deploy cluster
  8. Verify network
  9. Run OSTF

Actual result:
  Subnet net04_ext__subnet has only first range

Expected result:
  Subnet net04_ext__subnet has all ranges from settings

network config and neutron output - http://paste.openstack.org/show/434851/

astute.yaml on controller doesn't contain floating ranges which presented in network config in environment

env T1722189 (id:5) in logs snapshot - https://drive.google.com/a/mirantis.com/file/d/0B8U7EvTbuAOlVVNZelZxejRVNE0/view?usp=sharing

node list - http://paste.openstack.org/show/434878/

[Nastya Urlapova (aurlapova)]

If we don't support such functionality in puppet, as minimum we have to deprecate it in API.

[Aleksey Kasatkin (alekseyk-ru)]

Multiple floating ranges are not supported in manifests for now.

Please look at comments in https://bugs.launchpad.net/fuel/+bug/1371363

Now (in 7.0) we could just disable this ability in API. It was disabled in Nailgun's serializer and UI already.

[Nastya Urlapova (aurlapova)]

Aleksey, could you clarify - how it was disabled? you see in description we were able use it via cli.

[Ihor Kalnytskyi (ikalnytskyi)]

@Nastya,

I believe Aleksey means:

1. Move this issue to 8.0, since the fix requires changes in both Nailgun and puppet manifests.

2. Fix bug #1490657 instead, so Nailgun reject any attempt to set multiple ranges for Neutron ext network.

Are you ok with this solution?

[Ihor Kalnytskyi (ikalnytskyi)]

I think the best option's here is to mark it as Won't fixed for 7.0, and fix it in 8.0.

[Ihor Kalnytskyi (ikalnytskyi)]

Well, this is an artificial restriction in Nailgun due to lack of functionality in Library side. Moving it to Fuel Library, since it should be fixed there first.

[Matthew Mosesohn (raytrac3r)]

You're asking for a feature to support multiple floating ranges. This is a feature, so we should treat it as such. Lowering to medium priority because this doesn't break deployment

[Dmitry Pyzhov (dpyzhov)]

By default we don't backport feature requests

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/235879

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/237021

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/237021
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=56f55f373aff9a115a92414a6395780e75860d03
Submitter: Jenkins
Branch: master

commit 56f55f373aff9a115a92414a6395780e75860d03
Author: Stanislav Makar <email address hidden>
Date: Fri Oct 16 14:02:01 2015 +0000

    Enable rspec unit tests for osnailyfacter module

    Fix broken tests

    Change-Id: Icee2c6aaf861063fcbccaff4fddbecf377a0695f
    Partial-bug: #1490578
    Co-authored-by: Dmitry Ilyin <email address hidden>

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/239883

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/235879
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=cf54da03b19c9b3b87cc51cfde519b1ee5428de7
Submitter: Jenkins
Branch: master

commit cf54da03b19c9b3b87cc51cfde519b1ee5428de7
Author: Stanislav Makar <email address hidden>
Date: Fri Oct 16 11:55:02 2015 +0000

    Add multiple floating ranges support

    *Add new puppet function which formats allocation pools string for neutron
    subnet resource.
    *Remove ordering between neutron_network and neuton_subnet due to there
    is autorequire in puppets types.
    *Remove unneeded variable.

    Change-Id: I040c3bc66fa0f57ed6f79d8471bd0bd239efc527
    Closes-bug: #1490578

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/239883
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=915a59d82c98df961abc60ea93c5c7451e2744bf
Submitter: Jenkins
Branch: master

commit 915a59d82c98df961abc60ea93c5c7451e2744bf
Author: Ivan Kliuk <email address hidden>
Date: Wed Oct 28 11:40:31 2015 +0200

    Multiple floating ranges support

    * Remove multiple floating ranges checking from validation layer.
    * Rename '_generate_external_network' to 'generate_external_network'.
    * Add custom 'generate_external_network' for 'NeutronNetworkDeploymentSerializer80'.
    * Change 'NetworkCheck.neutron_check_network_address_spaces_intersection'
      functionality. Now it checks all the floating ranges belong to public network.
    * Remove irrelevant unit tests.
    * Add unit tests for 'NeutronNetworkDeploymentSerializer80.generate_external_network'.

    Change-Id: Ib277506db3c96890bc3ce2b4f312edd10841a95b
    Partial-bug: #1490578

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/242032

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/242032
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=5f7a43417a019f5451348f3c45c2fb564383a291
Submitter: Jenkins
Branch: master

commit 5f7a43417a019f5451348f3c45c2fb564383a291
Author: Ivan Kliuk <email address hidden>
Date: Tue Nov 3 19:17:24 2015 +0200

    [Tech debt] Multiple floating ranges support

    * Add 'objects.Release.is_multiple_floating_ranges_enabled' method.
    * According to open/closed principle, move logic of 'generate_external_network'
      method to 'NeutronNetworkDeploymentSerializer80' and revert changes made in
      'NeutronNetworkDeploymentSerializer51'.
    * Refactor and remove testing redundancy in 'TestNeutronDeploymentSerializer70'
      and 'TestNeutronDeploymentSerializer80'.

    Change-Id: I993c22bf2ca65f8e3c01301c92e3f0987b403244
    Partial-bug: #1490578

[Mikhail Samoylov (msamoylov)]

Bug was reproduced in fuel version.

release_versions:
  2015.1.0-8.0:
    VERSION:
      api: '1.0'
      astute_sha: 959b06c5ef8143125efd1727d350c050a922eb12
      build_id: '143'
      build_number: '143'
      feature_groups:
      - mirantis
      fuel-agent_sha: 7027b9c31f1986c3904877a2133b1fcb77094beb
      fuel-createmirror_sha: d3949f3094248b9686a4c20caea5ae13c9738a8d
      fuel-library_sha: 0db9fb73a6ebc25098be59151457b24018e967b6
      fuel-nailgun-agent_sha: 00b4b11553c250f22c0079fb74c8b782dcb7b740
      fuel-nailgun_sha: 050633358cacb4023e84fda0367e68858051f702
      fuel-ostf_sha: bc0844c493490ddbe9ca1bb5d9f88c068772c7e3
      fuel-upgrade_sha: 1e894e26d4e1423a9b0d66abd6a79505f4175ff6
      fuelmain_sha: 3b64b95f6b513b0d46b57525a858eea7a9809da3
      fuelmenu_sha: a133a8ae1621f7700a8123295681b8d3c98f74a4
      network-checker_sha: a57e1d69acb5e765eb22cab0251c589cd76f51da
      openstack_version: 2015.1.0-8.0
      production: docker
      python-fuelclient_sha: ee31f93f047a54b3bd2ab5d281c214bbd53f9156
      release: '8.0'
      shotgun_sha: 25dd78a3118267e3616df0727ce746e7dead2d67
shotgun_sha: 25dd78a3118267e3616df0727ce746e7dead2d67

[Mikhail Samoylov (msamoylov)]

Fuel snapshot link.

https://drive.google.com/a/mirantis.com/file/d/0B2SenDuhfXPlcGtTWktUazJsVms/view?usp=sharing

[Bogdan Dobrelya (bogdando)]

Please elaborate where is a known solution given or a patch? You put the triaged state meaning there is one, right?

[Stanislav Makar (smakar)]

I see that etc/astute.yaml on all nodes has only one range
  L3:
        nameservers: []
        subnet: 10.109.11.0/24
        floating:
        - 10.109.11.128:10.109.11.254
        gateway: 10.109.11.1
        enable_dhcp: false

that is way you got only one range

looks like the problem on nailgun side or network config with multiple floating ranges was no uploaded correctly

[Mikhail Samoylov (msamoylov)]

Output for network config uploading:
[root@nailgun ~]# fuel rel --rel 2 --network --upload --debug
PUT http://10.109.10.2:8000/api/v1/releases/2/networks data={"bonding": {"availability": [{"linux": "settings:storage.iser.value == false and settings:neutron_mellanox.plugin.value != 'ethernet'"}, {"ovs": "false"}], "properties": {"linux": {"lacp_rate": [{"for_modes": ["802.3ad"], "values": ["slow", "fast"]}], "mode": [{"values": ["balance-rr", "active-backup"]}, {"values": ["802.3ad"], "condition": "'experimental' in version:feature_groups or interface:pxe == false"}, {"values": ["balance-xor", "broadcast", "balance-tlb", "balance-alb"], "condition": "'experimental' in version:feature_groups"}], "xmit_hash_policy": [{"for_modes": ["802.3ad", "balance-xor", "balance-tlb", "balance-alb"], "values": ["layer2", "layer2+3", "layer3+4", "encap2+3", "encap3+4"]}]}}}, "nova_network": {"config": {"fixed_network_size": 256, "fixed_networks_cidr": "10.109.13.0/24", "fixed_networks_vlan_start": 103, "net_manager": "FlatDHCPManager", "floating_ranges": [["10.109.11.128", "10.109.11.254"]], "fixed_networks_amount": 1}, "networks": [{"vips": ["haproxy", "vrouter"], "name": "public", "notation": "ip_ranges", "render_type": null, "map_priority": 1, "configurable": true, "ip_range": ["10.109.11.2", "10.109.11.127"], "vlan_start": null, "render_addr_mask": "public", "cidr": "10.109.11.0/24", "gateway": "10.109.11.1", "use_gateway": true}, {"vips": ["haproxy", "vrouter"], "name": "management", "notation": "cidr", "render_type": "cidr", "map_priority": 2, "configurable": true, "use_gateway": false, "vlan_start": 101, "render_addr_mask": "internal", "cidr": "10.109.12.0/24"}, {"name": "storage", "notation": "cidr", "render_type": "cidr", "map_priority": 2, "configurable": true, "use_gateway": false, "vlan_start": 102, "render_addr_mask": "storage", "cidr": "10.109.14.0/24"}, {"ext_net_data": ["fixed_networks_vlan_start", "fixed_networks_amount"], "name": "fixed", "notation": null, "render_type": null, "map_priority": 2, "configurable": false, "use_gateway": false, "vlan_start": null, "render_addr_mask": null}]}, "neutron": {"config": {"parameters": {"database": {"passwd": "", "username": null, "database": null, "port": "3306", "provider": "mysql"}, "keystone": {"admin_user": null, "admin_password": ""}, "amqp": {"passwd": "", "username": null, "hosts": "hostname1:5672, hostname2:5672", "provider": "rabbitmq"}, "metadata": {"metadata_proxy_shared_secret": ""}}, "internal_cidr": "10.109.13.0/24", "floating_ranges": [["10.109.11.13", "10.109.11.23"], ["10.109.11.24", "10.109.11.34"], ["10.109.11.35", "10.109.11.45"], ["10.109.11.128", "10.109.11.254"]], "floating_name": "admin_floating_net", "base_mac": "fa:16:3e:00:00:00", "internal_gateway": "10.109.13.1", "internal_name": "admin_internal_net", "gre_id_range": [2, 65535], "vlan_range": [1000, 1030]}, "networks": [{"vips": ["haproxy", "vrouter"], "name": "public", "notation": "ip_ranges", "render_type": null, "map_priority": 1, "configurable": true, "floating_range_var": "floating_ranges", "ip_range": ["10.109.11.2", "10.109.11.127"], "vlan_start": null, "render_addr_mask": "public", "cidr": "10.109.11.0/24", "gatew...

[Mikhail Samoylov (msamoylov)]

Verificated in fuel version:
release_versions:
  2015.1.0-8.0:
    VERSION:
      api: '1.0'
      astute_sha: 959b06c5ef8143125efd1727d350c050a922eb12
      build_id: '172'
      build_number: '172'
      feature_groups:
      - mirantis
      fuel-agent_sha: 61f741f4d89a3f1095dd8b494f82695fb69153d6
      fuel-createmirror_sha: 22a7aacd95bbdca69f9e0f08b70facabdec8fb28
      fuel-library_sha: 31aa9fb129a8b8739f8ca6ea186caa3c9f46820f
      fuel-nailgun-agent_sha: 3e9d17211d65c80bf97c8d83979979f6c7feb687
      fuel-nailgun_sha: f8ba3f9cd449363f1c08621307e7504c304c29ae
      fuel-ostf_sha: 9690a2de829d3b063ed1e64b0b10dde39f711dc0
      fuel-upgrade_sha: 1e894e26d4e1423a9b0d66abd6a79505f4175ff6
      fuelmain_sha: 266f9b374934c64629a84b5146632cc0de70ac91
      fuelmenu_sha: 06bbcebed6c8d0b0f9279e2997d2f958c800e98c
      network-checker_sha: a57e1d69acb5e765eb22cab0251c589cd76f51da
      openstack_version: 2015.1.0-8.0
      production: docker
      python-fuelclient_sha: e685d68c1c0d0fa0491a250f07d9c3a8d0f9608c
      release: '8.0'
      shotgun_sha: c377d163519f6d10b69a654019d6086ba5f14edc
shotgun_sha: c377d163519f6d10b69a654019d6086ba5f14edc

Testing steps:
fuel env create --name myenv --rel 2 --nst vlan
fuel network --env 1 --download
vim /root/network_1.yaml - adding floating ranges
fuel network --env 1 --upload
 fuel node set --node 1,2 --role controller --env 1
fuel node set --node 4 --role compute --env 1
fuel network --env 1 --verify
fuel task
fuel --env=1 deployment --default
vim /root/deployment_1/primary-controller_1.yaml - check that floating ranges in yaml file
fuel --env 1 deploy-changes
 fuel network --env 1 --verify
fuel task
ssh root@10.109.10.6
root@node-2:~# egrep "10.109.11.(13|24|35)" /etc/astute.yaml
        - 10.109.11.13:10.109.11.23
        - 10.109.11.24:10.109.11.34
        - 10.109.11.35:10.109.11.45