Cannot change vip and vrouter_vip IPs when we are using nodegroups

[Mykola Grygoriev]

Not all parameters are applied during network.yaml uploading. They are: management_vip, management_vrouter_vip, public_vip and public_vrouter_vip.

VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "6.1"
  openstack_version: "2014.2.2-6.1"
  api: "1.0"
  build_number: "525"
  build_id: "2015-06-19_13-02-31"
  nailgun_sha: "dbd54158812033dd8cfd7e60c3f6650f18013a37"
  python-fuelclient_sha: "4fc55db0265bbf39c369df398b9dc7d6469ba13b"
  astute_sha: "1ea8017fe8889413706d543a5b9f557f5414beae"
  fuel-library_sha: "2e7a08ad9792c700ebf08ce87f4867df36aa9fab"
  fuel-ostf_sha: "8fefcf7c4649370f00847cc309c24f0b62de718d"
  fuelmain_sha: "a3998372183468f56019c8ce21aa8bb81fee0c2f"

Steps to reproduce:

1. Create environment with Ubuntu+Neutron+GRE
fuel env --create --name test --net neutron --nst gre --release 2

2. Create new nodegroup for this environment
fuel nodegroup --env 1 --create --name rack2

3. Download network.yaml of the environment.
fuel network --env 1 --download

4. Edit networks and set management_vip, management_vrouter_vip, public_vip and public_vrouter_vip from NOT default nodegroup networks in network_1.yaml. I've attached final edited network_1.yaml.

5. Upload network_1.yaml to the environment.
fuel network --env 1 --upload network_1.yaml

Expected behavior:
All settings are successfully applied to the environment 1.

Current behaviour:
All settings are applied to the environment 1, except management_vip, management_vrouter_vip, public_vip and public_vrouter_vip. They are still from the default nodegroup networks.
To check this we should again download network settings from the environment 1
fuel network --env 1 --download

[Przemyslaw Kaminski (pkaminski)]

For start, you are changing the management_vip without modifying the cidr:

- cidr: 192.168.0.0/24
  gateway: null
  group_id: 1
  id: 3
  ip_ranges:
  - - 192.168.0.1
    - 192.168.0.254
  meta:
    cidr: 192.168.0.0/24
    configurable: true
    map_priority: 2
    name: management
    notation: cidr
    render_addr_mask: internal
    render_type: cidr
    use_gateway: false
    vips:
    - haproxy
    - vrouter
    vlan_start: 101
  name: management
  vlan_start: 10

I guess this should be validated by the API too. BTW, this seems related to https://bugs.launchpad.net/fuel/+bug/1455610

[Przemyslaw Kaminski (pkaminski)]

ditto https://bugs.launchpad.net/fuel/+bug/1455610/comments/2 and https://bugs.launchpad.net/fuel/+bug/1455610/comments/3

[Przemyslaw Kaminski (pkaminski)]

Yeah, you have 2 public networks in that YAML file. It's incorrect, you better provide IP ranges and adjust the CIDR to 172.16.0.0/16.

[Przemyslaw Kaminski (pkaminski)]

BTW, I checked: you should change the cidr and ip ranges of the management network and then the vips are updated automatically.

[Mykola Grygoriev (mgrygoriev)]

Nodegroup feature allows us to have public network in different L3 segments, doesn't it? In real life we can have the situation when public network with group_id =1 is 6.6.6.0/24 and public network with group_id =2 is 205.205.205.0/24. So, in this case we need to have possibility to set public_vip and public_vrouter_vip from any of public networks. The same is for management network. In this case we cannot provide IP ranges and adjust CIDR to 0.0.0.0/0, can we?

[Łukasz Oleś (loles)]

First thing you can not manually set VIPs. VIPs are auto generated. If you want to have VIPs from non default node group you need first to assign nodes with controller role to this group.

So you should first add controllers node to env. Then assign them to your nodegroup:

fuel --env 1 nodegroup --assign --node 1 --group 2

Now, when you will download the settings you will see that VIPs are set correctly.

[Łukasz Oleś (loles)]

It's undocumented so I'm reasigning it to fuel-doc team.

[Aleksey Kasatkin (alekseyk-ru)]

VIPs cannot be changed via API for now. It is not only about multi node groups. It is the same for single (default) node group.

Related bug (but not the same): https://bugs.launchpad.net/fuel/+bug/1487021

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/216801

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/249656

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/251842

[Aleksey Kasatkin (alekseyk-ru)]

It is a feature request and it is FF now for 8.0 so moving this to 9.0.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/257919

[OpenStack Infra (hudson-openstack)]

Change abandoned by Igor Kalnitsky (<email address hidden>) on branch: master
Review: https://review.openstack.org/216801
Reason: No updates for 4 months.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Ivan Kliuk (<email address hidden>) on branch: master
Review: https://review.openstack.org/249656
Reason: The change-set is not relevant anymore.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/262245

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/268282

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/257919
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=169fae21b0b26db8eb4777936c83e8cae31f1aa9
Submitter: Jenkins
Branch: master

commit 169fae21b0b26db8eb4777936c83e8cae31f1aa9
Author: Ivan Kliuk <email address hidden>
Date: Sat Dec 5 17:16:38 2015 +0200

    New VIP-related fields in the database

    * Add 'is_user_defined' field to 'ip_addrs' table.
    * Rename 'vip_type' field to 'vip_name' of table 'ip_addrs'.
    * Add 'vip_namespace' field to 'ip_addrs' table.
    * Copy vip namespaces from plugin table network roles to 'vip_namespace' field
      according to the unique vip name.
    * Add database migrations.
    * Add unit test for migrations.

    Change-Id: Ia3e1d7f6e08dbebcb182de75eeaf58ddf6be4a8d
    Partial-bug: #1482399

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/251842
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=8f28676cf61599380ecafc7db1bbacbc36988a93
Submitter: Jenkins
Branch: master

commit 8f28676cf61599380ecafc7db1bbacbc36988a93
Author: Ilya Kutukov <email address hidden>
Date: Wed Dec 16 14:25:37 2015 +0300

    Basic VIP management handlers added to nailgun API

    New VIP handlers is introduced that allows list VIP for cluster and get or update
    specific VIP.

    /clusters/:cluster_id/network_configuration/ips/vips
    /clusters/:cluster_id/network_configuration/ips/:ip_address_id/vips

    Partial-Bug: #1482399
    Implements Blueprint: allow-any-vip
    Change-Id: I4a4058c112bbdb26ab410e5e97b4e1e3afe7a082

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/277636

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/262245
Committed: https://git.openstack.org/cgit/openstack/python-fuelclient/commit/?id=31190a8d38d493eb525c23b1254816c818fa9947
Submitter: Jenkins
Branch: master

commit 31190a8d38d493eb525c23b1254816c818fa9947
Author: Ilya Kutukov <email address hidden>
Date: Tue Dec 29 18:38:37 2015 +0300

    Basic VIP management commands added to Nailgun CLI v1

    Now Naigun CLI supports VIP configuration management commands
    that allow to download and upload VIP configuration:

    fuel vip --env 1 --ip-address-id 1 --download
    fuel vip --env 1 --upload ip_address.yaml

    Partial-Bug: #1482399
    Implements Blueprint: allow-any-vip

    Change-Id: Id5f22228b007d5881d8d1db4725772ef0ca3f9ce

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/277636
Committed: https://git.openstack.org/cgit/openstack/python-fuelclient/commit/?id=c45bd4ebb96b7fff91a00e6e8336bcf2053d3386
Submitter: Jenkins
Branch: master

commit c45bd4ebb96b7fff91a00e6e8336bcf2053d3386
Author: Ilya Kutukov <email address hidden>
Date: Tue Feb 9 03:39:40 2016 +0300

    Basic VIP management commands added to Nailgun CLI v2

    Now Naigun CLI v2 supports VIP configuration management commands
    that allow to download and upload VIP configuration:

    fuel2 vip download --env 1 --ip-address-id 1 --file firstvip.yaml
    fuel2 vip upload --env 1 --file ip_address.yaml

    Partial-Bug: #1482399
    Implements Blueprint: allow-any-vip

    Change-Id: Ib9b15bfdb7d4514919efbac2dab4416d088aa1e9

[Artem Roma (aroma-x)]

Functionality that allows creation/management of user defined VIPs was added for both Nailgun and fuel client.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/305748

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/305748
Committed: https://git.openstack.org/cgit/openstack/fuel-docs/commit/?id=0e73696c5f6b193f85c539e337eb42f50145f4f4
Submitter: Jenkins
Branch: master

commit 0e73696c5f6b193f85c539e337eb42f50145f4f4
Author: Evgeny Konstantinov <email address hidden>
Date: Thu Apr 14 13:48:50 2016 +0300

    Add VIP management/creation note to Fuel Mitaka relnotes

    Change-Id: I875b667712f8acfb408539d1d72b36a82a0c668c
    Related-Bug: #1482399

[Mikhail Samoylov (msamoylov)]

Verified in 402 MOS ISO
Steps:
1. Create cluster
2. Assign some nodes to cluster
3. Do like this:
http://paste.openstack.org/show/506293/

[JohnsonYi (yichengli)]

Same issue on fuel 8.0, add new node group and delete it, that cause the vip ip for b_vrouter changed. and ntp update can't work during new node deployment.

any progress for fuel 8.0?

[JohnsonYi (yichengli)]

and also will cause the management/storage/baremetal networks have extra gateway generated on fuel web.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/360086

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/360086
Committed: https://git.openstack.org/cgit/openstack/fuel-nailgun-extension-cluster-upgrade/commit/?id=40dd411fe461441e943de806f5de632eeb64dc4a
Submitter: Jenkins
Branch: stable/mitaka

commit 17ab5a3aaa7d2c4ae4652d1a4dc0410b31e7c47d
Author: Sergey Abramov <email address hidden>
Date: Wed Aug 24 18:32:39 2016 +0300

    Add create upgrade release handler

    Required for create new release just for upgrade, that have overwrited
    params. This params are valid for orig cluster release.

    Change-Id: Ib2387b9c2b74902c7289ee8f69a5f5d323ec82ca

commit 8de47e0dbf5955b3cbe59a3956e1ca1f4d8679ef
Author: Nikita Zubkov <email address hidden>
Date: Wed Aug 24 15:42:32 2016 +0300

    Add VIPs transformer

    Move upgrade vips code to the transformer

    Change-Id: Ia6935521b8b90b73fb74ba92859e7febf74c4ced

commit b54f9d4c29808c4561b20af508b85b9e40fa72b6
Author: Nikita Zubkov <email address hidden>
Date: Wed Aug 24 17:51:39 2016 +0300

    Move partition info transformation to extension

    octane have some dirty hacks to change volumes attributes using nailgun
    as a library, such modifications should be done in a extension

    Change-Id: I422bb368916f3a319e286edcc6103a2834097a87

commit b3ce0d348cc20ed988b5adf66474836f631eb12d
Author: Anastasiya <email address hidden>
Date: Mon Aug 1 15:47:13 2016 +0300

    Add tests for assign_node_to_cluster

    Change-Id: Iafa1baa6a1ca4d701ec89e49dd9d6f969804c82e

commit a764ba306dcd0cfb65a82c89f07ec0f3d129c66b
Author: Nikita Zubkov <email address hidden>
Date: Tue Aug 23 14:05:16 2016 +0300

    Update README

    Add instalation section

    Change-Id: Ib16132ab9c18d757e96e98304fdd8339ddf5497a

commit 95ff3a3598397bb6e3c68793051298aaa006d163
Author: Yuriy Taraday <email address hidden>
Date: Tue Aug 23 23:01:16 2016 +0300

    Add cluster transformations

    Implement transformations that are applied to cluster attributes during
    environment cloning.

    Conversion from text to text_list type has been limited to dns_list and
    ntp_list keys only to keep predictable behavior.

    Change-Id: I1ff596f850bd42243697cad1c1c35f0cf1386376

commit 163ce243fbade3dac05eb535ad2987687a57f87d
Author: Yuriy Taraday <email address hidden>
Date: Tue Aug 23 22:51:19 2016 +0300

    Add pluggable transformations for data migration

    This change introduces new transformation mechanism:
    - all available transformations are listed in setuptools entry points
      under namespace like this (for cluster transformations):

        nailgun.cluster_upgrade.transformations.cluster.9.0 =
            dns_list = ...
            ntp_list = ...
        nailgun.cluster_upgrade.transformations.cluster.8.0 =
            ...
        <etc>

    - config file will include section that specifies enabled
      transformations like this:

        CLUSTER_UPGRADE_TRANSFORMATIONS:
          cluster:
            9.0: dns_list ntp_list ...
            8.0: ...
            7.0: ...

      (only default values are implemented here, actual config support will
      follow)

    - when transformations...

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/367019

[OpenStack Infra (hudson-openstack)]

Change abandoned by Ilya Kharin (<email address hidden>) on branch: master
Review: https://review.openstack.org/367019

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/367419

[OpenStack Infra (hudson-openstack)]

Change abandoned by Ilya Kharin (<email address hidden>) on branch: master
Review: https://review.openstack.org/367419

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/367019
Committed: https://git.openstack.org/cgit/openstack/fuel-nailgun-extension-cluster-upgrade/commit/?id=e38d48cbc5d216ac0f853466ab48e5d6d9687521
Submitter: Jenkins
Branch: master

commit a4e2a67e3e5024b0ae65f445355965a1263fef73
Author: Ilya Kharin <email address hidden>
Date: Sat Sep 3 02:42:07 2016 +0300

    Add support to re-assign a set of nodes

    This patch adds an ability to re-assign a set of the given nodes at
    once. This feature was technically available but not exposed to the
    client. A groupped re-assigning allows to effectively re-provision nodes
    by creating an atomic task in Astute.

    Change-Id: I4a7c7e35d844683ef73ad7f8459d1892e80e0a64
    Related-Bug: #1616925

commit d87125662f2f55204244e43ec6522fe36c6bf21e
Author: Nikita Zubkov <email address hidden>
Date: Mon Aug 29 17:56:09 2016 +0300

    Add test for vip transformer

    Change-Id: I65124237604fe6718ad6c351825f192a98d470fb
    (cherry picked from commit 55422ddce7b0e9e1b9976504500de8a1688a99d7)

commit 3cac9b551cdecaf9148800cc6fd9b81fbb3149c5
Author: Nikita Zubkov <email address hidden>
Date: Tue Jul 19 18:24:42 2016 +0300

    Switch to upstream fuel-web repository

    Change-Id: I994304bdc8eaf7e4da175981cb721d41a286fed0
    Depends-On: Id0bc78478cf3f40767fed760cd54e487a934fa10

commit a78f4348f84dd470ba1f3b99d2c751fa2dd12d7a
Author: Anastasiya <email address hidden>
Date: Mon Aug 1 14:39:35 2016 +0300

    Move change_env_settings function from octane to cluster upgrade extension

    * change_env_settings function was moved to cluster upgrade extention
    * merge generated attributes code was written

    Change-Id: I6d1e27b8b0c01f3251067bc88931cd2354feb5ce
    Partial-Bug: #1602587
    (cherry picked from commit dc2e3f930957b2c8af2d6c6a60bfcc6c5e6bb061)

commit 280fc4f08258f1e85ba099f74c4956233652e9a2
Author: Ilya Kharin <email address hidden>
Date: Wed Aug 10 17:28:17 2016 +0300

    Add absent __init__.py to migrations/versions

    Without the versions/__init__.py file versions was not identified as
    a package and was not included in a distribution.

    Change-Id: I67f152ebb9234df880c61d79d154b1aabc8828c6
    Closes-Bug: #1611793

commit f7ebb08b46f5beb13701f7a6a71a1f4fea05f451
Author: Alexander Tsamutali <email address hidden>
Date: Mon Aug 1 15:47:58 2016 +0300

    Add package spec

    Change-Id: Id71764dff07a4b32851eb8ccf69c66dca4a7b6ab
    Related-Bug: #1604492

commit b93ebedc49f79f6ba4a710a9d1715c9f965b3081
Author: Anastasiya <email address hidden>
Date: Fri Jul 15 10:24:11 2016 +0300

    Correction of transformation for text_list

    * added removing of space in text_list
    * added test for merge_attributes

    Change-Id: I5582878fc7c524551593abf21dfd4ea45cd430c9
    Closes-bug: 1602607
    (cherry picked from commit fdd2a6226483c67ce8bc7adc8b2d354862125bac)

commit d4db5ba78ccefd08e465cd30116094678c5cb35f
Author: Nikita Zubkov <email address hidden>
Date: Wed Jul 13 13:43:58 2016 +0300

    Fix package namespace

commit 443fc43da6e963cd0825880a80678e8e385c0a3a
Author: Nikita Zubkov <email address hidden>
Date: Wed J...