[RFE][deployed-server][split-stack] openstack provider for multinode/pre-provisioned development environments

[Bogdan Dobrelya]

Note, this is not related to OVB/CI devmode RFEs like https://bugs.launchpad.net/tripleo/+bug/1657191

This extends the https://blueprints.launchpad.net/tripleo/+spec/split-stack-software-configuration (Split Stack Phase 2): "deploy OpenStack on already deployed baremetal nodes. We have this functionality today with the deployed-server templates, but we need to harden this work and make it more consumable outside of CI." as the following below.

There is also a nice write-up explaining the use case https://blog-slagle.rhcloud.com/?p=299

The user story is "Infrastructure scripts are doing many different things that I am as a developer don't need/want to do. Instead I want to deploy overcloud openstack on pre-provisioned VMs, using ansible playbooks only (composed from quickstart roles)"

The use case is:
* given a set of VM instances on generic openstack cloud provider, deployed either as a heat stack or the like (for example, like Traas does for RDO cloud) and given access creds/url to that cloud;
* given a dynamic openstack inventory generated by openstack.py from ansible repo;
* given a custom ansible play comprising quickstart/extras roles, with custom vars included,

* to DEPLOY a multinode (like tripleo-ci multinode scenarios do for nodepool, but here VM nodes pre-provisioned by hosting openstack cloud), based on the deployed-server quickstart-extras role;
* and having applied the following shortcuts for development environments/hacking/testing WIP patches:
  * No images/containers/packages to be built, no VMs provisioned with Ironic, and no repos or images to be injected (the deployed-server extras role covers that case, it seems)
  * Dependencies should not be cloned & built as artifacts with zuul_cloner et al. Instead, custom undercloud/overcloud install scripts are expected to contain all the steps needed to get those installed from dev forks into the dev env being deployed, see an example https://github.com/bogdando/tripleo-quickstart-extras/blob/dev/roles/undercloud-deploy/templates/undercloud-deploy-dev.sh.j2#L53

Caveats:
  * No tripleo-ci/quickstart shell scripts used, only ansible-playbook commands wanted.
  * Deployment split into the steps: an undercloud may be deployed as a separate, then deployment continued to finish multinode overcloud as well.
  * Ansible-playbook commands may overlap for long running undercloud deploy and overcloud-prep and repo-setup tasks, bringing a drastic shortcut to deploy time. Example https://github.com/bogdando/oooq-warp/commit/d0b068f66b8ff92c5ba34541f24f235ab7dd0c97

[Bogdan Dobrelya (bogdando)]

Addressed by: https://review.openstack.org/#/q/topic:custom_undercloud

[OpenStack Infra (hudson-openstack)]

Change abandoned by Bogdan Dobrelya (<email address hidden>) on branch: master
Review: https://review.openstack.org/465047
Reason: this change is not needed

[wes hayutin (weshayutin)]

This appears to need a little more clarification, perhaps a tripleo spec would help illuminate what the RFE is requesting.

It also may be adding too much to the CI vs. the project itself. I suspect an implementation that CI's https://review.openstack.org/#/c/461871/2/doc/source/advanced_deployment/split_stack.rst would be one step forward here.

Another step forward would be to provide provisioned overcloud nodes with libvirt, and via a host cloud.

Another step forward would be to add documentation for advanced users of quickstart of how to only use ansible-playbook commands vs.. quickstart.sh

Bogdan, we would like to invite you to help us understand your RFE. We suspect that rewriting this as a tripleo spec may help provide the context required for an in person discussion.

Thanks

[Bogdan Dobrelya (bogdando)]

@wes, this is supposed to be a simple glue for existing approaches and repos: traas heat templates for provisioning, quickstart and t-h-t for repos setup and software deployment. Not a rocket science, although the description may appear to be looking like it is... But it is really not, nothing to invent, design, just glue things with custom data provided and minot fixes to address gaps in quickstart roles. I strongly beleive this could benefit all developers as a viable alternative to ovb devmode, and doesn't require a full blown spec as it has no impacts but for developers one

[Bogdan Dobrelya (bogdando)]

Note, split-stack covers not only dev envs but production deployment options, IIUC. That's why it needs a full spec, unlike to this one, which is a very narrow subset of the latter.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/450792
Committed: https://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/commit/?id=f2114bab76bb6eb623e9b993280fb03243fdeca1
Submitter: Jenkins
Branch: master

commit f2114bab76bb6eb623e9b993280fb03243fdeca1
Author: Bogdan Dobrelya <email address hidden>
Date: Tue May 16 13:48:39 2017 +0200

    Custom containerized undercloud deployments

    * Parametrize network settings for undercloud deploy to align with
      the undercloud install's undercloud.conf and nova-join's DNS
      config steps.
    * Move the undercloud nameservers defaults to the extras-common.
    * Support extra deploy args for 'openstack undercloud deploy'.
      Defaults to the heat-native mode, may be altered to containerized
      heat-all agents, and other args like keep-running, if wanted.
    * Support custom t-h-t templates for underclouds as well (shares
      the custom t-h-t script and vars with the overcloud-prep-config)
    * Template prerequisite packages installed by the
      undercloud-install.sh script. These should be moved to the
      tripleoclient's 'openstack undercloud deploy', eventually.
    * Allow requested dev env packages to be updated by the
      undercloud-install.sh script (defaults to update nothing,
      which is backwards compatible). Note, this has nothing to the
      prerequisite packages.
    * Document missing variables for the undercloud-deploy role.
    * Document hacking dev branches for customized underclouds.

    Related-bug: #1691467
    Signed-off-by: Bogdan Dobrelya <email address hidden>
    Change-Id: Id32f492369dc74428bde8c4120308279cbba53fc

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/480263

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/480264

[OpenStack Infra (hudson-openstack)]

Change abandoned by Bogdan Dobrelya (<email address hidden>) on branch: master
Review: https://review.openstack.org/480264
Reason: https://review.openstack.org/#/c/480263

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/480263
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=c6cd190a9f63738deafa4c616e0be8e06afedc96
Submitter: Jenkins
Branch: master

commit c6cd190a9f63738deafa4c616e0be8e06afedc96
Author: Bogdan Dobrelya <email address hidden>
Date: Thu May 25 13:55:03 2017 +0200

    Allow overcloud deploy on UC w/o nova/ironic

    Add the --deployed-server flag that can only be used
    with --disable-validations. When specified, allows to
    deploy overcloud on pre-provisioned nodes and ignores
    missing nova/ironic UC services.

    Closes-bug: #1693448
    Related-bug: #1691467

    Change-Id: I0122106a7b45843d4147cfc160f516912f0158f9
    Signed-off-by: Bogdan Dobrelya <email address hidden>

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/496156

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/463701
Committed: https://git.openstack.org/cgit/openstack/tripleo-quickstart/commit/?id=fdaa9748e6dd6a3372b63ec8b0d31c6ccd89fa69
Submitter: Jenkins
Branch: master

commit fdaa9748e6dd6a3372b63ec8b0d31c6ccd89fa69
Author: Bogdan Dobrelya <email address hidden>
Date: Tue May 9 11:48:43 2017 +0200

    Add support for openstack inventory provider

    Add support for openstack hosted pre-provisioned undercloud and
    overcloud cluster nodes, identified by a clusterid metadata
    (optional). Adapt the tripleo-inventory role for that need. Access overcloud nodes over a particular private admin/control network
    (multiple networks supported) via the undercloud floating IP,
    acting as a bastion node public IP.

    List of changes:
    * Reuse access URL/creds and clouds.yaml template from the
      extras' OVB roles' defaults.
    * Split inventory generation tasks into the openstack provider
      case and remaining all/multinode/undercloud cases.
    * Add var overcloud_user (defaults to heat-admin), overcloud_key,
      undercloud_key instead of hardcoded values.
    * Rely on shade dynamic inventory and update requirements.txt for os-
      client-config and jmespath as well.
      for its dependencies to become the part of quickstart deps.
    * Add docs

    Related-bug: #1691467
    Change-Id: If2aef32a5df6fb407605a717523c5fcdec338534
    Signed-off-by: Bogdan Dobrelya <email address hidden>

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/496156
Committed: https://git.openstack.org/cgit/openstack/tripleo-quickstart/commit/?id=0156bcb74817f30b6a22ae578017a32e56b2a1d3
Submitter: Jenkins
Branch: master

commit 0156bcb74817f30b6a22ae578017a32e56b2a1d3
Author: Bogdan Dobrelya <email address hidden>
Date: Tue Aug 22 10:38:04 2017 +0200

    Add a note for the experimental openstack inventory

    Also add a release notes for Pike.

    Related-bug: #1691467

    Change-Id: I9803091c67f53d6b70ed2aa5556ee7edf75f909a
    Signed-off-by: Bogdan Dobrelya <email address hidden>

[Bogdan Dobrelya (bogdando)]

AFAICT, this was implemented for Pike (experimental undercloud deploy command related)

[Bogdan Dobrelya (bogdando)]

The follow up item for Queens is docs updated related mostly. Will be tracked as a separate.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/465047
Committed: https://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/commit/?id=36569be28d67fd130c7ac79d859f83ace630c672
Submitter: Zuul
Branch: master

commit 36569be28d67fd130c7ac79d859f83ace630c672
Author: Bogdan Dobrelya <email address hidden>
Date: Tue Mar 28 15:42:52 2017 +0200

    Unify custom t-h-t install steps for UC/OC/upgrade

    Allow undercloud installations/upgrades from custom t-h-t, similarly to
    overcloud_templates_* et al.

    Unify and share custom t-h-t install script for OC and UC and mixed
    update/upgrade cases via new shared role tht-prep-config.

    To diverge undercloud from overcloud install/update/upgrade cases, use
    the following shared mappings (vars on the right correspond to UC
    cases):

      * tht_templates_path: undercloud_templates_path (defaults to RPM dir)
      * tht_templates_repo: undercloud_templates_repo
      * tht_templates_branch: undercloud_templates_branch
      * tht_templates_refspec: undercloud_templates_refspec

    and a 2nd group (defaults set to do nothing, but only disable the
    t-h-t prepare script's code blocks under control of those):

      * composable_scenario: undercloud_composable_scenario
      * upgrade_composable_scenario: undercloud_upgrade_composable_scenario
      * prep_post_hook_script: undercloud_prep_post_hook_script
      * download_templates_rpm: download_undercloud_templates_rpm
      * tht_rpm_url: undercloud_tht_rpm_url

    Those in group 2 may be customized to mimic the overcloud mixed
    upgrades logic for underclouds/all-in-one for future use.

    And for overclouds respectively, use mappings:

      * composable_scenario: remains unchanged
      * upgrade_composable_scenario: remains unchanged
      * prep_post_hook_script: overcloud_prep_post_hook_script
      * download_templates_rpm: download_overcloud_templates_rpm
      * tht_rpm_url: remains unchanged
      * tht_templates_path: overcloud_templates_path
      ...

    Related-bug: #1691467
    Closes-bug: #1781227
    Change-Id: I4e0dcf24b93b8e4afb8359cb5108c4b7b5ba3d55
    Signed-off-by: Bogdan Dobrelya <email address hidden>