Tech Debt: Convert bash scripts in contrib into python code and add tests

[Julien Vey]

Our bash scripts in contrib should be integrated in the python tree, and we should add proper tests for this new code

[Angus Salkeld (asalkeld)]

In the case we are using a job runner like jenkins, it's handy to have them as a bash script.

[Julien Vey (vey-julien)]

Ed, are you currently working on this or can I take the issue ?

[Ed Cranford (ed--cranford)]

I've been nose-to-grindstone on our M2 Demo, please take this from me.

[Ravi Sankar Penta (ravips)]

I did some work to convert shell code to python, mostly code around app workflow (clone/build/test/deploy). While doing this, I tried to:
- Abstract clear interfaces between solum worker and the language pack handlers.
- Wrap common functionality (environment setup for app build, etc.) in the base language pack plugin.
   Woker loads language pack plugins selected by the operator.
- Simplify passing application info from assembly hander to worker.

Relevant patches(not merged):
https://review.openstack.org/#/c/132381/
https://review.openstack.org/#/c/132718/
https://review.openstack.org/#/c/132933/
https://review.openstack.org/#/c/133111/

At some point, I had working version of these patches with cedarish-docker plugin. A lot has changed after that and I don't have enough cycles to finish this task. Anyone who picks this task needs to do:
- Rebase latest code and resolve conflicts
- Incorporate the feedback got from the team for the above patches
- Add/Update tests to validate the functionality
- Fix any issues

As this task touches a lot of functionality activitly improved by other team meembers, we have few options to get this into master:
Option1: Make the new code co-exist with the existing shell code and switch one language pack plugin at a time to use the python code. Once all LPs are switched to python code, we can safely delete the shell code.
Option2: Replace shell code with these pathces ensuring at least one working LP plugin (mostly dockerfile LP). Gradually add support/fix other LP plugins to use python.

[Adrian Otto (aotto)]

 Akshay Chhajed, feel free to re-claim this bug if you'd like to work on it.

[Akshay Chhajed (akshaychhajed)]

OK. Thanks Adrian!

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/181181
Committed: https://git.openstack.org/cgit/openstack/solum/commit/?id=24649a7e24c22821b18c77a425e57a9f0ba73a86
Submitter: Jenkins
Branch: master

commit 24649a7e24c22821b18c77a425e57a9f0ba73a86
Author: James Li <email address hidden>
Date: Thu May 7 22:29:58 2015 +0000

    Convert bash scripts to Python

    This patch sets up a LP handler framework to accommodate
    build-lp, unittest-app and build-app functionalities.

    Additionally, the patch also implements isolation improvements in
    building an application container on worker node.
    Specifically, the git clone command for cloning application code
    is run inside a docker container. This helps in constraining
    the resources of memory, CPU, disk and network bandwidth
    to prevent Solum from the DoS attacks of a malicious git server.

    Partially-fixes-bug: #1302552
    Change-Id: Ie0f700c0d4bd74e24ac1a9727f4b17907f88cca1

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/192928
Committed: https://git.openstack.org/cgit/openstack/solum/commit/?id=962857598b0010e065ef2fc9fd39dfcfffda1bc7
Submitter: Jenkins
Branch: master

commit 962857598b0010e065ef2fc9fd39dfcfffda1bc7
Author: James Li <email address hidden>
Date: Wed Jun 17 21:28:54 2015 +0000

    Convert unittest-app bash script to python

    1. Added timeout for running user's arbitrary unittest script;
    2. Limit memory usage of a container in which user's unittest
    script runs.

    Partially-fixes-bug: #1302552
    Change-Id: Ibd0fe3b09295839e7f8279cddabb56442c1d87c8

[OpenStack Infra (hudson-openstack)]

Change abandoned by James Li (<email address hidden>) on branch: master
Review: https://review.openstack.org/196498

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/283370

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/283370
Committed: https://git.openstack.org/cgit/openstack/solum/commit/?id=fc58f9422ff3a205fb3f7a37caf49342f4cc57f4
Submitter: Jenkins
Branch: master

commit fc58f9422ff3a205fb3f7a37caf49342f4cc57f4
Author: James Li <email address hidden>
Date: Tue Feb 23 03:30:33 2016 +0000

    Convert build-app and a new worker handler

    Change-Id: Ief90ef40e2f21db6345316e3125f198131d2b034
    Partially-fixes-bug: #1302552